IELE
TKANSACTIONS
ON
INFORMATION THEORY,
VOL.
37.
NO.
I,
JANUARY
1991
Capacity
of
the Gaussian Arbitrarily
Varying Channel
Imre
Csiszhr and
Prakash
Narayan,
Member,
ZEEE
Abstract
-The Gaussian arbitrarily varying channel with input con-
straint
r
and state constraint
2
admits input sequences
x
=
(xl,
,
x,~)
of real numbers with
Cxf
5
nT
and state sequences
s
=
(5
,.
,
T,~)
of
real numbers with
LY~
5
HA;
the output sequence
x
+
s
+
V,
where
V
=
(
VI,
,
y,)
is a sequence of independent and identically distributed
Gaussian random variables with mean
0
and variance
c’.
It is proved
that the capacity of this arbitrarily varying channel for deterministic
codes and the average probability of error criterion equals
log(l+
r/(
2
t
c2))
if
Z
<
r
and is
0
otherwise.
Index
Terms
-Arbitrarily varying channel, Gaussian, capacity.
I.
INTRODUCTION
RBITRARILY varying channels (AVC’s) were intro-
A
duced by Blackwell
et
al. [5]
to
model communication
channels with unknown parameters that may vary with time
in an arbitrary and unknown manner during the transmission
of a codeword. In this paper, attention is restricted to AVC’s
without memory; further, it is assumed that the sequence of
channel states is selected arbitrarily subject to a constraint
specified later, and possibly depending on the codebook
but
independently of the codeword actually sent.
AVC’s exhibit various mathematical complexities even in
the case of discrete alphabets (cf. Csiszir-Korner [6, Section
2.61). In particular, their capacity may depend on whether or
not random codes are permitted, and whether the average or
maximum probability of error criterion is used. The random
coding capacity admits a simple characterization as a min-max
of mutual information,
a result dating back to Blackwell
et
al.
[5].
In contrast, the problem of capacity
for
determinis-
tic codes is much harder. In particular, for the maximum
probability of error criterion, a single-letter capacity formula
is known only under certain conditions on the structure of
the AVC (cf. Ahlswede [2] and Csiszir-Korner
[7]).
Unless stated otherwise, the term capacity will hereafter
always refer to capacity for deterministic codes and the ui‘er-
age probability
of
error
criterion. In the absence of state
constraints, Ahlswede
[l]
proved that this capacity was either
equal to the random coding capacity or otherwise
to
zero.
The necessary and sufficient condition for positive capacity,
Manuscript received April
25.
1989;
revised February
13,
1990.
I.
Csiszir was supported by the Hungarian National Foundation Scientific
Research Grant
No.
1806.
P.
Narayan was supported by the Systems
Research Center at the University
of
Maryland under NSF Grant
OIR-85-00108. This work was presented at the
IEEE
International
Symposium on Information Theory, San Diego, CA, January
14-19,
1990.
I.
Csiszir
is
with the Mathematical Institute
of
the Hungarian Academy
of
Sciences. H-1364 Budapest.
POB
127,
Hungary.
P.
Narayan is with the Electrical Engineering Department and the
Systems Research Center, University
of
Maryland, College Park.
MD
20742.
IEEE
Log Number
9038858.
as well as capacity under a state constraint, have been
determined by Csiszhr-Narayan
[8];
it was further shown
that Ahlswede’s alternatives do not necessarily obtain under
a state constraint.
Less attention has been bestowed in the literature on the
capacity of AVC’s with continuous alphabets. Presumably
motivated by random coding capacity, there have been
game-theoretic considerations concentrating on the min-max
of mutual information (cf. McEliece
[l
11). Hughes-Narayan
[lo] have used a geometric approach to determine the ran-
dom coding capacity of the Gaussian AVC defined formally
in the following paragraph. Blachman [4] has provided lower
and upper bounds on capacity in a communication situation
differing from ours in that the interference (i.e., state se-
quence) could depend on the actual codeword transmitted.
Our incomplete understanding of his paper seems to indicate
that he,
too,
considered random coding capacity.
To
our
knowledge, Ahlswede’s [3] is the only paper treating the
capacity of a continuous alphabet AVC for deterministic
codes. His AVC (a Gaussian channel with the noise variance
arbitrarily varying but not exceeding a given bound) allowed
a very simple approach, which may not be extendable
to
other cases of interest.
In this paper, we determine the capacity of the Gaussian
AVC formally defined
as
follows. Let the input and output
alphabets, and the set of states, be the real line. For any
input sequence
x
=
(x,;
.
.,
x,i)
and state sequence
s
=
(si;..,
s,?),
let the
output
be
x
+
s
+
V,
where
V=
(V,;
.
.,
V,,)
is a sequence of independent and identically
distributed (i.i.d.1 Gaussian random variables with mean
0
and variance
(T’.
We adopt an input constraint
r
and state
constraint
A,
namely the permissible input sequences
of
length
n
are those satisfying
n
11x11~~
C~,21nr,
(no)
(1.1)
r=l
and the permissible state sequences are those satisfying
n
Ilsll’=
zs,?~nA,
(A>O).
(1.2)
i=l
A code of block-length
n
comprises a set of codewords
xi;.
’,x,,,,
each in
R“,
and a decoder
cp:
R”
+{O;..,
N).
The average probability of error of this code, used on the
Gaussian AVC as above when the state sequence is
s,
equals
1.
F(s)=-
Pr{cp(x,+s+V)#i}. (1.3)
N,=I
0018-9448/01/0100-0018$01.00
U
1991
IEEE
CSISZAR AND NARAYAN: CAPACITY
OF
THE GAUSSIAN ARBITRARILY VARYING CHANNEL
I
‘j
The capacity C of the Gaussian AVC with input constraint
r
and state constraint
A
is the largest number with the
property that for every
6
>
0
and sufficiently large n, there
exist codes with
N
2
exp{n(C
-
a)}
codewords, each satisfy-
ing
(l.l),
such that the supremum
of
F(s)
subject
to
(1.2)
converges to
0
as n
+m.
Our main result is the following.
Theorem
1:
The capacity of the Gaussian AVC with in-
put constraint
r
and state constraint
A
is
if
riil
According to Hughes-Narayan
[lo],
the random coding
capacity of the Gaussian AVC equals
f
log(l+
r/(A
+
u2)).
Thus, in this case Ahlswede’s alternatives do obtain. Yet a
proof of the theorem above by the elegant “elimination
technique” of Ahlswede
[1]
is not apparent. Rather, we shall
use the straightforward but more computational method of
CsiszAr-Narayan [SI. Suitable approximation arguments
would enable a derivation of our theorem directly from the
results of
[SI.
Instead, we prefer to present a more transpar-
ent and direct proof, which will also serve to keep this paper
self-contained.
We also determine the capacity of the noiseless additive
AVC whose output is
r+s
rather than
r+s
+V.
The
capacity of this AVC is defined similarly to that of the
Gaussian AVC with the exception that
(1.3)
is now replaced
by
1
N
F(s)
=
-I(i:
cp(x,
+
s)
#
i}I.
Theorem
2:
The capacity of the noiseless additive AVC
with input constraint
r
and state constraint
A
is
if
TsA.
Whereas this result is not a formal special case of Theo-
rem 1, both theorems can be proved by the same method.
We shall prove the simpler Theorem
2
first
so
that the
reader may better understand the key ideas. Observe that
Theorem
1
requires a separate proof only in the case
A
+
u2
2
r.
In fact, since
(1.2)
implies for an arbitrary
E
>
0
that
11s
+
VJ12
I
n(A
+
u2
+
E)
with probability arbitrarily close to
1
if n is sufficiently large, in the case
A
+
u2
<
the
assertion of Theorem
1
follows immediately from that of
Theorem
2.
Actually, we shall show that the capacity as claimed in
Theorems
1
and
2
can be achieved using the minimum-dis-
tance decoder, namely
(1
5)
if
II~
-
.rill2
<
~ly
-
xjl12,
if
no such
1
I
i
I
N
exists.
for
j
+
i
d(Y)
=
{
i,
0,
It is worth pointing out that the result of Theorem
2
with this
decoder provides a solution to a weakened version of the
unsolved sphere-packing problem. This problem seeks the
exponential rate of the maximum number of nonintersecting
spheres of radius
JT?
in
R”
with centers in a sphere of radius
n.
In our case, the spheres may intersect but for any given
s
in
R”
of norm
I&,
only for a vanishingly small fraction
of sphere centers
xi
can
xi
+
s
be closer to another sphere
center than to
xi.
The number
C
in Theorem
2
then gives
the exponential rate of the maximum number of spheres
satisfying this condition. A similar weakened version of the
sphere-packing problem in Hamming space was solved in
[8]
as a special case of the coding theorem for the binary adder
AVC.
11.
PROOF
OF
THE
MAIN
RESULT
The proof
of
the converse parts of Theorems 1 and
2
being standard, is relegated to the Appendix. The essential
contribution
of
this paper consists in the direct part of
coding Theorems
1
and
2.
Our goal is to show that, when
r
>
A,
for all sufficiently
large n there exist
N
=
exp(nR) codewords
xI;
.
.,
x,,,
in
R“
satisfying
Ilx,l12
I
nT,
i
=
1;
.
.,
N,
with
R
arbitrarily close to
the asserted capacity value, such that for a suitable decoder
cp
the average probability
of
error
F(s)
is arbitrarily small
uniformly subject to
llsl12
5
nA.
Using the minimum distance decoder
4
of
(1.5)
for the
noiseless AVC,
(1.4)
becomes
and for the Gaussian case,
(1.3)
gives
iN
Xjl12
I
11s
+
VI12,
for some
j
z
i}.
We can assume without any loss of generality that
(2.2)
=
1,
0
<
A
<
1.
Further,
(2.1)
and
(2.2)
remain unchanged
if
all
vectors are multiplied by
l/&.
Hence it suffices to prove
that for every sufficiently small
6
>
0
and sufficiently large n
there exist
N
=
exp(nR) unit vectors
xI;.
.,xN
in
R“
with
C
-26
<
R
<
C
-
6
where
C
=
f
log(1
+
l/A)
for the noise-
less AVC and C
=
+
log(1
+
l/(A
+
u2))
for the Gaussian
AVC, such that
F’(s)
is arbitrarily small, uniformly subject to
lls1I2
I
A,
where
in the noiseless case, and
for some
j
#
i}
(2.4)
in the Gaussian case where
V=
(VI;
..,V,,)
is now a se-
quence of i.i.d. Gaussian random variables with mean
0
and
variance u2/n.
We claim that the unit vectors
xI;
.
.,
xN
of the following
Lemma
1
do possess the property above
if
7
and
E
are
sufficiently small.
Lemma
1
(Codeword Properties): For every
E
>
0,
86
<
q
<
1,
K
>
2~
and
N
=
exp(nR) with
2~
I
RI
K,
for n
2
n,,(E,q,K) there exist unit vectors
x,;
“,xN
in
R”
such that
for every unit vector
U
in
R”
and constants
CY,
p
in [0,1], we
20
IEEE
TRANSACTIONS ON INFORMATION THEORY.
VOL.
37.
NO.
I,
JANUARY
I991
Here
(.;)
denotes inner product and
1.1’
denotes “the
positive part
of.”
This lemma is an analog of the key Lemma
3
of
[SI,
and can be proved similarly. The proof is in the
Appendix.
Commencing with the noiseless case, in order to bound
(2.3)
for
lls112
I
A,
note that
Ilx;
+
s
-
x,l12
=
11x,1I2
+
1lS1l2 +
llXjl12
Hence
1
N
t?’(s)=-({i:
(x,,s)+(x,,x,)2
I+(x,,s),
1
-N
<
-
1
{
i
:
(
x;
,
s
)
I
-
77)
I
for some
j
f
i}
To
complete the proof of the direct part
of
Theorem
2,
it
suffices to check for every
(a,
p)
=
(1 -277,O)
and
(ak,
1
-
27
-
ak)/fi),
k
=
1;.
.,
K,
the condition
a’
+
p2
>
1
+
7
-
exp(-2R)
of
2)
of
Lemma
1.
(The condition
a
2
7
is clearly
satisfied provided
7
<
min{+,(l-
6)/2}.)
Differentiation
shows that
a2
+(1-27
-
a)’/A
is minimized by
a
=
(1
-
277)/1+
A,
and the minimum equals
(1
-2~)~/
1
+
A.
Thus,
the condition to be satisfied is
1
N
Obviously, if C
-26
<
R
<
C
-
6
for any fixed
6
>
0,
where
+
-
({i:
(x,,s)
+(xi,
xi)
>
1
-
7,
for some
j
z
i)
1.
1
1
c=-log
1+-
=--log
1--
(2.6) 2
(
i)
2
(
1;Aj’
The first term of the sum in
(2.6)
can be bounded by Lemma
I(i). In fact, letting
U
be the unit vector such that
(x,.
s)
I
-
77
implies by the assumption
A
<
1
that
(x,,
U)
I
-
7.
Thus
if
R
>
-
log(1-
v2),
we get that
s
exp
{n(e
-
z))
-+
0,
as
n
-+W.
(2.8)
The second term
of
the sum in
(2.6)
can be bounded using
2)
of Lemma
1
by suitably partitioning the set
of
possible values
of
the inner product
(x,,x,).
To
this end, let
al
=
1
-
77
-fi<a2<
...
<a,=1-2~,
with
ar+I-ak~q,
k=
1;
.
.,
K
-
1.
Then
(x,,
s)+
(x,, x,)
>
1
-
77
implies that
(x,, x,)
1
aI,
and
if
aL
I
(xI,
x,)
I
+
I
then necessarily
(x,,
s)
>
1-27
-
ah.
The latter, in turn, implies by
(2.7)
that
the inequality
(2.9)
will be satisfied if
77
is sufficiently small.
The proof for the Gaussian case (Theorem
1)
is similar but
bounding
(2.4)
is not as easy. We first present two simple
technical lemmas.
Lemma
2:
Let the r.v.
U
be uniformly distributed on the
unit n-sphere. Then for every vector
U
on this sphere and
any
0
<
a
<
1,
we have
1
01
-
1)/2
~r{l(~,u)l>
a)
I
2(1-
a2)
,
if
a
2
-
fi’
Proof:
Denote the angle between the unit vectors
U
and
U
by O(U,u). Then by Shannon
[12, (28)1,
With
a
=
cos$, it follows that
1
ifa2-
+zz
The proof is completed by observing that Pr{(U,
U)
I
-
a}
=
Lemma
3:
Let
U
and
U
be unit vectors with I(u,u)~~
7.
of
x
orthogo-
Pr{(U,
U)
2
a).
U
Then for any unit vector
x,
the component
x
nal to span
{U,
v)
has norm
IIX~II*I
1-(u,x)2-(U,x)2+47.
(2.10)
~
CSlSZAR
AND NAKAYAN: CAPACITY
OF
THE
GAUSSIAN ARBITRARILY VARYING CHANNEL
-
21
Further, for any pair of constants
a,p,
Ilau
+
pull2
I
(
a2
+
p2)(1
+
77).
(2.11)
Proof:
Let
u'=(u
-(u,u)u)/IIu
-(u,u)ull
be the unit
vector orthogonal to
U
such that span(u,u'}
=
span(u,u).
Then
I(u,x)
-(u,u)(u,x)
1
IIU
-(w)ull
I(u,x)
I
-
71
1+71
2
I(o',x)l=
2(1(~4-77)(1-77)
2
I(u,x)I
-277.
Since
J)x
)I2
=
1
-(U,
xI2
-(U',
x)',
this implies
(2.10).
Fi-
nally
((au
+
pull2
=
a2
+
p2
+2ap(
u,u)
I
(a2+
P')(l+
771,
as
12ap/(a2
+
p2)I
I
1,
thereby proving
(2.11).
Continuing with the proof of the direct part of Theorem
1,
note that on account of
(2.8)
it suffices to consider only those
terms in
(2.4)
for which
I(x,,
u)l
I
77,
where
U
is a unit vector
satisfying
s
=
ullsll.
We shall bound these terms using
IIx,
+
s
+
v-
x,1I2
=
11x,112
+
11s
+
v1t2
+
llx,112
+
2(x,,
s)
+
2(
XI,
-
2(
x,,
x,)
-
2(
x,,
s)
-
2(x,,
v).
(2.12)
Decomposing
x,
and
V
into components in
M,,u
=
spanb,,
U}
and in
Mi*:,
we have
(x,,
v)
=
(xy
U) U)
+
[
xy=,
V".'.')
=(x,,v"'")+(xy",v).
(2.13)
Since
V
=
(VI,.
.
.,
V,,)
is a sequence of i.i.d. Gaussian random
variables with mean
0
and variance
a2/n,
we have as
n
--fm
that
Pr
{I(
x,,~)
I
>
77)
-,
0,
Pr(
IIv"~~II
>
77)
+
o
uniformly in
i
and
U.
This along with
(2.12), (2.13),
implies
that
Pr
{llx,
+
s
+
v
-
x,l12
I
11s
+
v1l2,
for some
j
z
i)
=
Pr
((x,,x,)
+
(x,,s)
+
(xy+c,~)
>
1
+
(x,,~)
+
(x,,
V)
-
(x;"i
U,
v),
for some
j
+
i
1
I
pr
((
xy'.,
V)
>
1
-
377
-
((
x,,
x,)
I
-
l(x,,
s)
1,
for
some
j
#
i
+
E,
(2.14)
)
for all sufficiently large
n,
whenever
Kx,,
u)l
I
17.
uniformly subject to
llsl12
I
A,
it suffices to prove that
-
~r((xy',i,~)>1-377-I(x,,x,)1
Hence, in
order
to show that
Z'(s)
in
(2.4)
goes to
0
1
Kx,,u)I<q
-l(x,,u)lfi,
forsome
j#i)
(2.15)
converges to
0
uniformly for unit vectors
U
E
R",
as
n
*
m.
To this end, we partition the set
of
all possible values of
the inner products
(x,, x,)
and
(x,,
U).
Let
a1
=
0
<
"2
<
'
. .
<aK=l
and
pI=0<p2<
...
<p,=l.with
aA+l-ah4
17,
k=l;..,K-l,
and
/3/+l-p/~~,
I=l;..,L-l.
Fur-
ther let
F,~/
=
{j:
j+
i,
a,,
5
((x,,x,)I
I
ak+l,
PlI
1(x,J4
15
&+I)
and
G
=
{(
k,l):
1
I
k
I
K,
1
I
1
I
L,
ak
2
17,
af+p:>l+~~-exp(-2R)}.
Then the expression
(2.15)
is
1
I
c
$i:W4Il
(k,l)EG
As
the first term above goes to zero uniformly in
U
as
n
+CO
by
2)
of Lemma
1,
it remains to consider only the second
term. Recalling again that
V
=
(V,,
. .
.,
V,)
is an i.i.d. se-
quence
of
N(0,u2/n)
random variables, we note that
Pr(llVl12
>
u2
+
v}
-+
0
as
n
+W.
Therefore, it suffices to
prove that
-
Pr{
U
(IIVII~~~~+~,
1
i:Kx,,~)ls~
(k,l)EG
iEF,A/
where
w
=
(akxj
+
pp)/
IIakxi
+
ppll.
By
Lemma
3
(for
Kxi,u)l
I
q),
Ilakxj
+
ppll
I
J(ai
+
p:)(
1
+
77)
,
so
that
by Lemma
1(
i)
for all
n
sufficiently large, where we can assume that
a:
+p:
<
1+q (2.18)
(as otherwise
F,kl
=
4).
ILtE
TRANSACTIONS ON INFORM,\I ION THFOKY.
VOL 17
NO
I.
JANUARY
I‘NI
Further, by Lemma
3,
if
xi
=
x,’(i,u)
represents the unit
vector in the direction
of
xy
‘1,
for
j
E
F,h/
we have
IIxy
,zII
I
J1-(x,,x,)’-(x,,u)’+477
IJl-a:-pf+477
if
IKx,,
u)l
I
7.
Hence
Pr(llVII’
5
u2
+
77,
(xy’,,,~)
>
1-57
-
ah
-
p,fi>
(2.19)
where
U
is a r.v. distributed uniformly on the unit n-sphere
and
U‘
is any fixed unit vector in
R”.
Together with (2.17),
this implies that (2.16) is overbounded by
C,,,,,,,A‘,;‘,
where
Hence it suffices to show that
A$;)+
0 as
n
+CO
for every
(k,l)
E
G.
Since
(k,
I)
E
G,
there are
two
cases to consider:
a)
a;
+
P:
>
1
+
17
-
exp
(
-
2~),
and
b)
ax.
I
7,
a:
+
p,?
I
1
+
77
-exp(
-2~).
We first observe that in both cases
1-
ah
-p/fi-jV
>
0
(2.21)
provided that
7
is chosen sufficiently small. Indeed, in case
a), the expression in
(2.21)
is
2
1
-
fi
-677. In case b), the
assumption
R
<
C
-
6
=
+
log(1
+
1/(A
+
u2))-
6
implies
that
1
a:
+p:5
1
+
77-
1
exp(26)
l+y
A+u
11
<
1
+
77
-
-
exp(26).
1+A
Since
ah
+
P/fi
5
d(
a;
+
pf)(
1
+
11)
(as can be directly verified by squaring both sides), this yields
1
-a,
-p/fi-577
>
1-57
-
41
-
A(exp(26)
-
1)
+
v(
1
+
A)
>
0
if
77
is sufficiently small.
Now, in case
a)
we obtain, using Lemma
2,
that
if
E
and
77
are chosen small enough.
using Lemma
2
we obtain from (2.20) that
In case
b),
we have
R
+
+
log(1-
ai
-
P:
+
77)
>
0.
Then,
a;
-p;
+
q)+
e)
11
i
(l-a,<-P/fi-577)*
(
v2
+
q)(l-
ai
-
P,?
+47)
1
JI
/
Evaluating the maximum
of
(1
-
a
-pa
-5q)*
2
-
p2
+477
-
y(a,P)=l-a
>
U2fV
we obtain by differentiation that the maximum is attained at
and the value
of
the maximum is
(1
-w2
l+477-
1+
A+u2+v