IEEE TRANSACTIONS ON INFORMATION THEORY, VOL.
IT-27,
NO.
4,
JULY
1981
393
Practical Codes for Photon Communication:
ROBERT J. McELIECE,
MEMBER, IEEE
Abstract-In
a recent paper, Pierce studied the problems of communi-
cating at optical frequencies using photon-counting techniques, and con-
cluded that “at low temperatures we encounter insuperable problems of
encoding long before we approach [channel capacity].” In this paper it is
shown that even assuming a noiseless model for photon communication for
which capacity (measured in nats/photon) is infinite, it is unlikely that a
signaling efficiency of even 10 nats/photon could be achieved practically.
On the ‘positive side, it is shown that pulse-position modulation plus
Reed-Solomon coding yields practical results in the range of 2 to 3
nats/photon.
I.
INTRODUCTION
I
N [14], Pierce argued that if one uses photon-counting
techniques for communication at optical frequencies,
channel capacity is
hf/kT
nats/photon’ where
f
is the
photon center frequency and
T
is the noise temperature
(h
is Plan&s constant,
k
is Boltzmann’s constant). Later
Pierce, Posner, and Rodemich [15] derived the same result
more rigorously. In [14] Pierce also observed that the
techniques of linear amplification (which are used success-
fully at microwave frequencies) yield a capacity of 1
nat/photon. If one has deep-space applications in mind,
these results strongly favor photon-counting techniques.
For example, if
f
= 6
X
lOI Hz (green light) and if
T =
400” K (an average temperature of space at optical fre-
quencies [5]), we find
hf/kT
= 72 nats/photon. However,
channel capacity is an
absolute
limit on performance and
only tells us what is possible using arbitrarily complex
encoding and decoding strategies. This paper is a study of
the “practical” limits of photon communication.
Of course it is a general rule that the closer one ap-
proaches channel capacity, the more complex and costly
the needed coding strategies become. In the case of photon
communication, however, coding problems seem to become
serious much sooner than usual, and for an unexpected
reason. We shall see below that it is not the noise tempera-
ture, but the nature of the photon-counting process itself,
that causes the most serious problems; so that even in the
limiting case
T
= 0, when capacity is in principle infinite,
it seems unlikely that a signalling efficiency of even 10
Manuscript received April 28, 1980; revised September 1, 1980. This
work was supported by the National Aeronautics and Space Administra-
tion under Contract NAS7-100.
The author was with the Jet Propulsion Laboratory, California Institute
of Technology, Pasadena. He is now with the Department of Mathe-
matics, and the Coordinated Science Laboratory, University of Illinois,
Urbana, IL 61801.
‘The unit
nafs/photon
is somewhat unorthodox, but in the present
context seems very natural. Its main advantage is that it is independent of
time, which allows us to avoid questions of absolute bandwidth and power
and to focus on more fundamental physical limitations.
nats/photon could be achieved practically. This is because,
as we will show, when the signalling rate increases beyond
1 nat/photon one encounters an explosive increase in the
required bandwidth expansion. This negative result was
predicted by Pierce [14] who wrote “at low temperatures
we [will] encounter insuperable problems of encoding long
before we approach the theoretical limit of
[hf/kT
nats/photon]“.
On the positive side, however, we will show that with
pulse position modulation combined with Reed- Solomon
coding, it is possible to design a practical photon-counting
system which operates at about 3 nats/photon. Since chan-
nel
capacity
for linear amplification is only 1 nat/photon,
we can conclude from this that photon counting is in fact
significantly superior to linear amplification.
In Section II we present a channel model appropriate for
the study of noiseless photon communication which we call
the photon channel. In Section III we study the use of
q-ary pulse position modulation (q-PPM) on the photon
channel. There we show that q-PPM channel capacity is
log
q
nats/photon, and we give performance curves (error
probability versus signalling efficiency) for coded and un-
coded q-PPM. We conclude by showing that if
p
denotes
the signalling rate in nats/photon, and if p denotes the
minimum required bandwidth expansion, then p 1
ep/p
for PPM. In Section IV we show that this exponential
growth of /3 as a function of
p
is not due to some inherent
weakness of PPM by proving that p > (e”-’ -
1)/p
no
matter what modulation scheme is used. Finally in Section
V, we discuss the &,-parameters involved in photon com-
munication. We show that for q-PPM, R, = 1 - l/q
nats/photon, whereas for the unrestricted photon channel
R,
= 1. If one believes that
R,
is the rate above which
reliable communication becomes extremely difficult, our
claim that
p
= 10 is a “practical” limit even though chan-
nel capacity is infinite, is perhaps less baffling.
II.
THE
PHOTON CHANNEL MODEL
We assume that any photon communication system
works as follows. The time interval during which communi-
cation takes place is divided into many subintervals
(“slots”), each of duration
t,
seconds. The transmitter is a
laser which is pulsed during each time slot; it may be
pulsed with a different intensity in each slot. At the re-
ceiver is a photon counter which accurately counts the
number of photons received during each time slot. We
denote by xi the expected number of photons received
00 18-9448/8 l/0700-0393$00.75
0 198 1 IEEE
394
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT-27, NO.
4,
JULY
1981
during the ith time slot; xi will be called the
intensity
of the
i
th pulse.
It
may be that “noise photons” are present in such a
system, but in many cases of practical interest, noise pho-
tons are extremely rare. (For example, in a careful analysis
of a potentially practical system, Katz [6] estimated the
rate of arrival of noise photons to be around 10 -’ per
second.) In any event we shall make the assumption that
no noise photons exist.2 In this case, because of the Poisson
nature of photon arrivals, the probability that exactly
k
photons will be received during a slot in which the laser
was pulsed with intensity x is e -Xx
k/k !.
Thus we have a discrete memoryless channel with an
input alphabet equal to the set of nonnegative real num-
bers (the possible values for the intensities xi), and output
alphabet equal to the set of nonnegative integers (the
possible outputs of the photon counter). If a real number x
is transmitted, the probability that the integer
k
will be
received is given by
p(klx) = eeX$.
(2.1)
We call the channel described by (2.1) the
photon channel.
A code for this channel is a set of vectors X, =
(Xi,,. . .>
x,,), i = 1; . .)
M, of length
n.
Each component
xjj is a nonnegative real number, and represents an inten-
sity of the transmitting laser. Assuming that each compo-
nent of a codeword requires one time slot for transmission,
the rate of such a code is
R
= log
M/n
nats/slot.3
(2.2)
On the other hand, each component xii represents an
average number of (received) photons, and so the code’s
rate in nats/photon is
p
=
R/p
nats/p hoton, where
(2.3)
p =
i 1
2 xij
/nM,
photons/slot (average).
(2.4)
i,j
The reciprocal of the rate
R
in (2.2) is a measure of
“bandwidth expansion”. If we are transmitting at a rate of
say A nats/s, using a code of rate
R
nats/slot, it follows
that we require
A/R
slots/second. Thus the slot rate is
equal to the nat rate multiplied by the factor l/R. We thus
define
R = l/R = n/log
M
slots/t-rat,
(2.5)
and call
p
the
bandwidth expansion factor.
In the following sections, we will make various informa-
tion-theoretic calculations using this model.
The reader
should bear in mind that since our chief aim is to show
what is not possible, more elaborate models incorporating
external noise sources could only strengthen our conclu-
sions.
‘A
careful information-theoretic analysis of the photon channel when
noise photons are present is given in [ 151.
‘Throughout the paper all logarithms are natural.
III.
PULSE POSITION MODULATION
In [ 141, Pierce suggested the use of pulse position modu-
lation (PPM) for optical communication. In PPM, a fixed
integer
q
L 2 is selected, and the transmission interval is
divided into consecutive blocks of
q
slots each. In each
such block the laser is pulsed in exactly one of the
q
slots at
a fixed intensity h. We regard each of these
q
patterns as a
letter in the sender’s alphabet. For example with
q = 4,
if
we denote “no pulse” by 0 and “pulse” by 1, the letters are
1000, 0100, 0010, 0001. There are, however,
q
+ 1 possibili-
ties for the received letter, because of the possibility that no
photons may be received in a slot in which the laser was
pulsed. This erasure symbol (e.g. 0000 if
q = 4),
is by (2.1)
received with probability
PE
= e -‘. On the other hand, if
each of the
q
letters is sent with probability
q
-‘, each will
carry log
q
nats of information, using an average of A
photons, so the rate of this primitive signalling strategy is
p = (log
q)/A
nats/photon. Hence for uncoded PPM, the
relation between the error probability
PE
and the rate
p
is
pE =
q-U/P)
(3.1)
It follows from (3.1) that for any fixed
p
> 0, and e > 0,
there exists a
q
such that the corresponding PPM system
has rate exceeding
p
nats/photon and error probability less
than e. This shows that the capacity of the photon channel
(measured in nats/photon) is infinite; indeed, this is essen-
tially the argument given by Pierce in [ 141.
As a practical system, uncoded PPM leaves much to be
desired, however. In Fig. 1 we have plotted
PE
versus
p
for
PPM and
q
= 25, 2”, 220. With
q
= 2” for example, we
can achieve
PE
= 10 -’ and
p
= 1.0, but only at the cost of
an enormous bandwidth expansion factor (cf. (2.5)) of
/3 = 2”/20 log 2 = 75639. But we can do much better using
coded
PPM.
In coded PPM, the idea is to regard the
q
letter transmis-
sion alphabet as the input alphabet of a discrete memory-
less channel with
q
+ 1 output letters. The
(q
+ 1)st letter
(symbolically 00000) is regarded as an erasure symbol; thus
the photon channel of Section II, combined with q-ary
PPM becomes a q-ary erasure channel with erasure proba-
bility e -‘. The capacity of this channel, which is achieved
by a uniform probability distribution on the input al-
phabet, is (1 - e -“) log
q
nats/letter. Since each letter
requires an average of X photons, the channel capacity
measured in nats/photon is
C(q,
X) = ’ -{-’ logqnats/photon. (3.2)
If
q
is fixed, the supremum of
C(q, h)
over A > 0 occurs as
A + 0, and is
C(q) = log
q
nats/photon.
(3.3)
What (3.3) says is that if q-PPM is used, then for small
error probability the largest possible value of
p
(cf. (2.3)),
in the limit of arbitrarily complex coding, is log
q
nats/photon.
But what can be achieved practically? We have found
that if
q
is a power of two,
Reed-Solomon (RS) codes,
MCELIECE: PRACTICAL CODES FOR PHOTON COMMUNICATION
395
NATS,‘PHOTON
Fig. 1. Performance of uncoded PPM.
NATS/PHOTON
which are extremely efficient at correcting erasures, give
good performance. An (n, k) Reed-Solomon code with
symbol alphabet GF(
q),
and n =
q
- 1, can correct any
pattern of up to n -
k
erasures. Furthermore, when q is a
power of two, very efficient encoding and decoding proce-
dures exist; indeed Berlekamp [2] has described a hardware
implementation of a
q
= 256 RS decoder which operates at
40 Mbits/s.
If we use an (n,
k)
RS code for the present application,
each of the
qk
codewords carries klog
q
nats of informa-
tion, and each codeword requires n pulses. Thus if we are
transmitting p nats/photon, the average number of pho-
tons/pulse is
,+klong9
photons/pulse.
(3.4)
It follows that the erasure probability for the correspond-
ing q-ary erasure channel is
c = e-“= ~=VP,
(3.5)
where R =
k/n
is the rate of the RS code. Since the RS
code can correct all patterns of up to n -
k
erasures, the
decoding error probability
PE
satisfies
where c is given in (3.5). In Fig. 2 we have plotted this
bound on
PE
versus p for five typical RS codes. The curve
labelled
q
= 16 is for a (15,s) RS code with
q =
16. The
others are (31,16), q = 32; (63,32),
q = 64;
(127,64), q =
128; and (255,128),
q
= 256. (Recall that as codes for the
photon channel, the length is actually n = 16% 15 = 240 for
the
q =
16 code;
n =
31.32 = 992 for
q = 32; n = 4032
for
q
= 64; n = 16256 for
q
= 128; and n = 65280 for
q
= 256.) Each of these codes is the best RS code of its
length, at least in the limit as p + 0, and so no significant
improvement is possible merely by altering the code rate
k/n.
We see by comparing Figs. 1 and 2 that, for example, at
PE
= 10 -’ coded PPM with
q = 32
works as well as un-
coded PPM with
q = 2
*’ This represents an enormous
.
reduction in bandwidth exnansion (from 220/10g(220) =
Fig. 2.
Performance of Reed-Solomon coded PPM
75639 down to (32/log(32))
X
(31/16) = 18) at only a
modest increase in receiver complexity.
On the other hand, by extrapolation we can see from
Fig. 2 that even using coded PPM, one needs a very large
q
to obtain say
PE
= 10 -6 at p = 5. More generally, if q-ary
PPM is used, each of the q input letters carries at most
log
q
nats of information, so the bandwidth expansion j3
must be 2 q/log
q.
However, from (3.3), p < log q, and so
for p > 1, we must have
(3.7)
Thus if PPM is used, the bandwidth occupancy must grow
exponentially with p. In the next section we will see that
any communication strategy for the photon channel will
encounter similar difficulties.
IV. A
NEGATIVE RESULT
In the last section we saw that PPM forces an exponen-
tial increase in bandwidth expansion as a function of
p.
We
now show that any reliable coded communication system
for the photon channel must encounter similar difficulties,
viz. :
13’ “7 l.
(4.1)
To prove (4. l), we return to the photon channel model of
Section II, and consider the mutual information 1(X, K),
where X is a nonnegative random variable and
K
is a
nonnegative integer-valued random variable related to X
by the conditional probabilities (2.1). We now define
C(p) = sup{I(X;
K): E(X) = ,u}.
(4.2)
According to Shannon’s noisy-channel coding theorem (see
[4, ch. 7]), C(p) represents the maximum possible rate (in
nats per slot) of a reliable communication system which is
restricted to operate at an average of p photons per slot. By
a well-known inequality (see e.g. [9, ch. l]),
1(X; K) I H(K),
(4.3)
where H(K) denotes the entropy Zp, log
pk
of the random
variable
K.
Since for the nhoton channel
ECK I X) = X,
it
396
follows that
E(K)
=
E( E( K
1 X)) =
E(X),
and so
K
has
the same mean as X, viz., p.
The problem of maximizing the entropy of a nonnega-
tive integer-valued random variable with given mean was
solved by Stern [17], indeed in essentially this context.
(Stern’s problem was that of finding the maximum-entropy
photon source, given an average-power constraint.) The
result is
H(K)Ilog(l+p)+plog l+; ,
i 1
with equality if and only if Pr{K =
k} =
(1 -
p)pk, p =
p/(1 + p). Thus from (4.2) we have the estimate
1
c(p) I log(1 + p) + plog 1 + - .
( i P
(4.4)
It follows then from (4.4) and the converse to the noisy-
channel coding theorem, [4, th. 7.3.11, that the rate
R
of a
reliable communication system which operates at an aver-
age of p photons per slot is bounded by the right side of
(4.4). Using the inequality log (1 + p) I p, we have
R <
~(1 + log(1 + l/p)).
(4.5)
The rate
R
in (4.5) is in nats per slot. The rate measured in
nats/photon is by (2.3)
R/p,
and so
p < 1 + log(1 + l/p).
(4.6)
For p > 1 a simple manipulation of (4.6) yields
w4e
P-l- 1
)-I.
(4.7)
Now since the bound on the right side of (4.5) is an
increasing function of p, it follows from (4.5) and (4.7) that
Rc '
eP-lL 1 ’
which proves (4.1) since
R = /3 - '.
Equation (4.1) implies that one encounters an explosive
increase in the required bandwidth expansion beyond
p =
1. In the next section we will show that the R,-parameter
for the photon channel is
p.
= 1 nat/photon.
Thus for the photon channel, (4.1) gives rigorous
mathematical substantiation to the widely believed
“Ro-
conjecture”,
which is that for any channel
R,
is the rate
above which the implementation of reliable communication
becomes very difficult. Conversely, if one believes the
R,-conjecture, our claim that
p
= 10 is perhaps the ulti-
mate limit of a practical photon communication system,
even when channel capacity is infinite, may appear less
baffling.
IV.
THE ~~~~~~~~~~~ .
In this section we will show that the R,-parameter for
the photon channel of Section II is 1 nat/photon. We will
also show that if q-PPM is being used,
R,
is
(q -
1)/q
nats/photon. Thus although the capacity of q-PPM is
infinitely far removed from the capacity of the unrestricted
photon channel,
R,
for PPM is very close to the unre-
stricted
R,
for even small values of
q.
This result perhaps
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL.
IT-21, NO. 4, JULY 1981
justifies our feeling that there is no essential loss of perfor-
mance involved when PPM is used. (This feeling is rein-
forced by the results of Snyder and Rhodes [ 161 which
imply that among all modulation schemes using
q
letters,
q-PPM gives the largest possible value for
R,.)
Recall the definition of
R,
for a time-discrete memory-
less channel. Let A denote the input alphabet and B the
output alphabet, which we assume to be finite or count-
able. For x E A, y E B, denote by
p(y
Ix) the probability
that y will be received given that x is transmitted. For each
pair of input letters x,, x2,
define the Bhattacharyya dis-
tance between them as
43h%) = -1% x \lPblX,)P(Yl4. (5.1)
YEB
If X is a random variable taking values in the set A, and if
X,, X, are independent random variables with the same
distribution as X, define
R,(X) =
-log E(exp - d,(X,, X2)).
(5 4
Finally, the quantity
R,
is defined as
R, = sup R,( X),
(5 -3)
X
the supremum in (5.3) being taken over all possible ran-
dom variables X taking values in the set A.
Since the function f(t) = e et is convex upwards, it
follows from Jensen’s inequality [9, appendix B] that E(exp
- d) 1 exp -
E(d),
and hence from (5.2) that
R,(x) 5 E&& x,>),
(5 4
R,s supE(d,(X,, X,)).
X
(5.9
These two inequalities prove to be very useful in estimating
R,
in specific cases, as we will see below.
We consider the photon channel with q-PPM first. Here
]A] =
q,
B
=
A
U
{?}, where “?” is the erasure symbol,
and the channel transition probabilities are given by
(1 -e-“, ify = x,
P(YlX)= e-",
i 0,
From this we easily compute
tances are given by
ify = ?,
otherwise.
that the Bhattacharyya dis-
dB(x,, x2) =
0,
ifx1=x2,
A,
if x, # x2.
(5.6)
Hence by (5.4) we have, for any random variable X,
R,(X) 5 X.Pr{X, # X2},
(5 *7)
the units in (5.7) being nats/letter. If we denote the
probability Pr{ X = x} by
p(x),
then
W,#X*l = 1 -xz~P(x)2
51-i
4’
since by Schwarz’s inequality
@p(x)*
1)2 5 Zp(~)~.xl* =
MCELIECE: PRACTICAL CODES FOR PHOTON COMMUNiCATION
q*Ep(x)*.
Thus from (5.7)
R,(X)
I A 1 - f nats/letter,
i i
(5 4
or since each transmitted letter requires A photons on the
average,
R,
5 1 -.f nats/photon.
(5 *9>
On the other hand, if X is uniformly distributed on the
input alphabet, a simple calculation gives
R,(X)
= -i log(e-” + (1 -
e-‘)/q)
nats/photon.
(5.10)
The limit of (5.10) as h + 0 is easily seen to be
(q -
1)/q,
and so we conclude that
R,
2
(q
- 1)/q. This, combined
with (5.9) shows that for q-PPM,
R,(q)
=
(q
- 1)/q nats/photon. (5.11)
We turn now to the unrestricted photon channel. Here
the input alphabet
A
is the set of nonnegative real num-
bers, and the output alphabet
B
is the set of nonnegative
integers, with the transition probability given by (2.1). The
first step in computing
R,
for this channel is the computa-
tion of the Bhattacharyya distances. According to (5.1) and
(2. I>,
exp -
4-h x2)
= g ~pb+dp(klx2)
k=O
Hence
=e -(XI +xd/2 k;. & ggk
=e
-(x1
+xd/*e*
= exp - (6 - 6)*/Z. (5.12)
dB(-% x2) = (cl - 62)2/2.
(5.13)
Note also that if we only take the term
k =
0 in the sum in
(5.12) we get the estimate
dB(% x2) 5 b, + x2)/2.
(5.14)
It thus follows immediately from (5.14) and the bound
(5.4) that
R,(X)
I
E(X)
nats per slot.
(5.15)
391
In words, (5.15) says that if the average laser intensity is
E(X)
= p photons per slot, then the R,-parameter is at
most p nats per slot. In units of nats/photon then, it
follows from (5.15) that
R,I
1 nat/photon. (5.16)
But we have seen in (5.11) that
R,(q)
= 1 - l/q. Thus by
taking
q
sufficiently large,
R,
can be made as close to one
as desired. This fact combined with (5.16) shows that
R,
= 1, as claimed.
ACKNOWLEDGMENT
The results in this paper have been considerably in-
fluenced by the work of several other researchers at the Jet
Propulsion Laboratory, including most especially Joseph
Katz, Richard Lipes, Eugene Rodemich, Arthur Rubin,
and Lloyd Welch. Indeed, many of the results in this paper
have already appeared in [lo]-[13]. I would also like to
point out that in [ 121, it is shown that as
p
increases above
one the needed ratio of peak-to-average signal power also
increases exponentially.
REFERENCES
111
PI
[31
]41
[51
[61
]71
PI
[91
[lOI
[Ill
[121
L. D. Baumert, R. J. McEliece, and M. Rumsey, Jr., “Coding for
optical channels,” Jet Propulsion Laboratory Deep Space Network
Progress Reports, Pasadena, CA 91103, vol. 42-49, pp. 70-77,
1978.
E. R. Berlekamp, “The technology of error-correcting codes,”
Proc.
IEEE,
vol. 68, pp. 564-593, 1980.
S. Butman, J. Katz, and J. R. Lesh, “Practical limitations on
noiseless optical channel capacity,” Jet Propulsion Laboratory Deep
Space Network Progress Reports, Pasadena, CA 91103, vol. 42-55,
pp. 12-14, 1979.
R. G. Gallager,
Information
Theoy and Reliable Communicution.
New York: Wiley, 1968.
S. Gulkis, private communication. Excerpt: “By definition, the sky
brightness temperature is that temperature
T
which when used in
the Planck blackbody radiation law yields the observed sky bright-
ness . ‘(Using Fig. i in the paper by Longair and Sunyaev, Soviet
Phvsics. vol. 14. no. 5. March-Anril. 1972) the ontical radiation of
normal’galaxies in the 1~ to 10; range is estimated to be lO-23
Wrn-* Hz-’ sr- ‘_ Taking [this figure for I, and] Y = 3 X lOI
Hz...weobtainT=377K....”
J. Katz, “Comments on the photon channel,” privately circulated
Jet Propulsion Laboratory memo.
R. Lipes, “Pulse position modulation coding as near optimum
utilization of photon-counting channel with bandwidth and power
constraints,” Jet Propulsion Laboratory Deep Space Network Pro-
gress Reports. Pasadena, CA 91103. vol. 42-56. DD. 108- 113.
5. L. Massey, “Coding and modulation in digitaJ’~ommunications,”
in
Proc. Int. Zurich Seminar on Digital Communications, 1914.
R. J. McEliece,
The Theory
of
Information and Coding.
Reading,
MA: Addison- Wesley, 1977.
R. J. McEliece,
“The R,-parameter for optical communication
using photon-counting,” Jet Propulsion Laboratory Deep Space
Network Progress Reports, Pasadena, CA 91103, vol. 42-53, pp.
62-64, 1979.
R. J. McEliece, “Coding for the photon channel,” in
Proc. I979 Nut.
Telecommun. Conf.,
pp. 23.3.1-23.3.3.
R. McEliece, E. Rodemich, and A. Rubin, “Practical limits of
photon communication,” Jet Propulsion Laboratory Deep Space
Network Progress Reports, Pasadena, CA 91103, vol. 42-55, pp.
63-61, 1919.