IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 51, NO. 8, AUGUST 2005
2745
Insufficiency of Linear Coding in Network
Information Flow
Randall Dougherty, Christopher Freiling, and Kenneth Zeger, Fellow, IEEE
Abstract—It is known that every solvable multicast network has
a scalar linear solution over a sufficiently large finite-field alphabet.
It is also known that this result does not generalize to arbitrary
networks. There are several examples in the literature of solvable
networks with no scalar linear solution over any finite field. How-
ever, each example has a linear solution for some vector dimension
greater than one. It has been conjectured that every solvable net-
work has a linear solution over some finite-field alphabet and some
vector dimension. We provide a counterexample to this conjecture.
We also show that if a network has no linear solution over any finite
field, then it has no linear solution over any finite commutative ring
with identity. Our counterexample network has no linear solution
even in the more general algebraic context of modules, which in-
cludes as special cases all finite rings and Abelian groups. Further-
more, we show that the network coding capacity of this network
is strictly greater than the maximum linear coding capacity over
any finite field (exactly 10% greater), so the network is not even
asymptotically linearly solvable. It follows that, even for more gen-
eral versions of linearity such as convolutional coding, filter-bank
coding, or linear time sharing, the network has no linear solution.
Index Terms—Asymptotics, flows, linear coding, network infor-
mation theory, routing.
I. INTRODUCTION
I
N the context of network information theory [1], [18], a net-
work is a directed acyclic multigraph, some of whose nodes
are sources or sinks. Associated with the sources are messages
and associated with the sinks are demands.
1
The demands at
each sink are a subset of all the messages of all the sources. Each
directed edge
in a network carries information from node
to node . The goal is for each sink to deduce its demanded
messages from its in-edges by having information propagate
from the sources through the network. A multicast network is
Manuscript received March 2, 2004; revised January 7, 2005. This work was
supported by the Institute for Defense Analyses, the National Science Founda-
tion, and Ericsson.
R. Dougherty is with the Center for Communications Research, San Diego,
CA 92121-1969 USA (e-mail: rdough@ccrwest.org).
C. Freiling is with the Department of Mathematics, California State Uni-
versity, San Bernardino, San Bernardino, CA 92407-2397 USA (e-mail:
cfreilin@csusb.edu).
K. Zeger is with the Department of Electrical and Computer Engineering,
University of California, San Diego, La Jolla, CA 92093-0407 USA (e-mail:
zeger@ucsd.edu).
Communicated by M. Médard, Associate Editor for Communications.
Digital Object Identifier 10.1109/TIT.2005.851744
1
Here we use the terms “source” and “sink” in the graph-theoretic sense,
namely, nodes with no in-edges and no out-edges, respectively. More gener-
ally, a network can have messages and demands associated with non-sources
and non-sinks, respectively, but such a network can always be converted into a
network satisfying our given definition, without altering the network solvability
properties. Also, some definitions of a network allow directed cycles.
a network with exactly one source and such that each sink de-
mands all of the source’s messages.
A network’s messages are assumed to be arbitrary elements
of a fixed finite alphabet. At any node in the network, each
out-edge carries an alphabet symbol which is a function (called
an edge function) of the symbols carried on the in-edges to the
node, or a function of the node’s messages if it is a source. Also,
each sink has demand functions for each of its demands, which
attempt to deduce the node’s demands from its inputs. A net-
work code is a collection of edge functions, one for each edge
in the network, and demand functions, one for each demand of
each node in the network. A solution is a code which results in
every sink being able to deduce its demands from its demand
functions, and a network that has a solution is called solvable.
It was noted by Ahlswede, Cai, Li, and Yeung [1] that for some
networks, coding can achieve solutions that are otherwise un-
achievable using only routing or switching.
One way of modeling multiple uses of a network is to view
each network edge as carrying a vector of alphabet symbols. For
a network code using vector transmission, the out-edge of each
node carries a vector of alphabet symbols which is a function
of the vectors carried on the in-edges to the node, or a func-
tion of the node’s message vectors if it is a source. Also, each
source has a vector of messages and each sink demands a subset
of all the source vector messages. All edge vectors are assumed
to have the same dimension
and all message vectors are as-
sumed to have the same dimension
. Note that the definition
of a solution is with respect to the case when
. If there
is a solution with
, the solution is said to be scalar.
For general
and , a code that allows the sink nodes to deduce
their demands is called a
fractional coding solution.
For a network alphabet with an algebraic structure (such as a
ring or field), a fractional coding solution is said to be linear if all
edge functions and all demand functions are linear combinations
of their vector inputs, where the coefficients are matrices over
the alphabet. That is, in a linear solution, if a node has in-edges
and/or source messages carrying vectors
,
then an out-edge of the node carries a vector
where each matrix has elements in the alphabet , and is
of dimension
when is a source message and is of di-
mension
when is an in-edge. A demand function is
linear if it has an identical form as the equation for
, but with
the number of rows in each matrix equal to
instead of .
0018-9448/$20.00 © 2005 IEEE
2746 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 51, NO. 8, AUGUST 2005
The coding capacity of a network with respect to an alphabet
and a class of network codes (e.g., see [2] and a related
definition in [18, p. 339]) is
fractional coding solution in over
If consists of all network codes, then we simply refer to the
above quantity as the coding capacity of the network with re-
spect to
. The following result was recently shown in [2].
Lemma I.1: The coding capacity of a network is independent
of the alphabet size.
The linear coding capacity is the coding capacity when
con-
sists of all fractional linear codes. Whereas the coding capacity
of a network is known to be independent of the alphabet size [2],
the linear coding capacity of a network does in general depend
on the alphabet size chosen (e.g., see Theorems IV.3 and IV.4).
We say that a class of network codes is sufficient over a class of
alphabets if every solvable network has a solution in the class
of codes over some member of the alphabet class. A network is
asymptotically solvable with respect to an alphabet and a class
of codes if its coding capacity is at least
. We say that a class
of network codes is asymptotically sufficient over a class of al-
phabets if every solvable network is asymptotically solvable in
the class of codes over some member of the alphabet class.
In this paper, we first show that network linear codes are insuf-
ficient over finite field alphabets (Theorem II.4), and then over
commutative ring alphabets (Corollary III.2), and even over the
general class of alphabets consisting of
-modules
2
(Theorem
III.4). Finally, we show that network linear codes are asymptot-
ically insufficient over finite field alphabets (Corollary IV.6). In-
terestingly, a single network is used to establish all four of these
counterexamples. Also, we compute the exact network coding
capacity and the linear network coding capacity of this network
for any finite-field alphabet (Corollary IV.5). The method used
to obtain the network exploits techniques from the theory of ma-
troids, which we will discuss in a future publication.
Li, Yeung, and Cai [12] showed that any solvable multicast
network has a scalar linear solution over a sufficiently large fi-
nite-field alphabet. Riis [15] noted in particular that every solv-
able multicast network has a linear solution over GF
in some
vector dimension. For multicast networks, there have been var-
ious studies of algorithms for constructing scalar linear codes
as well as the alphabet sizes needed for obtaining scalar linear
solutions [3]–[6], [9]–[12].
For nonmulticast networks, various results have been given.
Riis [15] constructed a network which is solvable over a binary
alphabet, but which has no scalar linear solution over the finite
field GF
, and yet does have a linear solution over GF in
three dimensions. He also demonstrated in [15] solvable net-
works which can achieve linear solutions over GF
only if
the vector dimension grows at least linearly with the number of
nodes in the network.
Rasala Lehman and Lehman [11] gave a collection of net-
works which are solvable, but which have no scalar linear so-
lution over any finite-field alphabet. Médard, Effros, Ho, and
2
An
R
-module is like a vector space, but the coefficients of its vectors come
from a ring
R
rather than from a field.
Karger [13] pointed out that the networks in [11] have linear so-
lutions (based purely on routing) over every finite field in two
dimensions. Similarly, it was noted in [13] that a certain network
given by Koetter has no scalar linear solution but does have a
linear (routing) solution in two dimensions.
It is clear that linear codes in dimensions two and higher
are more powerful than scalar linear codes. Riis stated in
[14]: “Maybe the most important question is whether any flow
problem can be solved using linear coding.” In fact, Médard,
Effros, Ho, and Karger stated in [13]: “We conjecture that
linear coding under its most general definition is sufficient for
network coding in systems with arbitrary demands.” The “most
general definition” of linear coding is not specified in [13], but
some clarification is given by Jaggi, Effros, Ho, and Médard
[8] who state that the “most general possible linear codes” are
filter-bank network codes, a generalization of convolutional
codes. It is also stated in [8] that in [13] “it is conjectured that
(linear codes) are asymptotically optimal.”
We prove that vector linear coding is insufficient over the
general class of
-modules, which includes as special cases fi-
nite fields, commutative rings with identity, and Abelian groups.
Thus, the result is not restricted to alphabet cardinalities which
are powers of primes, nor to linearity with respect to only a
finite field. In addition, we show that linear coding (over fi-
nite fields) is not sufficient even asymptotically using fractional
coding, as the ratio of message dimensions to edge dimensions
approaches one. (In fact, we show that, in our example network,
nonlinear network coding gives exactly 10% more capacity than
the maximum capacity achievable using linear coding over fi-
nite fields.) From this, we deduce that even convolutional or
filter-bank linear coding is not sufficient for network coding.
3
Another form of “linearity” that one might consider (as sug-
gested by R. Yeung) consists of time sharing between linear
codes on different finite field alphabets. We note at the end of
Section IV that this form of linearity is not sufficient for our ex-
ample network either.
In what follows, the insufficiency of linear network codes is
shown for finite fields in Section II, for rings and modules in
Section III, and asymptotically for finite fields in Section IV.
We will often need to handle separately the cases of finite
fields with even cardinality (i.e., characteristic two) and odd car-
dinality (i.e., odd characteristic).
II. I
NSUFFICIENCY OF
NETWORK LINEAR CODES OVER
FINITE FIELDS
In this section, we establish the existence of a solvable net-
work that has no linear solution over any finite field and any
vector dimension.
First we give a useful lemma (an alternative proof follows
from the max-flow bound, e.g., see [18, p. 328]).
Lemma II.1: Suppose a network has a message
which is
demanded by a node
and is produced by exactly one source
node
. If there is a unique directed path from to , then the
coding capacity of the network is at most
.
3
Our proofs that linear codes are neither optimal for
R
-modules nor asymp-
totically optimal did not appear in the present paper until we submitted our revi-
sion in January 2005. (The quest for a proof based on
R
-modules was motivated
by a comment from one referee and for asymptotics by a question from another
referee.)
DOUGHERTY et al.: INSUFFICIENCY OF LINEAR CODING IN NETWORK INFORMATION FLOW 2747
Fig. 1. The network
N
has sources
n
,
n
,
n
emitting messages
a
,
b
,
c
,
respectively, and sinks
n
,
n
,
n
with demands
c
,
b
,
a
, respectively. Some
edges are labeled to their left to illustrate a scalar linear solution over any ring
alphabet with characteristic two (used in Lemma II.2), and edges are labeled
to their right with matrix coefficients
M
of an assumed solution used in Lem-
ma II.2.
Proof: Suppose there exists a fractional coding so-
lution over alphabet
with . If all messages other than
are fixed, then each edge of the path from to can take
on at most
different values. So can only decode at most
different values. But, so not every possible
message
at can be decoded at , a contradiction.
Suppose we impose on a network code the constraint that
for every node with in-degree one, the out-edges must carry the
same symbol as the lone in-edge, and for every source with ex-
actly one message, the out-edges must carry the source’s lone
message. Then, it is easy to see that the network has a linear so-
lution under this constraint for a given vector dimension over a
given finite field if and only if the network has an unconstrained
linear solution for the same vector dimension and over the same
finite field. This fact is used implicitly in the proofs of Lemmas
II.2 and II.3 and Theorem II.4 by assuming the described code
constraint.
Denote by
the network shown in Fig. 1.
Lemma II.2: The network
has a scalar linear solution
over any ring with characteristic two, but has no linear solution
for any vector dimension over a finite field with odd character-
istic. Also, the coding capacity of
is .
Proof: A scalar linear solution (as illustrated in Fig. 1) to
the network
over any ring of characteristic two is given by
the following edge functions and sink decoding functions:
(any edge function not shown is assumed to be an identity map-
ping). Note that the fact that the alphabet is a ring with charac-
teristic two is used only in decoding the message
at node
where .
Now, suppose the network has a linear solution over a finite
field
with odd characteristic and some vector dimension .
Let
be the identity matrix and for each and , let
be the vector carried on the edge from to . Then there exist
matrices with entries in (as illustrated in
Fig. 1), such that
(1)
(2)
(3)
(4)
(5)
(6)
Equating coefficients of
, , in (5) and (6) gives
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
By (9) and (13), the matrices
, , , , , are
invertible. Therefore,
by (8), and hence,
(16)
by (11). This implies that the matrices
and are in-
vertible. We have
by (10), so matrices
and are invertible. Since
by (14), the matrices and are invertible. Since
by (7), the matrix is invertible. Thus, since
by (15), the matrix is invertible. So,
is invertible for all .
Now, we have
[from (10)]
[from (16)]
(17)
2748 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 51, NO. 8, AUGUST 2005
Fig. 2. The network
N
has sources
n
,
n
,
n
,
n
,
n
with messages
a
,
b
,
c
,
d
,
e
, respectively. Sinks
n
through
n
each demand one of the messages,
as indicated. Some edges are labeled to illustrate a scalar linear solution over
any ring alphabet where
2
is invertible (used in Lemma II.3), and edges are
labeled to their right with matrix coefficients
M
of an assumed solution used
in Lemma II.3.
and
[from (12)]
[from (15)]
[from (8), (14)]
[from (16)]
(18)
But (17) and (18) imply that
which is impossible in a
field with odd characteristic.
Finally, since
has a scalar linear solution over GF , its
coding capacity (independent of alphabet size by Lemma I.1)
is at least
. Since is the only node that produces message
and node demands message , and since there is a unique
directed path from
to , the coding capacity of is at
most
by Lemma II.1. Hence, the coding capacity of is
exactly
.
Denote by the network shown in Fig. 2.
Lemma II.3: The network
has a scalar linear solution
over any ring where
is a unit, but has no linear solution for
any vector dimension over a finite field with characteristic two.
Also, the coding capacity of
is .
Proof: A scalar linear solution (as illustrated in Fig. 2) to
the network
over any ring where is invertible is given by
the following edge functions and sink decoding functions (any
edge function not shown is assumed to be an identity mapping):
Now, suppose the network has a linear solution over some
finite field
of characteristic two and with some finite vector
dimension
. Henceforth, let denote addition in . We can
write
(19)
(20)
(21)
(22)
(23)
where each
is a matrix with elements in , and
the messages
, , , , are -dimensional vectors. Equations
(21)–(23) come from the demands at sinks
, , . Let
be the identity matrix over . Equating coefficients of ,
, in (21)–(23) gives
(24)
(25)
(26)
(27)
(28)
(29)
(30)
where minus signs have been omitted since the finite-field al-
phabet has characteristic two. Equation (24) implies that
are invertible. Since the right-hand sides of (25)–(30) are invert-
ible, the left-hand-side matrices
must also be invertible. So is invertible for all . Thus,
[from (25), (26)]
DOUGHERTY et al.: INSUFFICIENCY OF LINEAR CODING IN NETWORK INFORMATION FLOW 2749
Fig. 3. The network
N
has sources
n
,
n
,
n
,
n
,
n
with messages
a
,
b
,
c
,
d
,
e
, respectively. Sinks
n
through
n
each demand one of the messages, as
indicated. Some edges are labeled to illustrate a nonlinear solution over an alphabet of size
4
(used in Theorem II.4).
[from (27), (28)]
[from (29),(30)]
and therefore,
Finally
so
(31)
Hence, for any message assigned to
, if the messages
and are assigned to and , respectively,
then
, by (19) and (20), and therefore,
by (31). A similar argument shows that, for any
message assigned to
, there exist messages that can be assigned
to
and that result in .
Thus, for every message vector assigned to
, there exist as-
signments of messages to
, , , such that all six inputs
to node are zero. This contradicts the assumption that the
demand
at node can be recovered, since is not uniquely
determined by the node’s inputs.
Finally, since
has a scalar linear solution over GF , its
coding capacity (independent of alphabet size by Lemma I.1)
is at least
. Since is the only node that produces message
and node demands message , and since there is a unique
directed path from
to , the coding capacity of is at
most
by Lemma II.1. Hence, the coding capacity of is
exactly
.
Denote by the network shown in Fig. 3, with nodes
. In the network, the left-most part is the
network (with sinks , , ) and the rest of is the
network.
Theorem II.4 shows that linear network codes are insufficient
over finite-field alphabets.
Theorem II.4: There exists a solvable network that has no
linear solution over any finite field and any vector dimension.
Proof: The proof is achieved with
, which combines
networks
and . Lemmas II.2 and II.3 show that network
does not have a vector linear solution over any finite-field
alphabet.
We now demonstrate a solution to the network over an al-
phabet of cardinality
, as indicated in Fig. 3. The symbols
and indicate addition and subtraction in the ring of in-
tegers modulo
, the symbol indicates addition in the ring
(i.e., bitwise XOR), and indicates the result of
exchanging the order of the bits in a 2-bit binary word
.We
represent the elements of the alphabet either as members of
when using or , or as elements of (i.e., 2-bit binary
words) when using
or . Note that the functions and
are linear over but not over GF or , the function
is linear over and GF but not over , and the
function
is not linear over any of these. The demands are
met as follows: