Showing papers presented at "Information Theory Workshop in 2004"

Minimum-energy multicast in mobile ad hoc networks using network coding

Abstract: The minimum energy required to transmit one bit of information through a network characterizes the most economical way to communicate in a network. In this paper, we show that, under a layered model of wireless networks, the minimum energy-per-bit for multicasting in a mobile ad hoc network can be found by a linear program; the minimum energy-per-bit can be attained by performing network coding. Compared with conventional routing solutions, network coding not only allows a potentially lower energy-per-bit to be achieved, but also enables the optimal solution to be found in polynomial time, in sharp contrast with the NP-hardness of constructing the minimum-energy multicast tree as the optimal routing solution. We further show that the minimum energy multicast formulation is equivalent to a cost minimization with linear edge-based pricing, where the edge prices are the energy-per-bits of the corresponding physical broadcast links. This paper also investigates minimum energy multicasting with routing. Due to the linearity of the pricing scheme, the minimum energy-per-bit for routing is achievable by using a single distribution tree. A characterization of the admissible rate region for routing with a single tree is presented. The minimum energy-per-bit for multicasting with routing is found by an integer linear program. We show that the relaxation of this integer linear program, studied earlier in the Steiner tree literature, can now be interpreted as the optimization for minimum energy multicasting with network coding. In short, this paper presents a unifying study of minimum energy multicasting with network coding and routing.

Digital fountains: a survey and look forward

Abstract: We survey constructions and applications of digital fountains, an abstraction of erasure coding for network communication. Digital fountains effectively change the standard paradigm where a user receives an ordered stream of packets to one where a user must simply receive enough packets in order to obtain the desired data. Obviating the need for ordered data simplifies data delivery, especially when the data is large or is to be distributed to a large number of users. We also examine barriers to the adoption of digital fountains and discuss whether they can be overcome.

On the advantage of network coding for improving network throughput

Abstract: Given a data network with link capacities, we consider the throughput of the network for a multicast session involving a source node and a given set of terminals It is known that network coding can improve the throughput of the network We study the coding advantage, ie the ratio of the throughput using network coding to that without using network coding We show that the maximum coding advantage for a given network is equal to the integrality gap of certain linear programming (LP) formulations for a Steiner tree This holds for both directed as well as undirected networks For directed networks, the coding advantage is equal to the integrality gap of the directed Steiner tree LP formulation; for undirected networks, the coding advantage is equal to the integrality gap of the bidirected cut LP formulation for the Steiner tree This relates the coding advantage to well studied notions in combinatorial optimization Further, this connection improves the known bounds on the coding advantage for both undirected as well directed networks

Outage minimization and optimal power control for the fading relay channel

Abstract: In this work, we show that in the wireless relay network, a tremendous savings in energy can be achieved by having side information at the transmitters and by employing power control. We present efficient protocols and the corresponding optimal power control policies that approach the universal lower bound on the outage probability of the block fading relay channel. Each of the proposed protocols have their own utility for specific channel conditions. However, a hybrid protocol between two known coding schemes is the best scheme for all channel conditions and is sufficient to approach the lower bound on outage probability. Unlike the single link channel, we show that exploiting the knowledge of the channel at the transmitters can significantly lower the outage even if the transmit powers at the source and relay have to be kept constant. In this case, it is also demonstrated that the lower bound on outage is closely followed by the outage probability of the hybrid protocol. Our results reveal that exploiting the right network protocol in conjunction with power control result in orders of magnitude savings in power over direct transmission for a target performance level.

Network coding gain of combination networks

Abstract: Network coding theory shows that the multicast rate in a network can be increased if coding is allowed in the network nodes Thus the capacity of a network with network coding is generally larger than that with routing alone We quantify this gain in closed form for a class of networks called combination networks From this result, it can readily be deduced that network coding gain can be unbounded

On the capacity of vector Gaussian interference channels

Abstract: The capacity of a vector Gaussian interference channel is investigated. Outer bounds, and where possible, capacity regions of a class of interference channels is characterized. The analysis of single transmit multiple receive antenna (SIMO) Gaussian interference channels with strong interference can be easily seen to be exactly analogous to that of a single transmit single receive antenna system. This paper demonstrates that, in contrast, multiple transmit single receive antenna (MISO) Gaussian interference channels are much harder to characterize. In this paper, the capacity region for a class of MISO interference channels with very strong interference is characterized. Also, the rank of the optimal transmit policy in a MISO Gaussian interference channel is shown to be bounded by the number of users in the system. Finally, outer bounds on the capacity region of the general multiple transmit and receive antenna (MIMO) Gaussian interference channels are derived. A new outer bound is obtained, which combines and improves previously known strategies for bounding the capacity of interference channels.

Shannon information and biological fitness

Abstract: When studying information, biologists and behavioral scientists often eschew Shannon entropy. Instead, they commonly use a decision-theoretic measure of the value of information, on the grounds that Shannon's measure draws no distinction between useful and useless information. Here we show that these two measures are intimately related in the context of biological evolution. We present a simple model of evolution in an uncertain environment, and calculate the increase in Darwinian fitness that is made possible by information about the environmental state. This fitness increase - the fitness value of information - is a composite of both the Shannon entropy and the decision-theoretic measure of information value. Furthermore, the Shannon entropy of the environment, which seemingly fails to take anything about Darwinian fitness into account, nonetheless imposes an upper bound on the fitness value of information.

Information geometric formulation and interpretation of accelerated Blahut-Arimoto-type algorithms

Abstract: We propose two related iterative algorithms for computing the capacity of discrete memoryless channels. The celebrated Blahut-Arimoto algorithm is a special case of our framework. The formulation of these algorithms is based on the natural gradient and proximal point methods. We also provide interpretations in terms of notions from information geometry. A theoretical convergence analysis and simulation results demonstrate that our new algorithms have the potential to significantly outperform the Blahut-Arimoto algorithm in terms of convergence speed.

Generalized belief propagation receiver for near-optimal detection of two-dimensional channels with memory

Abstract: We propose a generalized belief propagation (GBP) receiver for two-dimensional (2D) channels with memory, which is applicative to 2D intersymbol interference (ISI) equalization and multiuser detection (MUD). Our experimental study demonstrates that under non-trivial interference conditions, the performance of this fully tractable GBP receiver is almost identical to the performance of the optimal maximum a-posteriori (MAP) receiver.

On the optimum number of hops in linear wireless networks

Abstract: We consider a wireless communication system with a single source node, a single destination node, and multiple relay nodes placed equidistantly between them. We limit our analysis to the case of coded TDMA multihop transmission, i.e., the nodes do not cooperate and do not try to access the channel simultaneously. Given a global constraint on bandwidth, we determine the number of hops that achieves a desired end-to-end rate with the least total transmission power. Furthermore, we examine how the optimum number of hops changes when an end-to-end delay constraint is introduced using the sphere-packing bound and computer simulations. The analysis demonstrates that the optimum number of hops depends on the end-to-end rate and the path-loss exponent. Specifically, we show the existence of an asymptotic per-link spectral efficiency, which is the preferred spectral efficiency in TDMA multihop transmission.

Mutual information and conditional mean estimation in Poisson channels

Abstract: Following the recent discovery of new connections between information and estimation in Gaussian channels, this paper reports parallel results in the Poisson regime. Both scalar and continuous-time Poisson channels are considered. It is found that, regardless of the statistics of the input, the derivative of the input-output mutual information with respect to the dark current can be expressed in the expected difference between the logarithm of the input and the logarithm of its conditional mean estimate (noncausal in case of continuous-time). The same is true for the derivative with respect to input scaling, but with the logarithmic function replaced by x log x.

Transmitter cooperation in ad-hoc wireless networks: does dirty-paper coding beat relaying?

Abstract: We investigate capacity and achievable rates for transmitter cooperation schemes in ad-hoc wireless networks. In addition to cooperative dirty paper coding, we propose two new cooperative transmission techniques: time-division successive broadcasting and time-division relaying. We show that transmitter cooperation can significantly increase capacity, even if one of the cooperating nodes is halfway between the transmit and receive node clusters. However, the best form of cooperation depends on the relative geometry of the transmit and receive clusters. When the transmitters are close together, cooperative dirty paper coding achieves the highest rates. However, if one of the transmitters is relatively close to the receive cluster, cooperative broadcasting or relaying achieves higher rates than dirty paper coding. That is because, at large separations, the exchange of messages between the transmitters required for dirty paper coding consumes a substantial amount of power. We show that in most cases transmitter cooperation provides a substantial capacity improvement over noncooperative techniques, especially under an equal rate constraint.

Results on punctured LDPC codes

Abstract: In this paper we study some fundamental properties of punctured LDPC codes. We first prove that for any ensemble of LDPC codes, there exists a puncturing threshold p*. We then find lower bounds on the achievable rates of punctured codes over general MBIOS channels. These bounds are satisfied by using only one encoder and decoder for all rates. We then prove that for any rates R/sub 1/ and R/sub 2/ satisfying 0 < R/sub 1/ < R/sub 2/ < 1, there exists an ensemble of LDPC codes with the following property. The ensemble can be punctured from rate R/sub 1/ to R/sub 2/ resulting in asymptotically good codes for all rates R/sub 1/ /spl les/ R /spl les/ R/sub 2/. Specifically, this implies that rates arbitrarily close to one are achievable via puncturing. We also show that punctured LDPC codes are as good as ordinary LDPC codes. For binary erasure channel (BEC) and arbitrary positive numbers R/sub 1/ < R/sub 2/ < 1, we prove the existence of the sequences of punctured LDPC codes that are capacity achieving for all rates R/sub 1/ /spl les/ R /spl les/ R/sub 2/. Based on the above observations, we then propose a method to design good punctured LDPC codes over a broad range of rates. The method is very simple and does not suffer from the performance degradation at high rates. Finally, we show that punctured codes might be useful for proof of the existence of capacity-achieving LDPC codes over memoryless binary-input output-symmetric channels.

Opportunistic dynamic subchannel allocation in multiuser OFDM networks with limited feedback

Abstract: In this paper we present a simple scheme for subchannel allocation in OFDM multiuser networks in the presence of limited feedback, in particular, when only one bit of information per subchannel is available at the base station. Our objective is to maximize the sum rate capacity of the network in the downlink transmission. We show that even with very limited feedback the sum rate capacity growth is the same as the fully informed transmission. We also extend this result to the case when subchannels are correlated.

Zero-error information and applications in cryptography

Abstract: In analogy to the zero-error variant of the channel capacity, the zero-error information between two random variables is defined We show that our definition is natural in the sense that the representation of the channel capacity with respect to mutual information carries over to the zero-error variants of the quantities It is shown that the new notion, together with two operators introduced in the same context, namely the common random variable of two random variables and the dependent part of a random variable with respect to another, is useful for giving characterizations of the possibility of realizing cryptographic tasks - such as bit commitment, coin tossing, or oblivious transfer - from correlated pieces of information

Achievable diversity-vs-multiplexing tradeoffs in half-duplex cooperative channels

Abstract: In this paper, we propose novel cooperative transmission protocols for delay limited coherent fading channels consisting of N (half-duplex and single-antenna) partners and one cell site. In our work, we differentiate between the cooperative relay, broadcast, and multiple-access channels. The proposed protocols are evaluated using the Zheng-Tse diversity-multiplexing tradeoff. For the relay channel, we investigate two classes of cooperation schemes; namely, amplify and forward (AF) and decode and forward (DF). For the first class, we propose a new AF protocol and show it to outperform the space-time coded protocol of Laneman and Wornell without requiring decoding/encoding at the relays. For the class of DF protocols, we develop a dynamic decode and forward (DDF) protocol that achieves the optimal tradeoff for multiplexing gains 0 /spl les/ r /spl les/ 1/N. Furthermore, with a single relay, the DDF protocol is shown to dominate the class of AF protocols for all multiplexing gains. The superiority of the DDF protocol is shown to be more significant in the cooperative broadcast channel. The situation is reversed in the cooperative multiple-access channel where we propose a new AF protocol that achieves the optimal tradeoff for all multiplexing gains. A distinguishing feature of the proposed protocols in the three scenarios is that they do not rely on orthogonal subspaces, allowing for a more efficient use of resources.

New bounds on the entropy rate of hidden Markov processes

Abstract: Let {X/sub t/} be a stationary finite-alphabet Markov chain and {Z/sub t/} denote its noisy version when corrupted by a discrete memoryless channel Let P(X/sub t//spl isin//spl middot/|Z/sub -/spl infin///sup t/) denote the conditional distribution of X/sub t/ given all past and present noisy observations, a simplex-valued random variable We present a new approach to bounding the entropy rate of {Z/sub t/} by approximating the distribution of this random variable This approximation is facilitated by the construction and study of a Markov process whose stationary distribution determines the distribution of P(X/sub t//spl isin//spl middot/|Z/sub -/spl infin///sup t/) To illustrate the efficacy of this approach, we specialize it and derive concrete bounds for the case of a binary Markov chain corrupted by a binary symmetric channel (BSC) These bounds are seen to capture the behavior of the entropy rate in various asymptotic regimes

On multiple access for distributed dependent sources: a content-based group testing approach

Abstract: In this paper we consider the multiple access problem with distributed dependent sources We derive the optimal designs for the case of N correlated binary sources whose data is modelled as a two-state Markov chain The solution can be classified as a group testing technique where data values at the sensors are determined through the successive refinements of the tests over smaller groups The tests form, progressively, an accurate map of the sensor data at the central receiver We derive the conditions on the parameters of the data model for which the group testing approach is superior to time sharing In contrast to standard multiple access techniques, this is the first method proposed for data retrieval from distributed dependent sources which is content-based rather than user-based

Capacity of space-time wireless channels: a physical perspective

Abstract: Existing results on MIMO channel capacity assume a rich scattering environment in which the channel power scales quadratically with the number of antennas, resulting in linear capacity scaling with the number of antennas. However, such scaling in channel power is physically impossible indefinitely. We thus address the following fundamental question: for a given channel power scaling law, what is the best achievable capacity scaling law? For a channel power scaling, /spl rho//sub c/(N) = O(N/sup /spl gamma//), /spl gamma/ /spl isin/ (0,2], we argue that the channel capacity cannot scale faster than C(N) = O(/spl radic/(/spl rho//sub c/(N))) = O(N/sup /spl gamma//2/). Our approach is based on a family of space-time channels corresponding to different distributions of channel power in the spatial signal space dimensions. We develop the concept of an ideal MIMO channel that achieves the optimal scaling law for a given /spl rho//sub c/(N). For a given number of antennas, unlike existing results that either emphasize the low or high SNR regimes, we propose a methodology for capacity-optimal signaling at any SNR. The methodology is based on creating the ideal channel from any given physical scattering environment via adaptive-resolution array configurations.

Gaussian multiple-access channels under received-power constraints

Abstract: Performance limits for multiple-access channels are well known as long as the messages of different nodes are independent. Considerable research efforts have been devoted to the problem when the source information is dependent across nodes, revealing capacities for certain special cases. This work presents a new scenario for which conclusive results can be established: The additive white Gaussian multiple-access channel under a constraint on the power at the receiver. The results of this paper have natural applications to sensor networks. By contrast to recent scaling-law results, this paper establishes coinciding upper and lower bounds, and hence, the exact performance limits, for some scenarios of interest.

Pseudo-codewords of cycle codes via zeta functions

Abstract: Cycle codes are a special case of low-density parity-check (LDPC) codes and as such can be decoded using an iterative message-passing decoding algorithm on the associated Tanner graph. The existence of pseudo-codewords is known to cause the decoding algorithm to fail in certain instances. In this paper, we draw a connection between pseudo-codewords of cycle codes and the so-called edge zeta function of the associated normal graph and show how the Newton polytope of the zeta function equals the fundamental cone of the code, which plays a crucial role in characterizing the performance of iterative decoding algorithms.

Distributed source coding using serially-concatenated-accumulate codes

Abstract: We describe a practical method for distributed compression of q-ary sources using multi-level serially concatenated-accumulate codes. Our approach works well at high compression rates, and allows for graceful and incremental rate-adaptivity. Simulations show that the compression efficiency is near the information-theoretic limits for correlations between sources that obey a Gaussian or Laplacian distribution.

Genomic analysis using methods from information theory

Abstract: Methods used in information theory can be used in genomic analysis to provide a more meaningful insight into the genetic process. Using an information transfer model between certain polymorphisms in the human genome (SNP) and certain diseases (e.g. Alzheimer), Shannon's mutual information can identify the relevant SNP. Several compression algorithms approximating the entropy of DNA sequences are used to distinguish between introns and exons. Certain distance measures derived from DNA sequences allow classification leading to evolutionary trees. They can also be used for content recognition.

A concatenated [(4, 1, 3)] quantum convolutional code

Abstract: We construct a concatenated [(4, 1, 3)] quantum convolutional code from a (2, 1, 2) classical convolutional code. This quantum convolutional code can correct one general error.

Decentralized network coding

Abstract: This paper proposes deterministic algorithms for decentralized network coding. Decentralized coding allows us to locally specify the coding operations at network nodes without knowledge of the overall network topology, and to accommodate future changes in the network such as addition of receivers. To the best of our knowledge, these are the first deterministic decentralized algorithms proposed for network coding.

Universal variable-length data compression of binary sources using fountain codes

Abstract: This paper proposes a universal variable-length lossless compression algorithm based on fountain codes. The compressor concatenates the Burrows-Wheeler block sorting transform (BWT) with a fountain encoder, together with the closed-loop iterative doping algorithm. The decompressor uses a belief propagation algorithm in conjunction with the iterative doping algorithm and the inverse BWT. Linear-time compression/decompression complexity and competitive performance with respect to state-of-the-art compression algorithms are achieved.

MIMO channels in the low SNR regime: communication rate, error exponent and signal peakiness

Abstract: We consider noncoherent MIMO fading channels and characterize the reliability function in the low-SNR regime as a function of the number of transmit and receive antennas. We assume no CSI is available at the transmitter or the receiver. For the case when the fading matrix H has independent entries, we show that the number of transmit antennas plays a key role in reducing the peakiness in the input signal required to achieve the optimal error exponent for a given communication rate. Further, by considering a correlated channel model, we show that the maximum performance gain (in terms of the error exponent and communication rate) is achieved when the entries of the channel fading matrix are fully correlated.

On multiple description source coding with decoder side information

Abstract: We formulate a multi-terminal source coding problem, where we are required to construct a multiple-description code for a source sequence when side information about dependent random processes is available at the decoder only, or at both the decoder and the encoder We describe an achievable rate-distortion region for these problems in two cases: where there is common side-information at the decoders and when they are different In the quadratic Gaussian case, and when there is common side information among the decoders, we show that the rate region when both the encoder and decoder have access to the side information coincides with that of decoder-only side information This is analogous to the single-description (Wyner-Ziv) case, and an explicit characterization of the rate-distortion region is provided for this case

On the outage probability of quasi-orthogonal space-time codes

Abstract: Recently, a quasi-orthogonal space-time block code (QSTBC) capable of achieving a significant fraction of the outage mutual information of a multiple-input-multiple output (MIMO) wireless communication system for the case of four transmit and one receive antennas was proposed. We generalize these results to 2/sup n/ transmit and an arbitrary number of receive antennas. Furthermore, we derive an analytical lower bound for the fraction of outage probability achieved with QSTBC and show that this bound is tight for low signal-to-noise-ratios (SNR) values and also for increasing number of receive antennas. We present also an upper bound, which is tight for high SNR values and derive analytical expressions for the case of four transmit antennas. Furthermore, by utilizing the special structure of the QSTBC we propose a new transmit strategy, which decouples the signals transmitted from different antennas in order to detect the symbols separately with a linear ML-detector rather than joint detection, an up to now only known advantage of orthogonal space-time codes (OSTBC).

Collaborative beamforming in ad hoc networks

Abstract: The performance of collaborative beamforming is analyzed using the theory of random arrays. The statistical average and distribution of the beam patterns of randomly generated phased arrays are derived in the framework of wireless ad hoc sensor networks. Each sensor node is assumed to have a single isotropic antenna and nodes in the cluster collaboratively transmit the signal such that the signal in the target direction is coherently added in the far-field region. The distribution of the maximum beam pattern sidelobe is also analyzed and a closed-form bound is derived based on the Gaussian approximation method.