This comprehensive treatment of network information theory and its applications provides the first unified coverage of both classical and recent results. With an approach that balances the introduction of new models and new coding techniques, readers are guided through Shannon's point-to-point information theory, single-hop networks, multihop networks, and extensions to distributed computing, secrecy, wireless communication, and networking. Elementary mathematical tools and techniques are used throughout, requiring only basic knowledge of probability, whilst unified proofs of coding theorems are based on a few simple lemmas, making the text accessible to newcomers. Key topics covered include successive cancellation and superposition coding, MIMO wireless communication, network coding, and cooperative relaying. Also covered are feedback and interactive communication, capacity approximations and scaling laws, and asynchronous and random access channels. This book is ideal for use in the classroom, for self-study, and as a reference for researchers and engineers in industry and academia.

Network Information Theory

Information Theory and Reliable Communication

Overcomplete representations are attracting interest in signal processing theory, particularly due to their potential to generate sparse representations of signals. However, in general, the problem of finding sparse representations must be unstable in the presence of noise. This paper establishes the possibility of stable recovery under a combination of sufficient sparsity and favorable structure of the overcomplete system. Considering an ideal underlying signal that has a sufficiently sparse representation, it is assumed that only a noisy version of it can be observed. Assuming further that the overcomplete system is incoherent, it is shown that the optimally sparse approximation to the noisy data differs from the optimally sparse decomposition of the ideal noiseless signal by at most a constant multiple of the noise level. As this optimal-sparsity method requires heavy (combinatorial) computational effort, approximation algorithms are considered. It is shown that similar stability is also available using the basis and the matching pursuit algorithms. Furthermore, it is shown that these methods result in sparse approximation of the noisy data that contains only terms also appearing in the unique sparsest representation of the ideal noiseless sparse signal.

Stable recovery of sparse overcomplete representations in the presence of noise

We propose a new method for reconstruction of sparse signals with and without noisy perturbations, termed the subspace pursuit algorithm. The algorithm has two important characteristics: low computational complexity, comparable to that of orthogonal matching pursuit techniques when applied to very sparse signals, and reconstruction accuracy of the same order as that of linear programming (LP) optimization methods. The presented analysis shows that in the noiseless setting, the proposed algorithm can exactly reconstruct arbitrary sparse signals provided that the sensing matrix satisfies the restricted isometry property with a constant parameter. In the noisy setting and in the case that the signal is not exactly sparse, it can be shown that the mean-squared error of the reconstruction is upper-bounded by constant multiples of the measurement and signal perturbation energies.

Subspace Pursuit for Compressive Sensing Signal Reconstruction

We propose a new method for reconstruction of sparse signals with and without noisy perturbations, termed the subspace pursuit algorithm. The algorithm has two important characteristics: low computational complexity, comparable to that of orthogonal matching pursuit techniques when applied to very sparse signals, and reconstruction accuracy of the same order as that of LP optimization methods. The presented analysis shows that in the noiseless setting, the proposed algorithm can exactly reconstruct arbitrary sparse signals provided that the sensing matrix satisfies the restricted isometry property with a constant parameter. In the noisy setting and in the case that the signal is not exactly sparse, it can be shown that the mean squared error of the reconstruction is upper bounded by constant multiples of the measurement and signal perturbation energies.

We examine the problem of periodic nonuniform sampling of a multiband signal and its reconstruction from the samples. This sampling scheme, which has been studied previously, has an interesting optimality property that uniform sampling lacks: one can sample and reconstruct the class /spl Bscr/(/spl Fscr/) of multiband signals with spectral support /spl Fscr/, at rates arbitrarily close to the Landau (1969) minimum rate equal to the Lebesgue measure of /spl Fscr/, even when /spl Fscr/ does not tile R under translation. Using the conditions for exact reconstruction, we derive an explicit reconstruction formula. We compute bounds on the peak value and the energy of the aliasing error in the event that the input signal is band-limited to the "span of /spl Fscr/" (the smallest interval containing /spl Fscr/) which is a bigger class than the valid signals /spl Bscr/(/spl Fscr/), band-limited to /spl Fscr/. We also examine the performance of the reconstruction system when the input contains additive sample noise.

Perfect reconstruction formulas and bounds on aliasing error in sub-Nyquist nonuniform sampling of multiband signals

An achievable region for the L-channel multiple description coding problem is presented. This region generalizes two-channel results of El Gamal and Cover (1982) and of Zhang and Berger (1987). It further generalizes three-channel results of Gray and Wyner (1974) and of Zhang and Berger. A source that is successively refinable on chains is shown to be successively refinable on trees. A new outer bound on the rate-distortion (RD) region for memoryless Gaussian sources with mean squared error distortion is also derived. The achievable region meets this outer bound for certain symmetric cases.

/pdf/multiple-description-coding-with-many-channels-zqdffphl2d.pdf

Multiple description coding with many channels

We study the problem of optimal sub-Nyquist sampling for perfect reconstruction of multiband signals. The signals are assumed to have a known spectral support /spl Fscr/ that does not tile under translation. Such signals admit perfect reconstruction from periodic nonuniform sampling at rates approaching Landau's (1967) lower bound equal to the measure of /spl Fscr/. For signals with sparse /spl Fscr/, this rate can be much smaller than the Nyquist rate. Unfortunately the reduced sampling rates afforded by this scheme can be accompanied by increased error sensitivity. In a previous study, we derived bounds on the error due to mismodeling and sample additive noise. Adopting these bounds as performance measures, we consider the problems of optimizing the reconstruction sections of the system, choosing the optimal base sampling rate, and designing the nonuniform sampling pattern. We find that optimizing these parameters can improve system performance significantly. Furthermore, uniform sampling is optimal for signals with /spl Fscr/ that tiles under translation. For signals with nontiling /spl Fscr/, which are not amenable to efficient uniform sampling, the results reveal increased error sensitivities with sub-Nyquist sampling. However, these can be controlled by optimal design, demonstrating the potential for practical multifold reductions in sampling rate.

Optimal sub-Nyquist nonuniform sampling and reconstruction for multiband signals

We present a new construction of 16-QAM Golay sequences of length n = 2/sup m/. The number of constructed sequences is (14 + 12m)(m!/2)4/sup m+1/. When employed as a code in an orthogonal frequency-division multiplexing (OFDM) system; this set of sequences has a peak-to-mean envelope power ratio (PMEPR) of 3.6. By considering two specific subsets of these sequences, we obtain new codes with PMEPR bounds of 2.0 and 2.8 and respective code sizes of (2 + 2m)(m!/2)4/sup m+1/ and (4 + 4m)(m!/2)4/sup m+1/. These are larger than previously known codes for the same PMEPR bounds.

A new construction of 16-QAM Golay complementary sequences

We address the problem of sampling of 2D signals with sparse multi-band spectral structure. We show that the signal can be sampled at a fraction of the its Nyquist density determined by the occupancy of the signal in its frequency domain, but without explicit knowledge of its spectral structure. We find that such a signal can almost surely be reconstructed from its multi-coset samples provided that a universal pattern is used. Also, the scheme can attain the Landau-Nyquist minimum density asymptotically. The spectrum blind feature of our reconstruction scheme has potential applications in Fourier imaging. We apply the sampling scheme on a test image to demonstrate its performance.

/pdf/further-results-on-spectrum-blind-sampling-of-2d-signals-3wa0au4a2i.pdf

Raman Venkataramani

Papers

Perfect reconstruction formulas and bounds on aliasing error in sub-Nyquist nonuniform sampling of multiband signals

Multiple description coding with many channels

Optimal sub-Nyquist nonuniform sampling and reconstruction for multiband signals

A new construction of 16-QAM Golay complementary sequences

Further results on spectrum blind sampling of 2D signals