We consider the problem of transmitting a band-limited Gaussian source on an additive band-limited Gaussian noise channel. The well-known "threshold effect" dictates that the more powerful a code is, the more sensitive it is to the exact knowledge of the channel noise. A code is said to be robust if it is asymptotically optimal for a wide range of channel noise. Thus, robust codes have a "graceful degradation" characteristic and are free of the threshold effect. It is demonstrated that robust codes exist whenever the source and channel bandwidths are equal. In the unequal-bandwidth case, a collection of nearly robust joint source-channel codes is constructed using a hybrid digital-analog (HDA) coding technique. For designing nearly robust codes, a matched tandem code whose channel encoder's output is partially/fully matched to its input is proposed and the existence of an asymptotically optimal matched tandem code is shown. The nearly robust codes achieve the Shannon limit (theoretically optimum distortion) and have a less severe threshold effect. Finally, for the case of two different noise conditions, the distortion regions of these codes are determined.

Hybrid digital-analog (HDA) joint source-channel codes for broadcasting and robust communications

We discuss reliable transmission of a discrete memoryless source over a discrete memoryless broadcast channel, where each receiver has side information (of arbitrary quality) about the source unknown to the sender. When there are K=2 receivers, the optimum coding strategy using separate and stand-alone source and channel codes is to build two independent binning structures and send bin indices using degraded message sets through the channel, yielding a full characterization of achievable rates. However, as we show with an example, generalization of this technique to multiple binning schemes does not fully resolve the K>2 case. Joint source-channel coding, on the other hand, allows for a much simpler strategy (i.e., with no explicit binning) yielding a successful single-letter characterization of achievable rates for any Kges2. This characterization, which utilizes a trivial outer bound to the capacity region of general broadcast channels, is in terms of marginal source and channel distributions rather than a joint source-channel distribution. This contrasts with existing results for other multiterminal scenarios and implies that optimal schemes achieve "operational separation." On the other hand, it is shown with an example that an optimal joint source-channel coding strategy is strictly advantageous over the combination of stand-alone source and channel codes, and thus "informational separation" does not hold

Slepian-Wolf coding over broadcast channels

In this paper, we derive information theoretic performance bounds to sensing and reconstruction of sparse phenomena from noisy projections. We consider two settings: output noise models where the noise enters after the projection and input noise models where the noise enters before the projection. We consider two types of distortion for reconstruction: support errors and mean-squared errors. Our goal is to relate the number of measurements, m , and SNR, to signal sparsity, k, distortion level, d, and signal dimension, n . We consider support errors in a worst-case setting. We employ different variations of Fano's inequality to derive necessary conditions on the number of measurements and SNR required for exact reconstruction. To derive sufficient conditions, we develop new insights on max-likelihood analysis based on a novel superposition property. In particular, this property implies that small support errors are the dominant error events. Consequently, our ML analysis does not suffer the conservatism of the union bound and leads to a tighter analysis of max-likelihood. These results provide order-wise tight bounds. For output noise models, we show that asymptotically an SNR of ((n)) together with (k (n/k)) measurements is necessary and sufficient for exact support recovery. Furthermore, if a small fraction of support errors can be tolerated, a constant SNR turns out to be sufficient in the linear sparsity regime. In contrast for input noise models, we show that support recovery fails if the number of measurements scales as o(n(n)/SNR), implying poor compression performance for such cases. Motivated by the fact that the worst-case setup requires significantly high SNR and substantial number of measurements for input and output noise models, we consider a Bayesian setup. To derive necessary conditions, we develop novel extensions to Fano's inequality to handle continuous domains and arbitrary distortions. We then develop a new max-likelihood analysis over the set of rate distortion quantization points to characterize tradeoffs between mean-squared distortion and the number of measurements using rate-distortion theory. We show that with constant SNR the number of measurements scales linearly with the rate-distortion function of the sparse phenomena.

Information Theoretic Bounds for Compressed Sensing

We consider the problem of broadcasting a single Gaussian source to two listeners over a Gaussian broadcast channel, with rho channel uses per source sample, where rho>1. A distortion pair (D1 ,D2) is said to be achievable if one can simultaneously achieve a mean-squared error (MSE) D1 at receiver 1 and D2 at receiver 2. The main result of this correspondence is an outer bound for the set of all achievable distortion pairs. That is, we find necessary conditions under which (D1,D2) is achievable. We then apply this result to the problem of point-to-point transmission over a Gaussian channel with unknown signal-to-noise ratio (SNR) and rho>1. We show that if a system must be optimal at a certain SNRmin, then, asymptotically, the system distortion cannot decay faster than O(1/SNR). As for achievability, we show that a previously reported scheme, due to Mittal and Phamdo (2002), is optimal at high SNR. We introduce two new schemes for broadcasting with bandwidth expansion, combining digital and analog transmissions. We finally show how a system with a partial feedback, returning from the bad receiver to the transmitter and to the good receiver, achieves a distortion pair that lies on the outer bound derived here

Distortion Bounds for Broadcasting With Bandwidth Expansion

We consider transmitting a source across a pair of independent, nonergodic channels with random states (e.g., slow-fading channels) so as to minimize the average distortion. The general problem is unsolved. Hence, we focus on comparing two commonly used source and channel encoding systems which correspond to exploiting diversity either at the physical layer through parallel channel coding or at the application layer through multiple description (MD) source coding. For on-off channel models, source coding diversity offers better performance. For channels with a continuous range of reception quality, we show the reverse is true. Specifically, we introduce a new figure of merit called the distortion exponent which measures how fast the average distortion decays with signal-to-noise ratio. For continuous-state models such as additive white Gaussian noise (AWGN) channels with multiplicative Rayleigh fading, optimal channel coding diversity at the physical layer is more efficient than source coding diversity at the application layer in that the former achieves a better distortion exponent. Finally, we consider a third decoding architecture: MD encoding with joint source-channel decoding. We show that this architecture achieves the same distortion exponent as systems with optimal channel coding diversity for continuous-state channels, and maintains the advantages of MD systems for on-off channels. Thus, the MD system with joint decoding achieves the best performance from among the three architectures considered, on both continuous-state and on-off channels.

/pdf/source-channel-diversity-for-parallel-channels-2aya1b3yy5.pdf

Source-channel diversity for parallel channels

We consider the problem of transmitting a Gaussian source over a Gaussian broadcast channel with p>1 channel uses per source sample. Our goal is to minimize simultaneously the squared-error distortion D/sub 1/ at receiver 1 (connected to the good channel) and D/sub 2/ at receiver 2 (connected to the bad channel). The distortion region, defined as the set of all simultaneously-achievable distortion pairs (D/sub 1/, D/sub 2/), is yet unknown. A separation-based system, consisting of a successive refinement source encoder followed by a broadcast channel encoder, is suboptimal. A variant of the problem is that of sending a Gaussian source over an AWGN channel, where the SNR is unknown except that SNR>SNR/sub min/ where SNR/sub min/ is known. In these conditions, the distortion cannot decay faster than 1/(SNR) as the SNR improves.

/pdf/distortion-bounds-for-broadcasting-with-bandwidth-expansion-3fwj5uqmrg.pdf

Distortion bounds for broadcasting with bandwidth expansion

Suppose that we want to send a description of a single source to two listeners through a Gaussian broadcast channel, where the channel is used once per source sample. The problem of joint source-channel coding is to design a communication system to minimize the distortion D/sub 1/ at receiver 1 and at the same time minimize the distortion D/sub 2/ at receiver 2. If the source is Gaussian, the optimal solution is well known, and it is achieved by an uncoded "analog" scheme. We consider a Gaussian mixture source. We derive inner and outer bounds for the distortion region of all (D/sub 1/, D/sub 2/) pairs that are simultaneously achievable. The outer bound is based on the entropy power inequality, while the inner bound is attained by a digital-over-analog encoding scheme, which we present. We also show that if the modes of the Gaussian mixture are highly separated, our bounds are tight, and hence, our scheme attains the entire distortion region. This optimal region exceeds the region attained by separating source and channel coding, although it does not contain the "ideal" point (D/sub 1/, D/sub 2/)=(R/sup -1/(C/sub 1/), R/sup -1/(C/sub 2/)).

Z. Reznic

Papers

Distortion Bounds for Broadcasting With Bandwidth Expansion

Distortion bounds for broadcasting with bandwidth expansion

Joint source-channel coding of a Gaussian mixture source over the Gaussian broadcast channel