Showing papers presented at "Information Theory Workshop in 2008"

Successive Refinement Via Broadcast: Optimizing Expected Distortion of a Gaussian Source Over a Gaussian Fading Channel

Abstract: We consider the problem of transmitting a Gaussian source on a slowly fading Gaussian channel, subject to the mean-squared error distortion measure. The channel state information is known only at the receiver but not at the transmitter. The source is assumed to be encoded in a successive refinement (SR) manner, and then transmitted over the channel using the broadcast strategy. In order to minimize the expected distortion at the receiver, optimal power allocation is essential. We propose an efficient algorithm to compute the optimal solution in linear time , when the total number of possible discrete fading states. Moreover, we provide a derivation of the optimal power allocation when the fading state is a continuum, using the classical variational method. The proposed algorithm as well as the continuous solution is based on an alternative representation of the capacity region of the Gaussian broadcast channel.

On the entropy of sums

Abstract: It is shown that the entropy of a sum of independent random vectors is a submodular set function, and upper bounds on the entropy of sums are obtained as a result in both discrete and continuous settings. These inequalities complement the lower bounds provided by the entropy power inequalities of Madiman and Barron (2007). As applications, new inequalities for the determinants of sums of positive-definite matrices are presented.

Secure lossless compression with side information

Abstract: Secure data compression in the presence of side information at both a legitimate receiver and an eavesdropper is explored. A noise-free, limited rate link between the source and the receiver, whose output can be perfectly observed by the eavesdropper, is assumed. As opposed to the wiretap channel model, in which secure communication can be established by exploiting the noise in the channel, here the existence of side information at the receiver is used. Both coded and uncoded side information are considered. In the coded side information scenario, inner and outer bounds on the compression-equivocation rate region are given. In the uncoded side information scenario, the availability of the legitimate receiverpsilas and the eavesdropperpsilas side information at the encoder is considered, and the compression-equivocation rate region is characterized for these cases. It is shown that the side information at the encoder can increase the equivocation rate at the eavesdropper. Hence, the side information at the encoder is shown to be useful in terms of security; this is in contrast with the pure lossless data compression case where side information at the encoder would not help.

Relay strategies for interference-forwarding

Abstract: We consider relaying strategies in networks with multiple source-destination pairs and possibly additional outside sources of interference. We study these networks in the discrete, memoryless setup, and focus on relaying strategies based on forwarding the interference. In particular, the relay encodes the interference signal so as to make it easier for the receiver to remove it. The objective is to help receivers with weak interference by making the interference strong enough so that these receivers are able to cancel it completely. Our proposed approach is a combination of ideas from decode-and-forward (DF) and/or estimate-and-forward (EF) but applied to the interfering signal rather than the desired signal. When based only on DF, the relay first decodes (part of) the interfering signal it wants to enhance. It then encodes the interference in such a way as to increase the interference at the assisted receiver. The rate of the relayed interference is not limited by the rate from the relay to the original destination of the forwarded message, thus, interference cancellation is not a by-product of enhancing the desired information at its intended destination, but a goal in itself. We call this method interference-forwarding (IF). IF can also be based on EF where, instead of forwarding the exact interfering signal, the relay simply sends a compressed version of it to the assisted receiver. Rate increase can thus be obtained even if the signal received at the relay is independent of the desired message and consists only of interference and noise.

Additive extensions of a quantum channel

Abstract: We study extensions of a quantum channel whose one-way capacities are described by a single-letter formula. This provides a simple technique for generating powerful upper bounds on the capacities of a general quantum channel. We apply this technique to two qubit channels of particular interest-the depolarizing channel and the channel with independent phase and amplitude noise. Our study of the latter demonstrates that the key rate of BB84 with one-way post-processing and quantum bit error rate q cannot exceed H(1/2-2q(1-q))-H(2q(1-q)).

Relaying a fountain code across multiple nodes

Abstract: Fountain codes are designed for erasure channels, and are particularly well suited for broadcast applications from a single source to its one hop receivers. In this context, the problem of designing a rateless code in the network case for on-the-fly recoding is very important, as relaying the data over multiple nodes is fundamentally useful in a network. Clearly, fountain codes are unsuited for on-line recoding (and simply forwarding over subsequent hops is provably suboptimal). Random linear codes are throughput optimal, but they do not enjoy the low complexity that is a prime feature of fountain codes. Can we get the low complexity of say, LT codes, while maintaining on-the-fly recoding and being throughput optimal? This paper proposes a novel solution to the above question. We consider packet level coding on a line network of discrete memoryless erasure channels (with potentially unlimited nodes), and exhibhit a coding scheme with (1) tatelessness (2) logarithmic per-symbol coding complexity (3) throughput optimality (achieves rates equal the min cut capacity) and (4) avoids the delay of having to decode and then re-encode entire block lengths at intermediate nodes.

Interference alignment on the deterministic channel and application to fully connected AWGN interference networks

Abstract: An interference alignment example is constructed for the deterministic channel model of the K user interference channel. The deterministic channel example is then translated into the Gaussian setting, creating the first known example of a fully connected Gaussian K user interference network with single antenna nodes, real, non-zero and constant channel coefficients, and no propagation delays where the degrees of freedom outerbound is achieved. An analogy is drawn between the propagation delay based interference alignment examples and the deterministic channel model which also allows similar constructions for the 2 user X channel as well.

On the algebraic structure of the Silver code: A 2 × 2 perfect space-time block code

Abstract: Recently, a family of full-rate, full-diversity space-time block codes (STBCs) for 2 times 2 multiple-input multiple-output (MIMO) channels was proposed in the works of Tirkkonen et al., using a combination of Clifford algebra and Alamouti structures, namely twisted space-time transmit diversity code. This family was recently rediscovered by Paredes et al., and they pointed out that such STBCs enable reduced-complexity maximum-likelihood (ML) decoding. Independently, the same STBCs were found in the work of Samuel and Fitz (2007) and named multi-strata space-time codes. In this paper we show how this code can be constructed algebraically from a particular cyclic division algebra (CDA). This formulation enables to prove that the code has the non-vanishing determinant (NVD) property and hence achieves the diversity-multiplexing tradeoff (DMT) optimality. The fact that the normalized minimum determinant is 1/radic(7) places this code in the second position with respect to the golden code, which exhibits a minimum determinant of 1/radic(5), and motivates the name silver code.

Decode-and-Forward Relaying With Quantized Channel State Feedback: An Outage Exponent Analysis

Abstract: The problem of resource allocation to maximize the outage exponent over a fading relay channel using the decode-and-forward protocol with quantized channel state feedback (CSF) is studied. Three different scenarios are considered: relay-to-source, destination-to-relay, and destination-to-source-and-relay CSF. In the relay-to-source CSF scenario, it is found that using merely one bit of CSF to control the source transmit power is sufficient to achieve the multiantenna upper bound in a range of multiplexing gains. In the destination-to-relay CSF scenario, the systems slightly outperform dynamic decode-and-forward (DDF) at high multiplexing gains, even with only one bit of feedback. Finally, in the destination-to-source-and-relay CSF scenario, if the source-relay channel gain is unknown to the feedback quantizer at the destination, the diversity gain only grows linearly in the number of feedback levels, in sharp contrast to an exponential growth for multiantenna channels. In this last scenario, a simple scheme is shown to perform close to the corresponding upper bound.

Secret-sharing schemes based on self-dual codes

Abstract: Secret sharing is an important topic in cryptography and has applications in information security. We use self-dual codes to construct secret-sharing schemes. We use combinatorial properties and invariant theory to understand the access structure of these secret-sharing schemes. We describe two techniques to determine the access structure of the scheme, the first arising from design properties in codes and the second from the Jacobi weight enumerator, and invariant theory.

Interference-assisted secret communication

Abstract: Wireless communication is susceptible to adversarial eavesdropping due to the broadcast nature of the wireless medium. In this paper it is shown how eavesdropping can be alleviated by exploiting the superposition property of the wireless medium. A wiretap channel with a helping interferer (WT-HI), in which a transmitter sends a confidential message to its intended receiver in the presence of a passive eavesdropper, and with the help of an independent interferer, is considered. The interferer, which does not know the confidential message, helps in ensuring the secrecy of the message by sending independent signals. An achievable secrecy rate for the WT-HI is given. The results show that interference can be exploited to assist secrecy in wireless communications. An important example of the Gaussian case, in which the interferer has a better channel to the intended receiver than to the eavesdropper, is considered. In this situation, the interferer can send a (random) codeword at a rate that ensures that it can be decoded and subtracted from the received signal by the intended receiver but cannot be decoded by the eavesdropper. Hence, only the eavesdropper is interfered with and the secrecy level of the confidential message is increased.

Broadcast gossip algorithms

Abstract: Motivated by applications to wireless sensor, peer-to-peer, and ad hoc networks, we study distributed broadcasting algorithms for exchanging information and for computing in an arbitrarily connected network of nodes. Specifically, we propose a broadcasting-based gossiping algorithm to compute the (possibly weighted) average of the initial measurements of the nodes at every node in the network. We show that the broadcast gossip algorithms almost surely converge to a consensus. In addition, the random consensus value is, in expectation, equal to the desired value, i.e., the average of initial node measurements. However, the broadcast gossip algorithms do not converge to the initial average in absolute sense because of the fact that the sum is not preserved at every iteration. We provide theoretical results on the mean square error performance of the broadcast gossip algorithms. The results indicate that the mean square error strictly decreases through iterations until the consensus is achieved. Finally, we assess and compare the communication cost of the broadcast gossip algorithms required to achieve a given distance to consensus through numerical simulations.

Achievable rates for the restricted half-duplex two-way relay channel under a partial-decode-and-forward protocol

Abstract: In this paper, we state a new achievable rate region for the two-phase two-way relay channel with a half-duplex relay node. The new region is obtained using a partial-decode-and-forward protocol, which is a superposition of both, decode-and-forward and compress-and-forward. It contains the achievable rate regions according to Oechtering et al. (2008) and Schmurr et al. (2007) as special cases.

Cooperative relaying with state available at the relay

Abstract: We consider a state-dependent full-duplex relay channel with the state of the channel non-causally available at only the relay. In the framework of cooperative wireless networks, some specific terminals can be equipped with cognition capabilities, i.e, the relay in our model. In the discrete memoryless (DM) case, we derive lower and upper bounds on channel capacity. The lower bound is obtained by a coding scheme at the relay that consists in a combination of codeword splitting, Gelpsilafand-Pinsker binning, and a decode-and-forward scheme. The upper bound is better than that obtained by assuming the availability of state at the source, the relay, and the destination. For the Gaussian case, we also derive lower and upper bounds on channel capacity. The lower bound, obtained by a coding scheme based on combination of codeword splitting and generalized dirty paper coding, is tight in some cases if the channel is physically degraded. The upper bound is also better than that obtained by assuming that the channel state is available at the source, the relay, and the destination.

Nonclassicality without entanglement enables bit commitment

Abstract: We investigate the existence of secure bit commitment protocols in the convex framework for probabilistic theories. The theory makes only minimal assumptions, and can be used to formalize quantum theory, classical probability theory, and a host of other possibilities. We prove that in all such theories that are locally non-classical but do not have entanglement, there exists a bit commitment protocol that is exponentially secure in the number of systems used.

A lower bound on the probability of error in quantum state discrimination

Abstract: We give a lower bound on the probability of error in quantum state discrimination. The bound is a weighted sum of the pairwise fidelities of the states to be distinguished.

On hypotheses testing for ergodic processes

Abstract: We address three problems of statistical analysis of time series: goodness-of-fit (or identity) testing, process discrimination, and the change point problem. For each of the problems we construct a test that is asymptotically accurate for the case when the data is generated by stationary ergodic processes. All problems are solved in a similar way by using empirical estimates of the distributional distance between the processes.

Confidential messages to a cooperative relay

Abstract: We extend the broadcast channel with confidential messages to the situation where the receiver of the secret message also serves as a relay. We analyze the fundamental cooperation versus secrecy trade-offs for discrete memoryless channels and obtain the exact rate-equivocation region in this case. For the Gaussian channel, we consider various strategies leading to different levels of secrecy. Our study highlights the fundamental role of jamming as a means to increase secrecy rates, but also emphasizes the importance of carefully designed relaying strategies.

Capacity outer bounds for broadcast channels

Abstract: Outer bounds on the capacity region of broadcast channels are reviewed and a new outer bound is presented.

LDPC codes which can correct three errors under iterative decoding

Abstract: In this paper, we provide necessary and sufficient conditions for a column-weight-three LDPC code to correct all patterns up to three errors when decoded using Gallager A algorithm We then provide a construction technique which results in a code satisfying the above conditions We also provide numerical assessment of code performance via simulation results

Feedback control performance over a noisy communication channel

Abstract: We consider the problem of minimizing the variance in the output of a plant that is driven by a Gaussian disturbance using measurements of the plant output obtained from a Gaussian channel. For the special case in which only the variance at the terminal time is penalized, we derive an optimal linear time-varying communication and control strategy, and argue that nonlinear strategies cannot achieve better performance.

Practical codes for queueing channels: An algebraic, state-space, message-passing approach

Abstract: This paper examines more closely the probabilistic dynamics of queueing timing channels and discusses a new practical coding scheme which is tailored to them and approaches capacity. We consider using sparse graph coset codes over nonbinary finite fields. We use a shaping technique to map algebraic symbols to non-uniform codewords using the inverse cumulative distribution of a target random variable. We exploit the graphical structure of the conditional distribution of the departure process given the arrival process to arrive at a Forney factor graph of the joint likelihood that has graphical structure reminiscent of coding on inter-symbol interference channels with LDPC codes. We show through simulation that this technique, when using low-complexity iterative decoding, is capacity-approaching.

Non-additive quantum codes from Goethals and Preparata codes

Abstract: We extend the stabilizer formalism to a class of nonadditive quantum codes which are constructed from non-linear classical codes. As an example, we present infinite families of nonadditive codes which are derived from Goethals and Preparata codes.

Streaming algorithms for estimating entropy

Abstract: We give a method for estimating the empirical Shannon entropy of a distribution in the streaming model of computation. Our approach reduces this problem to the well-studied problem of estimating frequency moments. The analysis of our approach is based on new results which establish quantitative bounds on the rate of convergence of Renyi entropy towards Shannon entropy.

The rate loss of single letter characterization for the “dirty” multiple access channel

Abstract: For general memoryless systems, the typical information theoretic solution, when exists, has a ldquosingle-letterrdquo form. This reflects the fact that optimum performance can be approached by a random code (or a random binning scheme), generated using independent and identically distributed copies of some single-letter distribution. Is that the form of the solution of any (information theoretic) problem? In fact, some counter examples are known, perhaps the most famous being the Korner-Marton ldquotwo help onerdquo problem, where the modulo-two sum of two binary sources is to be decoded from their independent encodings. In this paper we provide another counter example, the ldquodoubly-dirtyrdquo multiple access channel (MAC). Like the Korner-Marton problem, this example is associated with a multiterminal scenario where side information is distributed among several terminals; each transmitter knows part of the channel interference but the receiver is not aware of any part of it. We give an explicit solution for the capacity region of a binary version of the doubly-dirty MAC, demonstrate how this capacity region can be approached using a linear coding scheme, and prove that the ldquobest known single-letter regionrdquo is strictly contained in it. We also state a conjecture regarding a similar rate loss of single letter characterization in the Gaussian case.

Optimal state estimation for discrete-time Markovian Jump Linear Systems, in the presence of delayed output observations

Abstract: In this paper, we investigate the design of optimal state estimators for Markovian jump linear systems. We consider that the state has two components: the first component is finite valued and is denoted as mode, while the second (continuous) component is in a finite dimensional Euclidean space. The continuous state is driven by a zero mean, white and Gaussian process noise. The observation output has two components: the first is the mode and the second is a linear combination of the continuous state observed and zero mean, white Gaussian noise. Both output components are affected by delays, not necessarily equal. Our paradigm is to design optimal estimators for the current state, given the current output observation. We provide a solution to this paradigm by giving a recursive estimator for the continuous state, in the minimum mean square sense, and a finitely parameterized recursive scheme for computing the probability mass function of the current mode conditioned on the observed output. We show that when the mode is observed with a greater delay then the continuous output component, the optimal estimator nonlinear in the observed outputs.

MDL, penalized likelihood, and statistical risk

Abstract: We determine, for both countable and uncountable collections of functions, information-theoretic conditions on a penalty pen(f) such that the optimizer f of the penalized log likelihood criterion log 1/likelihood(f)+pen(f) has risk not more than the index of resolvability corresponding to the accuracy of the optimizer of the expected value of the criterion. If F is the linear span of a dictionary of functions, traditional description-length penalties are based on the number of non-zero terms (the lscr0 norm of the coefficients). We specialize our general conclusions to show the lscr1 norm of the coefficients times a suitable multiplier lambda is also an information-theoretically valid penalty.

Compression-based methods for nonparametric density estimation, on-line prediction, regression and classification for time series

Abstract: We address the problem of nonparametric estimation of characteristics for stationary and ergodic time series. We consider finite-alphabet time series and the real-valued ones and the following problems: estimation of the (limiting) probability P(u0 hellip us) for every s and each sequence u0 hellip us of letters from the process alphabet (or estimation of the density p(x0, hellip , xs) for real-valued time series), so-called on-line prediction, where the conditional probability P(xt+1/x1x2 hellip xt) (or the conditional density p(xt+1/x1x2 hellip xt)) should be estimated (in the case where x1x2 hellip xt is known), regression and classification (or so-called problems with side information). We show that any universal code (or a universal data compressor) can be used as a basis for constructing asymptotically optimal methods for the above problems.

Successive cancellation for cyclic interference channels

Abstract: The Chong-Motani-Garg inner bound for the two-user interference channel is generalized to a special case of many-user interference channels. By taking a geometric viewpoint and using rate-splitting arguments, it is shown that all operating points in the Chong-Motani-Garg inner bound allow decoding by successive cancellation at all receivers.

Asymptotic capacity and optimal precoding strategy of multi-level precode & forward in correlated channels

Abstract: We analyze a multi-level MIMO relaying system where a multiple-antenna transmitter sends data to a multiple-antenna receiver through several relay levels, also equipped with multiple antennas Assuming correlated fading in each hop, each relay receives a faded version of the signal transmitted by the previous level, performs precoding on the received signal and retransmits it to the next level Using free probability theory and assuming that the noise power at the relay levels - but not at the receiver - is negligible, a closed-form expression of the end-to-end asymptotic instantaneous mutual information is derived as the number of antennas in all levels grow large with the same rate This asymptotic expression is shown to be independent from the channel realizations, to only depend on the channel statistics and to also serve as the asymptotic value of the end-to-end average mutual information We also provide the optimal singular vectors of the precoding matrices that maximize the asymptotic mutual information : the optimal transmit directions represented by the singular vectors of the precoding matrices are aligned on the eigenvectors of the channel correlation matrices, therefore they can be determined only using the known statistics of the channel matrices and do not depend on a particular channel realization