Showing papers presented at "Information Theory Workshop in 2010"

Message passing algorithms for compressed sensing: I. motivation and construction

Abstract: In a recent paper, the authors proposed a new class of low-complexity iterative thresholding algorithms for reconstructing sparse signals from a small set of linear measurements [1]. The new algorithms are broadly referred to as AMP, for approximate message passing. This is the first of two conference papers describing the derivation of these algorithms, connection with the related literature, extensions of the original framework, and new empirical evidence. In particular, the present paper outlines the derivation of AMP from standard sum-product belief propagation, and its extension in several directions. We also discuss relations with formal calculations based on statistical mechanics methods.

Noisy network coding

Abstract: A new coding scheme for multicasting multiple sources over a general noisy network is presented. The scheme naturally extends both network coding over noiseless networks by Ahlswede, Cai, Li, and Yeung, and compress-forward coding for the relay channel by Cover-El Gamal to general discrete memoryless and Gaussian networks. The scheme also recovers as special cases the results on coding for wireless relay networks and deterministic networks by Avestimehr, Diggavi, and Tse, and coding for wireless erasure networks by Dana, Gowaikar, Palanki, Hassibi, and Effros. The key idea is to use block Markov message repetition coding and simultaneous decoding. Instead of sending multiple independent messages over several blocks and decoding them sequentially as in previous relaying schemes, the same message is sent multiple times using independent codebooks and the decoder performs joint typicality decoding on the received signals from all the blocks without explicitly decoding the compression indices. New results on semideterministic relay networks and Gaussian networks demonstrate the potential of noisy network coding as a robust and scalable scheme for communication over wireless networks.

Message passing algorithms for compressed sensing: II. analysis and validation

Abstract: In a recent paper, the authors proposed a new class of low-complexity iterative thresholding algorithms for reconstructing sparse signals from a small set of linear measurements [1]. The new algorithms are broadly referred to as AMP, for approximate message passing. This is the second of two conference papers describing the derivation of these algorithms, connection with related literature, extensions of original framework, and new empirical evidence. This paper describes the state evolution formalism for analyzing these algorithms, and some of the conclusions that can be drawn from this formalism. We carried out extensive numerical simulations to confirm these predictions. We present here a few representative results.

Improved large-MIMO detection based on damped belief propagation

Abstract: In this paper, we consider the application of belief propagation (BP) to achieve near-optimal signal detection in large multiple-input multiple-output (MIMO) systems at low complexities. Large-MIMO architectures based on spatial multiplexing (V-BLAST) as well as non-orthogonal space-time block codes (STBC) from cyclic division algebra (CDA) are considered. We adopt graphical models based on Markov random fields (MRF) and factor graphs (FG). In the MRF based approach, we use pairwise compatibility functions although the graphical models of MIMO systems are fully/densely connected. In the FG approach, we employ a Gaussian approximation (GA) of the multi-antenna interference, which significantly reduces the complexity while achieving very good performance for large dimensions. We show that i) both MRF and FG based BP approaches exhibit large-system behavior, where increasingly closer to optimal performance is achieved with increasing number of dimensions, and ii) damping of messages/beliefs significantly improves the bit error performance.

On coding for cooperative data exchange

Abstract: We consider the problem of data exchange by a group of closely-located wireless nodes. In this problem each node holds a set of packets and needs to obtain all the packets held by other nodes. Each of the nodes can broadcast the packets in its possession (or a combination thereof) via a noiseless broadcast channel of capacity one packet per channel use. The goal is to minimize the total number of transmissions needed to satisfy the demands of all the nodes, assuming that they can cooperate with each other and are fully aware of the packet sets available to other nodes. This problem arises in several practical settings, such as peer-to-peer systems and wireless data broadcast. In this paper, we establish upper and lower bounds on the optimal number of transmissions and present an efficient algorithm with provable performance guarantees. The effectiveness of our algorithms is established through numerical simulations.

On LP decoding of polar codes

Abstract: Polar codes are the first codes to provably achieve capacity on the symmetric binary-input discrete memoryless channel (B-DMC) with low encoding and decoding complexity. The parity check matrix of polar codes is high-density and we show that linear program (LP) decoding fails on the fundamental polytope of the parity check matrix. The recursive structure of the code permits a sparse factor graph representation. We define a new polytope based on the fundamental polytope of the sparse graph representation. This new polytope P is defined in a space of dimension O(N logN) where N is the block length. We prove that the projection of P in the original space is tighter than the fundamental polytope based on the parity check matrix. The LP decoder over P obtains the ML-certificate property. In the case of the binary erasure channel (BEC), the new LP decoder is equivalent to the belief propagation (BP) decoder operating on the sparse factor graph representation, and hence achieves capacity. Simulation results of SC (successive cancelation) decoding, LP decoding over tightened polytopes, and (ML) maximum likelihood decoding are provided. For channels other than the BEC, we discuss why LP decoding over P with a linear objective function is insufficient.

Explicit codes minimizing repair bandwidth for distributed storage

Abstract: We consider the problem of minimizing the bandwidth required to repair a failed node when data is stored across n nodes in a distributed manner, so as to facilitate reconstruction of the entire data by connecting to any k out of the n nodes. We provide explicit and optimal constructions which permit exact replication of a failed systematic node.

Secrecy-achieving polar-coding

Abstract: A polar coding scheme is suggested for the binary-input memoryless symmetric and degraded wire-tap channel. The provided scheme achieves the entire rate-equivocation region for the considered model.

Non-binary polar codes using Reed-Solomon codes and algebraic geometry codes

Abstract: Polar codes, introduced by Arikan, achieve symmetric capacity of any discrete memoryless channels under low encoding and decoding complexity. Recently, non-binary polar codes have been investigated. In this paper, we calculate error probability of non-binary polar codes constructed on the basis of Reed-Solomon matrices by numerical simulations. It is confirmed that 4-ary polar codes have significantly better performance than binary polar codes on binary-input AWGN channel. We also discuss an interpretation of polar codes in terms of algebraic geometry codes, and further show that polar codes using Hermitian codes have asymptotically good performance.

Coupled graphical models and their thresholds

Abstract: The excellent performance of convolutional low-density parity-check codes is the result of the spatial coupling of individual underlying codes across a window of growing size, but much smaller than the length of the individual codes. Remarkably, the belief-propagation threshold of the coupled ensemble is boosted to the maximum-a-posteriori one of the individual system. We investigate the generality of this phenomenon beyond coding theory: we couple general graphical models into a one-dimensional chain of large individual systems. For the later we take the Curie-Weiss, random field Curie-Weiss, If-satisfiability, and Q-coloring models. We always find, based on analytical as well as numerical calculations, that the message passing thresholds of the coupled systems come very close to the static ones of the individual models. The remarkable properties of convolutional low-density parity-check codes are a manifestation of this very general phenomenon.

Multiple Access Wiretap channels with strong secrecy

Abstract: The problem of secure communication over Multiple-Access Wiretap channel (MAC-WTC) under strong secrecy criterion is investigated. A new technique based on channel output statistics approximation is developed for establishing the strong security over multi-user channels. In particular, this technique shows that how simple wiretap coding results in secure communication under strong secrecy criterion instead of weak secrecy criterion. As a side result of the paper, two results on the output statistics of MAC are provided. Such results can be used to approximate the mutual information between input and output of MAC with respect to a given codebook of arbitrary rate.

Overlapped Chunked network coding

Abstract: Network coding is known to improve the throughput and the resilience to losses in most network scenarios. In a practical network scenario, however, the accurate modeling of the traffic is often too complex and/or infeasible. The goal is thus to design codes that perform close to the capacity of any network (with arbitrary traffic) efficiently. In this context, random linear network codes are known to be capacity-achieving while requiring a decoding complexity quadratic in the message length. Chunked Codes (CC) were proposed by Maymounkov et al. to improve the computational efficiency of random codes by partitioning the message into a number of non-overlapping chunks. CC can also be capacity-achieving but have a lower encoding/decoding complexity at the expense of slower convergence to the capacity. In this paper, we propose and analyze a generalized version of CC called Overlapped Chunked Codes (OCC) in which chunks are allowed to overlap.1 Our theoretical analysis and simulation results show that compared to CC, OCC can achieve the capacity with a faster speed while maintaining almost the same advantage in computational efficiency.

Non-systematic codes for physical layer security

Abstract: This paper is a first study on the usage of non-systematic codes based on scrambling matrices for physical layer security. The chance of implementing transmission security at the physical layer is known since many years, but it is now gaining an increasing interest due to its several possible applications. It has been shown that channel coding techniques can be effectively exploited for designing physical layer security schemes, in such a way that an unauthorized receiver, experiencing a channel different from that of the authorized receiver, is not able to gather any information. Recently, it has been proposed to exploit puncturing techniques in order to reduce the security gap between the authorized and unauthorized channels. In this paper, we show that the security gap can be further reduced by using non-systematic codes, able to scramble information bits within the transmitted codeword.

A survey of reed-muller codes from polar coding perspective

Abstract: A survey of Reed-Muller (RM) coding is given with the goal of establishing a continuity between RM codes and polar codes. The focus is mainly on recursive decoding methods for RM codes and other ideas that are most relevant to polar coding.

Orbit codes — A new concept in the area of network coding

Abstract: We introduce a new class of constant dimension codes called orbit codes. The basic properties of these codes are derived. It will be shown that many of the known families of constant dimension codes in the literature are actually orbit codes.

Optimal mappings for joint source channel coding

Abstract: What is the optimal source-channel communication system for a given finite block length? The problem of obtaining the vector transformations that optimally map between the m-dimensional source space and the k-dimensional channel space is considered under a given channel power constraint and mean square error distortion measure. Closed form necessary conditions for optimality of the encoder and decoder mappings are derived. The optimal mappings are obtained using an iterative algorithm that updates encoder and decoder mappings according to optimality conditions at each iteration. Such mappings are used in a practical analog joint source channel system that transmits a continuous alphabet discrete time source over a noisy channel. Numerical results are presented for several source-channel distributions and it is shown that the optimal mappings outperform the previous heuristic mappings for both bandwidth expansion and compression.

Sensor network localization from local connectivity: Performance analysis for the MDS-MAP algorithm

Abstract: Sensor localization from only connectivity information is a highly challenging problem. To this end, our result for the first time establishes an analytic bound on the performance of the popular MDS-MAP algorithm based on multidimensional scaling. For a network consisting of n sensors positioned randomly on a unit square and a given radio range r = o(1), we show that resulting error is bounded, decreasing at a rate that is inversely proportional to r, when only connectivity information is given. The same bound holds for the range-based model, when we have an approximate measurements for the distances, and the same algorithm can be applied without any modification.

Wiretap channel with shared key

Abstract: This paper studies the problem of secure communication over a wiretap channel where the transmitter and the legitimate receiver share a secret key, which is concealed from the eavesdropper. We find the secrecy capacity under this scenario. This result generalizes that of Yamamoto, which is applicable only to less noisy wiretap channels, to the general wiretap channel when no distortion is allowed at the legitimate receiver.

Polar codes for the two-user binary-input multiple-access channel

Abstract: Arikan's polar coding method is extended to two-user multiple-access channels. It is shown that if the two users of the channel use the Arikan construction, the resulting channels will polarize to one of five possible extremals, on each of which uncoded transmission is optimal. The sum rate achieved by this coding technique is the one that corresponds to uniform input distributions. The encoding and decoding complexities and the error performance of these codes are as in the single-user case: O(n log n) for encoding and decoding, and o(exp(−n½−∈)) for block error probability, where n is the block length.

Windowed erasure decoding of LDPC Convolutional Codes

Abstract: We consider windowed decoding of LDPC Convolutional Codes on the Binary Erasure Channel (BEC) to study the trade-off between the decoding latency and the code performance. We make some key observations about regular LDPC Convolutional code ensembles under windowed decoding and give modified constructions of these codes that allow us to efficiently trade-off performance for gains in latency.

Fading channels with 1-bit output quantization: Optimal modulation, ergodic capacity and outage probability

Abstract: The achievable rate of communications systems depends on the quantization resolution at the receiver. Earlier work has shown that the capacity of real-valued AWGN channels with 1-bit output quantization is achieved with BPSK. This paper studies optimal modulation schemes, the ergodic capacity and the outage probability for complex-valued fading channels with 1-bit output quantization, assuming full channel knowledge at the receiver. It is shown that circular symmetry with at most one amplitude per phase is a necessary condition for optimal modulation. Circular-symmetric PSK achieves the ergodic capacity in case of Rayleigh fading. Considering the outage probability for Rayleigh fading, L-PSK with large L shows the best performance among conventional modulation schemes.

Strong secrecy for erasure wiretap channels

Abstract: We show that duals of certain low-density parity-check (LDPC) codes, when used in a standard coset coding scheme, provide strong secrecy over the binary erasure wiretap channel (BEWC). This result hinges on a stopping set analysis of ensembles of LDPC codes with block length n and girth ⋛ for some ⋛. We show that if the minimum left degree of the ensemble is l min , the expected probability of block error is O(1/n⌈l mink/2⌉ −k) when the erasure probability ∊ ef , where ∊ ef depends on the degree distribution of the ensemble. As long as l min and k > 2, the dual of this LDPC code provides strong secrecy over a BEWC of erasure probability greater than 1–∊ ef .

A renormalization group decoding algorithm for topological quantum codes

Abstract: Topological quantum error-correcting codes are defined by geometrically local checks on a two-dimensional lattice of quantum bits (qubits), making them particularly well suited for fault-tolerant quantum information processing. Here, we present a decoding algorithm for topological codes that is faster than previously known algorithms and applies to a wider class of topo-logical codes. Our algorithm makes use of two methods inspired from statistical physics: renormalization groups and mean-field approximations. First, the topological code is approximated by a concatenated block code that can be efficiently decoded. To improve this approximation, additional consistency conditions are imposed between the blocks, and are solved by a belief propagation algorithm.

Fast-group-decodable space-time block code

Abstract: To make the implementation of high-rate space-time block code (STBC) realistic in practical systems, fast-decodable (FD) and group-decodable (GD) code structures have been separately introduced into STBC to reduce the sphere decoding complexity. However, no STBC has both the two code structures until now. In this paper, high-rate fast-group-decodable STBC (FGD-STBC, which has both FD and GD code structures) is proposed for the first time. We first derive the condition for FD code structure with the lowest sphere decoding complexity, then we prove that such FD code structure can be integrated with the GD-STBC for FGD-STBC construction. Analysis and simulation show that the proposed FGD-STBC has much lower decoding complexity and comparable performance with the existing FDSTBC.

Rate compatible protocol for information reconciliation: An application to QKD

Abstract: Information Reconciliation is a mechanism that allows to weed out the discrepancies between two correlated variables. It is an essential component in every key agreement protocol where the key has to be transmitted through a noisy channel. The typical case is in the satellite scenario described by Maurer in the early 90's. Recently the need has arisen in relation with Quantum Key Distribution (QKD) protocols, where it is very important not to reveal unnecessary information in order to maximize the shared key length. In this paper we present an information reconciliation protocol based on a rate compatible construction of Low Density Parity Check codes. Our protocol improves the efficiency of the reconciliation for the whole range of error rates in the discrete variable QKD context. Its adaptability together with its low interactivity makes it specially well suited for QKD reconciliation.

Efficient two-write WOM-codes

Abstract: A Write Once Memory (WOM) is a storage medium with binary memory elements, called cells, that can change from the zero state to the one state only once. Examples of WOMs are punch cards, optical disks, and more recently flash memories. A t-write WOM-code is a coding scheme for storing t messages in n cells in such a way that each cell can change its value only from the zero state to the one state. The WOM-rate of a t-write WOM-code is the ratio of the total amount of information written to the WOM in t writes to the number of cells. In this paper we present a family of 2-write WOM-codes. It is shown how to construct from each linear code C a 2-write WOM-code. Then, we find 2-write WOM-codes that improve the best known WOM-rate with two writes. This scheme is proved to be capacity achieving when the parity check matrix of the linear code C is chosen uniformly at random. Finally, we show how to take advantage of 2-write WOM-codes in order to construct codes for the Blackwell channel.

Polar codes for Slepian-Wolf, Wyner-Ziv, and Gelfand-Pinsker

Abstract: Polar codes, combined with successive cancellation algorithms, are known to be asymptotically optimal for both the channel as well as the lossy source coding problem. The complexity of the encoding and the decoding algorithm in both cases is O(N log(N)), where N is the blocklength of the code. We show that polar codes also achieve optimum performance for the Slepian-Wolf, the Wyner-Ziv, and the Gelfand-Pinsker problem. The optimality of polar codes for these scenarios rests on the fact that polar codes are optimal for both the channel and the lossy source coding problems. Our results extend to general versions of these problems.

Bit-interleaved coded modulation with shaping

Abstract: The performance of bit-interleaved coded modulation (BICM) with shaping (i.e., non-equiprobable bit probabilities) is studied. For the AWGN channel, the rates achievable with BICM and shaping are practically identical to those of coded modulation or multilevel coding, virtually closing the gap that made BICM suboptimal in terms of information rates.

Information-theoretical analysis of private content identification

Abstract: In recent years, content identification based on digital fingerprinting attracts a lot of attention in different emerging applications. At the same time, the theoretical analysis of digital fingerprinting systems for finite length case remains an open issue. Additionally, privacy leaks caused by fingerprint storage, distribution and sharing in a public domain via third party outsourced services cause certain concerns in the cryptographic community. In this paper, we perform an information-theoretic analysis of finite length digital fingerprinting systems in a private content identification setup and reveal certain connections between fingerprint based content identification and Forney's erasure/list decoding [1]. Along this analysis, we also consider complexity issues of fast content identification in large databases on remote untrusted servers.

Coding against myopic adversaries

Abstract: A variant on the arbitrarily varying channel (AVC) is proposed in which the jammer is allowed to base its actions on a noisy version of the transmitted codeword. It is shown via a random coding argument that the capacity is the minimum over all discrete memoryless channels (DMCs) that can be induced by memoryless strategies of the adversary. This generalizes two existing models in the AVC literature: the standard AVC in which the jammer does not know the channel input, and the AVC in which the jammer knows the channel input exactly.