Showing papers presented at "Information Theory Workshop in 2014"

From the Information Bottleneck to the Privacy Funnel

Abstract: We focus on the disclosure-privacy trade-off encountered by users who wish to disclose some information to an analyst, that is correlated with their private data, in the hope of receiving some utility. To preserve privacy, user data is transformed before it is disclosed, according to a probabilistic privacy mapping. We formalize the disclosure-collateral setting, and provide a single letter characterization of the trade-off between disclosure and privacy. We then formulate the design of the privacy mapping as the Privacy Funnel optimization. This optimization problem being non-convex, we leverage connections to the information bottleneck method, to provide a greedy algorithm, and an alternating iteration algorithm, that are locally optimal, and that we evaluate on synthetic data. We then turn our attention to the case Gaussian user data, and provide a closedform privacy mapping that is optimal in the class of Gaussian mappings. Finally, we show how the Privacy Funnel relates to other privacy-utility frameworks, and justify the generality of the log-loss as an inference cost function for private data.

Reliable deniable communication with channel uncertainty

Abstract: Alice wishes to potentially communicate with Bob over a compound Binary Symmetric Channel while Willie listens in over a compound Binary Symmetric Channel that is noisier than Bob's. The channel noise parameters for both Bob and Willie are drawn according to uniform distribution over a range, but none of the three parties know their exact values. Willie's goal is to infer whether or not Alice is communicating with Bob. We show that Alice can send her messages reliably to Bob while ensuring that even whether or not she is actively communicating is deniable to Willie. We find the best rate at which Alice can communicate both deniably and reliably using Shannon's random coding and prove a converse.

S-AMP: Approximate Message Passing for General Matrix Ensembles

Abstract: We propose a novel iterative estimation algorithm for linear observation models called S-AMP. The fixed points of S-AMP are the stationary points of the exact Gibbs free energy under a set of (firstand second-) moment consistency constraints in the large system limit. S-AMP extends the approximate message-passing (AMP) algorithm to general matrix ensembles with a well-defined large system size limit. The generalization is based on the S-transform (in free probability) of the spectrum of the measurement matrix. Furthermore, we show that the optimality of S-AMP follows directly from its design rather than from solving a separate optimization problem as done for AMP.

A code design framework for multi-rack distributed storage

Abstract: In practical distributed storage networks, data centres house hundreds of racks, each of which contains several storage nodes. However, the majority of works in distributed storage assume a simple network model with a collection of identical storage nodes with same communication cost between the nodes. In this paper, we consider a more realistic rack model of storage network and present a code design framework for this model. Using our code construction method, node failures within a rack can be repaired locally by survived nodes in the same rack or by the other survived racks when the information content of the same rack is not sufficient to repair the failed nodes.

Wireless-powered cooperative communications via a hybrid relay

Abstract: In this paper, we consider a wireless-powered cooperative communication network, which consists of a hybrid access-point (AP), a hybrid relay, and an information source. In contrast to the conventional cooperative networks, the source in the considered network is assumed to have no embedded energy supply. Thus, it first needs to harvest energy from the signals broadcast by the AP and/or relay, which have constant power supply, in the downlink (DL) before transmitting the information to the AP in the uplink (UL). The hybrid relay can not only help to forward information in the UL but also charge the source with wireless energy transfer in the DL. Considering different possible operations of the hybrid relay, we propose two cooperative protocols for the considered network. We jointly optimize the time and power allocation for DL energy transfer and UL information transmission to maximize the system throughput of the proposed protocols. Numerical results are presented to compare the performance of the proposed protocols and illustrate the impacts of system parameters.

A matrix completion approach to linear index coding problem

Abstract: In this paper, a general algorithm is proposed for rate analysis and code design of linear index coding problems. Specifically a solution for minimum rank matrix completion problem over finite fields representing the linear index coding problem is devised in order to find the optimum transmission rate given vector length and size of the field. The new approach can be applied to both scalar and vector linear index coding. Keywords—Index Coding, Minimum Rank Matrix completion solution of a given rank, when a complete sub- matrix with the same rank is available and the matrix completion solution is unique. The approach relies on identifying the complete sub-matrix by exhaustive search and completing the incomplete rows (columns) in its row (column) subspace in an iterative manner and by search. In this paper, we propose a new algorithm for matrix completion with minimum rank over finite fields. In the new algorithm, first the complete sub-matrix of highest rank is identified using a heuristic scheme of polynomial complexity, and then row (column) projection is used to expand the complete sub-matrix iteratively. The goal of projection step is to find possible completions of an incomplete row or column such that they are in the span of the complete sub-matrix. During projection, block erasure channel decoding approaches are utilized for enhanced computational efficiency. The row and column projections are administered over a decision tree, which efficiently manages the completion in specific cases when multiple completion solutions are to be examined. This also accommodates the cases where the existing complete sub- matrix is of a smaller rank compared to the (possibly unknown) optimum rank of the prospective complete matrix. As we shall demonstrate, the proposed algorithm provides a solution to identify or approximate the optimum rate for a general linear index coding problem, and to design the associate coding scheme for a given vector (block) length and field size. The structure of this article is as follows. In Section II, the index coding problem and its associated minimum rank matrix completion problem over finite fields are elaborated. Section III presents the proposed algorithm for matrix completion and section IV presents the simulation results.

Syndrome decoding of symbol-pair codes

Abstract: Cassuto and Blaum proposed new error correcting codes which are called symbol-pair codes. They gave a coding framework for channels whose outputs are overlapping pairs of symbols in storage applications. It is called symbol-pair read channel. The pair distance and pair error are used in symbol-pair read channel. Cassuto et al. and showed Yaakobi et al. presented decoding algorithms for symbol-pair codes. However, their decoding algorithms cannot always correct errors whose number is not more than half the minimum pair distance. In this paper, we propose a new decoding algorithm using the syndrome of symbol-pair codes. In addition, we show that the proposed algorithm can correct all pair errors within the pair error correcting capability.

Reliable, deniable and hidable communication: A quick survey

Abstract: We survey here recent work pertaining to “deniable” communication - i.e., talking without being detected. We first highlight connections to other related notions (anonymity and secrecy). We then contrast the notions of deniability and secrecy. We highlight similarities and distinctions of deniability with a variety of related notions (LPD communications, stealth, channel resolvability) extant in the literature.

Correlation between channel state and information source with empirical coordination constraint

Abstract: Correlation between channel state and source symbol is under investigation for a joint source-channel coding problem. We investigate simultaneously the lossless transmission of information and the empirical coordination of channel inputs with the symbols of source and states. Empirical coordination is achievable if the sequences of source symbols, channel states, channel inputs and channel outputs are jointly typical for a target joint probability distribution. We characterize the joint distributions that are achievable under lossless decoding constraint. The performance of the coordination is evaluated by an objective function. For example, we determine the minimal distortion between symbols of source and channel inputs for lossless decoding. We show that the correlation source/channel state improves the feasibility of the transmission.

A new construction of zero correlation zone sequences from generalized reed-muller codes

Abstract: In this paper, we present a direct construction of zero-correlation zone (ZCZ) sequence sets (each associated with a graph) from the second order cosets of the first-order generalized Reed-Muller codes. This settles an open problem introduced by Rathinakumar and Chaturvedi in their 2008 paper.

Max-min Fair Wireless Energy Transfer for Secure Multiuser Communication Systems

Abstract: This paper considers max-min fairness for wireless energy transfer in a downlink multiuser communication system. Our resource allocation design maximizes the minimum harvested energy among multiple multiple-antenna energy harvesting receivers (potential eavesdroppers) while providing quality of service (QoS) for secure communication to multiple single-antenna information receivers. In particular, the algorithm design is formulated as a non-convex optimization problem which takes into account a minimum required signal-to-interference-plus-noise ratio (SINR) constraint at the information receivers and a constraint on the maximum tolerable channel capacity achieved by the energy harvesting receivers for a given transmit power budget. The proposed problem formulation exploits the dual use of artificial noise generation for facilitating efficient wireless energy transfer and secure communication. A semidefinite programming (SDP) relaxation approach is exploited to obtain a global optimal solution of the considered problem. Simulation results demonstrate the significant performance gain in harvested energy that is achieved by the proposed optimal scheme compared to two simple baseline schemes.

Generalized low-density (GLD) lattices

Abstract: We propose the construction of a new family of lattice sphere packings. Given a small-dimensional lattice, we start by building a first lattice in a large dimension by the direct sum of the small lattice. Then, the coordinates of the first large lattice are permuted to yield a second large-dimensional lattice. Finally, our generalized low-density (GLD) lattice is the intersection of the first and the second lattice. We restrict our construction in this paper to integer lattices. GLD lattices are the result of mixing classical lattice theory with modern coding theory. They are potential candidates not only for channel coding as coded modulations, but also for physical-layer network coding and for secure digital communications.

Recent advances in joint wireless energy and information transfer

Abstract: In this paper, we provide an overview of the recent advances in microwave-enabled wireless energy transfer (WET) technologies and their applications in wireless communications. Specifically, we divide our discussions into three parts. First, we introduce the state-of-the-art WET technologies and the signal processing techniques to maximize the energy transfer efficiency. Then, we discuss an interesting paradigm named simultaneous wireless information and power transfer (SWIPT), where energy and information are jointly transmitted using the same radio waveform. At last, we review the recent progress in wireless powered communication networks (WPCN), where wireless devices communicate using the power harvested by means of WET. Extensions and future directions are also discussed in each of these areas.

Repeated deletion channels

Abstract: We consider the question of the capacity of the deletion channel when multiple copies of the codeword are sent independently through the channel. This type of coding problem is straightforward for standard erasure and error channels, but is more challenging in the deletion channel setting. Our results show that when the same codeword is sent k times through the channel, each time with independent deletions with probability d, the optimal communication rate under a random codebook is 1 − αH(dk) + O(dk) for an explicitly given constant α.

From LDPC to chunked network codes

Abstract: Chunked network code is a variation of random linear network code with low computational cost and small coefficient vector overhead. In a chunked network code, intermediate network nodes only apply network coding among packets of the same chunk. In this paper, we propose an approach to construct chunks using LDPC codes. For a given LDPC code, the chunks are simply formed by first partitioning the variable nodes into disjoint groups and then filling each group with a number of variable nodes of degree zero. The chunked network codes constructed using this approach are called L-chunked codes. We analyze the asymptotic achievable rates of L-chunked codes using belief propagation decoding for an arbitrary rank distribution of the chunk transfer matrices. Numerical evaluation shows that L-chunked codes achieve a rate very close to optimal.

Efficient secret sharing schemes achieving optimal information rate

Abstract: One of the important problems in secret sharing schemes is to establish bounds on the size of the shares to be given to participants in secret sharing schemes. The other important problem in secret sharing schemes is to reduce the computational complexity in both secret distribution phase and secret reconstruction phase. In this paper, we design efficient threshold (n, k) secret sharing schemes to achieve both of the above goals. In particular, we show that if the secret size |s| is larger than max{1 + log 2 n, n(n − k)/(n − 1)}, then ideal secret sharing schemes exist. In the efficient ideal secret sharing schemes that we will construct, only XOR-operations on binary strings are required (which is the best we could achieve). These schemes will have many applications both in practice and in theory. For example, they could be used to design very efficient verifiable secret sharing schemes which will have broad applications in secure multi-party computation and could be used to design efficient privacy preserving data storage in cloud systems.

Secrecy measures for broadcast channels with receiver side information: Joint vs individual

Abstract: We study the transmission of a common message and three confidential messages over a broadcast channel with two legitimate receivers and an eavesdropper. Each legitimate receiver is interested in decoding two of the three confidential messages, while having the third one as side information. In order to measure the ignorance of the eavesdropper about the confidential messages, we investigate two different secrecy criteria: joint secrecy and individual secrecy. For both criteria, we provide a general achievable rate region. We establish both the joint and individual secrecy capacity if the two legitimate receivers are less noisy than the eavesdropper. We further investigate the scenario where the eavesdropper is less noisy than the two legitimate receivers. It is known that the joint secrecy constraints can not be fulfilled under this scenario, however, we manage to establish a non vanishing capacity region for the individual secrecy case. ar and K¨ orner extended Wyner's result to the general broadcast channel (BC) with confidential messages. In (4), Kang and Liu generalized the previous two approaches by studying the presence of a shared secret key in the wiretap channel. They derived the secrecy capacity for this scenario by combining the wiretap coding principle along with Shannon's one-time pad idea. The problem of secure communication in BC with more than two receivers remains an open topic. In (5), Chia and El Gamal investigated the secrecy capacity for transmitting one common and one confidential message over a BC with two receivers and one eavesdropper. They provided a general achievable region and managed to establish the capacity when the two receivers are less noisy than the eavesdropper. In this paper, we study a related problem. However, in stead of having only one confidential message to both legitimate receivers, we extend the system such that each legitimate receiver has an extra individual confidential message. This extension allows us to differentiate between joint and individual secrecy. Our setup has an additional feature, that each legitimate receiver possesses the individual confidential message of the other one as side information as shown in Figure 1. In (6) the

Sequential decoding of polar codes with arbitrary binary kernel

Abstract: The problem of efficient soft-decision decoding of polar codes with any binary kernel is considered The proposed approach represents a generalization of the sequential decoding algorithm introduced recently for the case of polar codes with Arikan kernel Numeric results show that the proposed algorithm enables near-ML decoding of polar codes with BCH kernel

A new multiple folded successive cancellation decoder for polar codes.

Abstract: We consider a new variant of successive cancellation decoder (SCD) for polar codes based on the concept of folding, which was proposed in [1], [2] as technique to reduce the decoding latency at the cost of a higher computational complexity. In this paper, we first formally define the multiple folding operation (iterated κ times), which decomposes the original encoding graph into a number of smaller polar encoding graphs. More specifically, we show that the multiple folding gives rise to a two stage interpretation of the graph representing the polar encoder and the SCD. Based on this, we propose the improved multiple folded successive cancellation decoder (IMFSCD), which combines SCD in one stage and maximum-likelihood decoding in the other. This decoder exhibits a latency gain by a factor of 2κ, still retaining a complexity close to the classic SCD. The small increase in complexity is due to a short maximum likelihood decoder (MLD) used in place of a SCD in the last decoding stage within the IMFSCD. Moreover, we observe by simulation that the decoder performance is exactly the same as that of an SCD at all rates.

Low-complexity channel resolvability codes for the symmetric multiple-access channel

Abstract: We investigate channel resolvability for the l-user multiple-access channel (MAC) with two different families of encoders. The first family consists of invertible extractors, while the second one consists of injective group homomorphisms, and was introduced by Hayashi for the point-to-point channel resolvability. The main benefit of these two families is to provide explicit low-complexity channel resolvability codes in the case of symmetric MACs. Specifically, we provide two examples of families of invertible extractors suitable for MAC resolvability with uniform input distributions, one based on finite-field multiplication, which can be implemented in O(n logn) for a limited range of values of the encoding blocklength n, and a second based on modified Toeplitz matrices, which can be implemented in O(n logn) for a wider range of values of n. We also provide an example of family of injective group homomorphisms based on finite-field multiplication suitable for MAC resolvability with uniform input distributions, which can be implemented in O(n logn) for some values of n.

Strong coordination over a three-terminal relay network

Abstract: We study the problem of strong coordination in a three-terminal relay network, in which agents communicate to ensure that their actions follow a joint behavior specified by a prescribed joint distribution of actions. The model unifies several coordination schemes, including line and broadcast coordination. We derive several inner bounds to the strong capacity region; in particular, we prove the achievability of a subset of coordination rate-tuples, which provides insight into the relative performance of line, broadcast, and relay coordination.

Decoding delay reduction in network coded cooperative systems with intermittent status update

Abstract: In this paper, we study the problem of decoding delay reduction for instantly decodable network coding (IDNC) in broadcast cooperative systems, where a group of closely located clients cooperate with each other to obtain their missing packets. In such cooperative systems, one of the clients (referred to as the leader) decides the transmitting client and the packet combination for each transmission. We consider intermittent system status update (SSU) at the leader such that all other clients feed back their packet reception status to the leader after several cooperative transmissions. We first introduce an intermittent local IDNC (IL-IDNC) graph to represent all potential packet combinations for a transmitting client. We then formulate the joint client and packet selection problem that results in the minimum expected decoding delay in each cooperative transmission as a maximum weight clique problem over all the IL-IDNC graphs. Since solving the formulated problem is computationally complex, we propose a heuristic algorithm to select the transmitting client and the packet combination that can reduce the decoding delay. Simulation results show that the proposed heuristic algorithm can achieve a tolerable degradation compared to the full SSU performance while using a smaller number of SSUs.

Fundamental Limits of Universal Lossless One-to-One Compression of Parametric Sources

Abstract: In this paper, the problem of universal lossless one-to-one compression (without prefix constraint) is studied. A converse bound is obtained on the average minimax (and maximin) redundancy that shows the redundancy is at least (d 2)=2logn+O(1) for the universal compression of a sequence of length n from a d-dimensional parametric source. Further, the type-size coding strategy is shown to be minimax optimal up to o(logn) for the class of memoryless sources, achieving the converse leading to characterization of the fundamental performance limit of universal compression for memoryless sources. Finally, through a numerical example, our results imply that the reduction on the codeword length due to relaxing the prefix constraint is negligible when compared to the cost of universality.

State amplification and state masking for the binary energy harvesting channel

Abstract: In this paper, we consider a binary energy har- vesting transmitter that wishes to control the amount of side information the receiver can obtain about its energy harvests. Specifically, we study state amplification and state masking, which define the maximum and minimum amount of state information conveyed to the receiver for a given message rate, re- spectively. For an independent and identically distributed energy harvesting process, we first find the amplification and masking regions for a transmitter without a battery and a transmitter with an infinite battery. Next, we find inner bounds for these regions for a unit-sized battery at the transmitter using two different encoding schemes, using instantaneous Shannon strategies and using a scheme based on the equivalent timing channel introduced in our previous work. We observe that the former provides better state amplification, while the latter provides better state masking. I. INTRODUCTION

Quantum synchronizable codes from quadratic residue codes and their supercodes

Abstract: Quantum synchronizable codes are quantum error-correcting codes designed to correct the effects of both quantum noise and block synchronization errors. While it is known that quantum synchronizable codes can be constructed from cyclic codes that satisfy special properties, only a few classes of cyclic codes have been proved to give promising quantum synchronizable codes. In this paper, using quadratic residue codes and their supercodes, we give a simple construction for quantum synchronizable codes whose synchronization capabilities attain the upper bound. The method is applicable to cyclic codes of prime length.

Almost affine locally repairable codes and matroid theory

Abstract: In this paper we provide a link between matroid theory and locally repairable codes (LRCs) that are almost affine. The parameters (n, k, d, r) of LRCs are generalized to matroids. A bound on the parameters (n, k, d, r), similar to the bound in [P. Gopalan et al., “On the locality of codeword symbols,” IEEE Trans. Inf. Theory] for linear LRCs, is given for matroids. We prove that the given bound is not tight for a certain class of parameters, which implies a non-existence result for a certain class of optimal locally repairable almost affine codes. Constructions of optimal LRCs over small finite fields were stated as an open problem in [I. Tamo et al., “Optimal locally repairable codes and connections to matroid theory”, 2013 IEEE ISIT]. In this paper optimal LRCs which do not require a large field are constructed for certain classes of parameters.

Shaping low-density lattice codes using Voronoi integers

Abstract: A lattice code construction that employs two separate lattices, a high dimension lattice for coding gain and a low-dimension lattice for shaping gain, is described. Systematic lattice encoding is a method to encode an integer sequence to a lattice point that is nearby that integer sequence. We describe the “Voronoi integers” ℤm/Λ s , the set of integers inside the fundamental Voronoi region of a shaping lattice Λ s , and a concrete scheme to label these integers. By first shaping the information using the Voronoi integers in low dimension, and then performing systematic lattice encoding using a high-dimension lattice, good shaping and coding gains can be simultaneously obtained. We concentrate on the case of using the E 8 lattice for shaping and low-density lattice codes (LDLC) with dimension ∼ 10,000 for coding. While optimal shaping provides a well-known 1.53 dB gain, previously reported shaping gains with LDLC lattices are on the order of 0.4 dB. The proposed method preserves the shaping gain of the E 8 lattice, that is, as much as 0.65 dB. This shaping operation can be implemented with lower complexity than previous LDLC approaches.

Haplotype assembly: An information theoretic view

Abstract: This paper studies the haplotype assembly problem from an information-theoretic perspective. A haplotype is a sequence of nucleotide bases on a chromosome, often conve- niently represented by a binary string, that differ from the bases in the corresponding positions on the other chromosome in a homologous pair. Information about the order of bases in a genome is readily inferred using short reads provided by high-throughput DNA sequencing technologies. Associating reads that cover variant positions with specific chromosomes in a homologous pairs, which enables haplotype assembly, is challenging due to limited lengths of the reads and presence of sequencing errors. In this paper, the recovery of the target pair of haplotype sequences using short reads is rephrased as a joint source-channel coding problem. Two messages, representing haplotypes and chromosome memberships of reads, are encoded and transmitted over a channel with erasures and errors, where the channel model reflects salient features of high-throughput sequencing. The focus of this paper is on determining the required number of reads for reliable haplotype reconstruction, and both the necessary and sufficient conditions are presented with order- wise optimal bounds.

Feedback systems using non-binary LDPC codes with a limited number of transmissions

Abstract: One advantage of incremental transmissions with feedback in point-to-point memoryless channels is a reduction in average blocklength required to approach capacity. This paper optimizes the size of each incremental transmission for non-binary (NB) LDPC codes to maximize throughput in VLFT and two-phase VLF settings. The optimization problem uses an approximation based on the inverse-Gaussian p.d.f. of the blocklength required for successful decoding. By using the optimized incremental transmission lengths (with an average blocklength of less than 500 bits), NB-LDPC codes for VLFT setting limited to 5 transmissions achieve a throughput greater than 96% of that obtained by an unlimited-transmission VLFT scheme with the same average blocklength. With a similar average blocklength, a two-phase VLF system limited to five transmissions (with optimized lengths) using the binary image of NB-LDPC codes achieves greater than 90% of the capacity of binary-input AWGN channel with SNR=2 dB. Two-phase VLF does not match the throughput of VLFT, but it is more practical than VLFT because it does not assume noiseless transmitter confirmation.