This comprehensive treatment of network information theory and its applications provides the first unified coverage of both classical and recent results. With an approach that balances the introduction of new models and new coding techniques, readers are guided through Shannon's point-to-point information theory, single-hop networks, multihop networks, and extensions to distributed computing, secrecy, wireless communication, and networking. Elementary mathematical tools and techniques are used throughout, requiring only basic knowledge of probability, whilst unified proofs of coding theorems are based on a few simple lemmas, making the text accessible to newcomers. Key topics covered include successive cancellation and superposition coding, MIMO wireless communication, network coding, and cooperative relaying. Also covered are feedback and interactive communication, capacity approximations and scaling laws, and asynchronous and random access channels. This book is ideal for use in the classroom, for self-study, and as a reference for researchers and engineers in industry and academia.

Network Information Theory

Information Theory and Reliable Communication

Sphere Packings, Lattices and Groups

Systems and methods for implementing array cameras configured to perform super- resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. Lens stack arrays in accordance with many embodiments of the invention include lens elements formed on substrates separated by spacers, where the lens elements, substrates and spacers are configured to form a plurality of optical channels, at least one aperture located within each optical channel, at least one spectral filter located within each optical channel, where each spectral filter is configured to pass a specific spectral band of light, and light blocking materials located within the lens stack array to optically isolate the optical channels.

Capturing and processing of images using monolithic camera array with heterogeneous imagers

Product design is a highly involved, often ill-defined, complex and iterative process, and the needs and specifications of the required artifact get more refined only as the design process moves toward its goal. An effective computer support tool that helps the designer make better-informed decisions requires efficient knowledge representation schemes. In today's world, there is a virtual explosion in the amount of raw data available to the designer, and knowledge representation is critical in order to sift through this data and make sense of it. In addition, the need to stay competitive has shrunk product development time through the use of simultaneous and collaborative design processes, which depend on effective transfer of knowledge between teams. Finally, the awareness that decisions made early in the design process have a higher impact in terms of energy, cost, and sustainability, has resulted in the need to project knowledge typically required in the later stages of design to the earlier stages. Research in design rationale systems, product families, systems engineering, and ontology engineering has sought to capture knowledge from earlier product design decisions, from the breakdown of product functions and associated physical features, and from customer requirements and feedback reports. VR (Virtual reality) systems and multidisciplinary modeling have enabled the simulation of scenarios in the manufacture, assembly, and use of the product. This has helped capture vital knowledge from these stages of the product life and use it in design validation and testing. While there have been considerable and significant developments in knowledge capture and representation in product design, it is useful to sometimes review our position in the area, study the evolution of research in product design, and from past and current trends, try and foresee future developments. The goal of this paper is thus to review both our understanding of the field and the support tools that exist for the purpose, and identify the trends and possible directions research can evolve in the future.

/pdf/the-evolution-challenges-and-future-of-knowledge-2l8tzvf3b5.pdf

The evolution, challenges, and future of knowledge representation in product design systems

We consider the problem of channel state amplification in a Gaussian channel with additive Gaussian channel states, where the encoder observes noncausally a noisy version of these states. A complete characterization is provided for the minimum reconstruction distortion under a transmitter power constraint, and it is shown that a simple analog scheme with power control is optimal. More precisely, if the power available to the encoder is below certain threshold, the analog scheme using full power is optimal, however, when the power available to the encoder is above that threshold, analog transmission using only a fixed amount of the available power is optimal. Furthermore, the problem of simultaneous message transmission and Gaussian state amplification with noisy observations is studied, for which an inner bound and two nontrivial outer bounds to the optimal tradeoff between the transmission rate and the state reconstruction distortion are provided. The coding scheme underlying the inner bound combines analog signaling and Gelfand-Pinsker coding, where the latter deviates from the operating point of Costa’s dirty paper coding. The first outer bound is obtained by extending the channel decomposition technique, while the second outer bound requires a strategic analysis of the covariance matrix of the relevant random variables.

Gaussian State Amplification with Noisy Observations

We consider symmetric multiple description coding for the Gaussian source, and provide upper and lower bounds for the individual description rate-distortion function. One of the main contributions of this work is a novel lower bound on the sum rate under symmetric distortion constraints, which yields a lower bound on the individual rate for the symmetric case. Two upper bounds are derived, the first of which is based on successive refinement coding coupled with multilevel diversity coding (SR-MLD), and the second is based on the multi-layer coding scheme proposed in literature. We show that the gaps between the lower bound and the upper bounds are no larger than certain constants depending only on the number of descriptions, but not the distortion constraints. Moreover, regardless of the number of descriptions, the gap between the lower bound and the upper bound using the SR-MLD coding scheme is less than 1.5 bits, and for the other case, the gap is less than 1 bit.

On the Symmetric Gaussian Multiple Description Rate-Distortion Function

We propose a new capacity-achieving code for the private information retrieval (PIR) problem, and show that it has the minimum message size (being one less than the number of servers) and the minimum upload cost (being roughly linear in the number of messages) among a general class of capacity-achieving codes, and in particular, among all capacity-achieving linear codes. Different from existing code constructions, the proposed code is asymmetric, and this asymmetry appears to be the key factor leading to the optimal message size and the optimal upload cost. The converse results on the message size and the upload cost are obtained by a strategic analysis of the information theoretic proof of the PIR capacity, from which a set of critical properties of any capacity-achieving code in the code class of interest is extracted.

Capacity-Achieving Private Information Retrieval Codes with Optimal Message Size and Upload Cost

In order to characterize the fundamental limit of the tradeoff between the amount of cache memory and the delivery transmission rate in multiuser caching systems, various coding schemes have been proposed. These schemes can largely be categorized into two classes, namely uncoded prefetching schemes and coded prefetching schemes. The significant differences in the coding components between the two classes may leave the impression that they are largely unrelated. In this work, we provide a connection between the uncoded prefetching scheme proposed by Maddah Ali and Niesen (and its improved version by Yu et al.) and the coded prefetching scheme proposed by Tian and Chen. A critical observation is first given where a coding component in the Tian-Chen scheme can be replaced by a binary code, which enables us to view the two schemes as the extremes of a more general scheme. An explicit example is given to show that the intermediate operating points of this general scheme can in fact provide new memory-rate tradeoff points previously not known to be achievable in the literature. This new general coding scheme is then presented and analyzed rigorously, which yields a new inner bound to the memory-rate tradeoff for the caching problem. This inner bound does not have a closed form, but can be computed efficiently using a linear program.

From Uncoded Prefetching to Coded Prefetching in Coded Caching Systems

We provide an explicit calculation of the rate-distortion function for the doubly-symmetric binary source (DSBS), when the side information may be absent at the decoder. The rate-distortion function for general discrete memoryless source was characterized by Heegard and Berger in 1985 [IT-31(6)], who showed that a two-stage coding structure is in fact optimal. However, an explicit characterization of the rate-distortion function for DSBS, and more importantly the optimal forward testing channel structure for this source, was not found despite several attempts. In this work, we resolve this open problem. It is shown that in the two-stage coding structure, the optimal testing channel for the first stage decoder (who does not have side information) is the same as the optimal testing channel for the ordinary symmetric binary source, and this confirms a conjecture made by Fleming and Effros.

Chao Tian

Papers

Gaussian State Amplification with Noisy Observations

On the Symmetric Gaussian Multiple Description Rate-Distortion Function

Capacity-Achieving Private Information Retrieval Codes with Optimal Message Size and Upload Cost

From Uncoded Prefetching to Coded Prefetching in Coded Caching Systems

A Calculation of the Heegard-Berger Rate-distortion Function for a Binary Source