Consider a discrete source producing a sequence of message letters from a finite alphabet. A single-letter distortion measure is given by a non-negative matrix (d ij ). The entryd ij measures the ?cost? or ?distortion? if letteriis reproduced at the receiver as letterj. The average distortion of a communications system (source-coder-noisy channel-decoder) is taken to bed= ? i.j P ij d ij whereP ij is the probability ofibeing reproduced asj. It is shown that there is a functionR(d) that measures the ?equivalent rate? of the source for a given level of distortion. For coding purposes where a leveldof distortion can be tolerated, the source acts like one with information rateR(d). Methods are given for calculatingR(d), and various properties discussed. Finally, generalizations to ergodic sources, to continuous sources, and to distortion measures involving blocks of letters are developed. In this paper a study is made of the problem of coding a discrete source of information, given afidelity criterionor ameasure of the distortionof the final recovered message at the receiving point relative to the actual transmitted message. In a particular case there might be a certain tolerable level of distortion as determined by this measure. It is desired to so encode the information that the maximum possible signaling rate is obtained without exceeding the tolerable distortion level. This work is an expansion and detailed elaboration of ideas presented earlier [1], with particular reference to the discrete case. We shall show that for a wide class of distortion measures and discrete sources of information there exists a functionR(d) (depending on the particular distortion measure and source) which measures, in a sense, the equivalent rateRof the source (in bits per letter produced) whendis the allowed distortion level. Methods will be given for evaluatingR(d) explicitly in certain simple cases and for evaluatingR(d) by a limiting process in more complex cases. The basic results are roughly that it is impossible to signal at a rate faster thanC/R(d) (source letters per second) over a memoryless channel of capacityC(bits per second) with a distortion measure less than or equal tod. On the other hand, by sufficiently long block codes it is possible to approach as closely as desired the rateC/R(d) with distortion leveld. Finally, some particular examples, using error probability per letter of message and other simple distortion measures, are worked out in detail.

Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959.

In certain communication systems where information is to be transmitted from one point to another, additional side information is available at the transmitting point. This side information relates to the state of the transmission channel and can be used to aid in the coding and transmission of information. In this paper a type of channel with side information is studied and its capacity determined.

Channels with Side Information at the Transmitter

We generalize theGel'fand-Pinsker modeltoen- compassthesetupofa memoryless multiple-access channel. According tothissetup, onlyoneoftheencoders knowsthe state ofthechannel (non-causally), whichisalsounknowntothe receiver. Twoindependent messages aretransmitted: acommon message andamessage transmitted bytheinformed encoder. We findexplicit characterizations ofthecapacity region withboth non-causal andcausal state information. Further, weapplythe general formula totheGaussian casewithnon-causal channel state information, underanindividual powerconstraint aswell asasumpowerconstraint. Inthiscase, thecapacity region is achievable byageneralized writing-on-dirty-paper scheme. I.INTRODUCTION

Cooperative Multiple Access Encoding withStates AvailableatOne Transmitter

We investigate the information theoretic limits of two types of state-dependent models in this dissertation. These models capture a wide range of wireless communication scenarios where there are interference cognition among transmitters. Hence, information theoretic studies of these models provide useful guidelines for designing new interference cancellation schemes in practical wireless networks. In particular, we first study the two-user state-dependent Gaussian multiple access channel (MAC) with a helper. Inner and outer bounds on the capacity region are first derived, which improve the state-of-the-art bounds given in the literature. Further comparison of these bounds yields either segments on the capacity region boundary or the full capacity region by considering various regimes of channel parameters. We then study the two-user Gaussian state-dependent Z-IC and state-dependent IC. Three interference regimes are studied, and the capacity region or the sum capacity boundary is characterized either fully or partially under various channel parameters. The impact of the correlation between the states on the cancellation of state and interference as well as the achievability of the capacity is demonstrated via numerical analysis. Numerical investigation indicates that for the regular IC, the correlation between states impacts the achievability of the channel capacity in a different way from that of the Z-IC.

/pdf/information-theoretic-limits-of-state-dependent-networks-3pb7zkzvx9.pdf

Yunhao Sun

Papers

Information Theoretic Limits of State-dependent Networks