scispace - formally typeset
Search or ask a question

Showing papers in "IEEE Transactions on Information Theory in 1979"


Journal ArticleDOI
TL;DR: In this article, the capacity of the Gaussian relay channel was investigated, and a lower bound of the capacity was established for the general relay channel, where the dependence of the received symbols upon the inputs is given by p(y,y) to both x and y. In particular, the authors proved that if y is a degraded form of y, then C \: = \: \max \!p(x,y,x,2})} \min \,{I(X,y), I(X,Y,Y,X,Y
Abstract: A relay channel consists of an input x_{l} , a relay output y_{1} , a channel output y , and a relay sender x_{2} (whose transmission is allowed to depend on the past symbols y_{1} . The dependence of the received symbols upon the inputs is given by p(y,y_{1}|x_{1},x_{2}) . The channel is assumed to be memoryless. In this paper the following capacity theorems are proved. 1)If y is a degraded form of y_{1} , then C \: = \: \max \!_{p(x_{1},x_{2})} \min \,{I(X_{1},X_{2};Y), I(X_{1}; Y_{1}|X_{2})} . 2)If y_{1} is a degraded form of y , then C \: = \: \max \!_{p(x_{1})} \max_{x_{2}} I(X_{1};Y|x_{2}) . 3)If p(y,y_{1}|x_{1},x_{2}) is an arbitrary relay channel with feedback from (y,y_{1}) to both x_{1} \and x_{2} , then C\: = \: \max_{p(x_{1},x_{2})} \min \,{I(X_{1},X_{2};Y),I \,(X_{1};Y,Y_{1}|X_{2})} . 4)For a general relay channel, C \: \leq \: \max_{p(x_{1},x_{2})} \min \,{I \,(X_{1}, X_{2};Y),I(X_{1};Y,Y_{1}|X_{2}) . Superposition block Markov encoding is used to show achievability of C , and converses are established. The capacities of the Gaussian relay channel and certain discrete relay channels are evaluated. Finally, an achievable lower bound to the capacity of the general relay channel is established.

4,311 citations


Journal ArticleDOI
TL;DR: It is proved that the Shannon zero-error capacity of the pentagon is \sqrt{5} and a well-characterized, and in a sense easily computable, function is introduced which bounds the capacity from above and equals the capacity in a large number of cases.
Abstract: It is proved that the Shannon zero-error capacity of the pentagon is \sqrt{5} . The method is then generalized to obtain upper bounds on the capacity of an arbitrary graph. A well-characterized, and in a sense easily computable, function is introduced which bounds the capacity from above and equals the capacity in a large number of cases. Several results are obtained on the capacity of special graphs; for example, the Petersen graph has capacity four and a self-complementary graph with n points and with a vertex-transitive automorphism group has capacity \sqrt{5} .

1,733 citations


Journal ArticleDOI
TL;DR: A new class of high-speed, high-throughput, stable, multiaccessing algorithms is presented and it is shown that these algorithms are stable (in that all moments of delay exist) and are optimal in a certain sense.
Abstract: The multiaccessing of a broadcast communication channel by independent sources is considered. Previous accessing techniques suffer from long message delays, low throughput, and/or congestion instabilities. A new class of high-speed, high-throughput, stable, multiaccessing algorithms is presented. Contentions resolving tree algorithms are introduced, and they are analyzed for specific probabilistic source models. It is shown that these algorithms are stable (in that all moments of delay exist) and are optimal in a certain sense. Furthermore, they have a maximum throughput of 0.430 packets/slut and have good delay properties. It is also shown that, under heavy traffic, the optimally controlled tree algorithm adaptively changes to the conventional time-division multiple access protocol.

1,002 citations


Journal ArticleDOI
Allen Gersho1
TL;DR: A heuristic argument generalizing Bennett's formula to block quantization where a vector of random variables is quantized is given, leading to a rigorous method for obtaining upper bounds on the minimum distortion for block quantizers.
Abstract: In 1948 W. R. Bennett used a companding model for nonuniform quantization and proposed the formula D \: = \: \frac{1}{12N^{2}} \: \int \: p(x) [ E(x) ]^{-2} \dx for the mean-square quantizing error where N is the number of levels, p (x) is the probability density of the input, and E \prime (x) is the slope of the compressor curve. The formula, an approximation based on the assumption that the number of levels is large and overload distortion is negligible, is a useful tool for analytical studies of quantization. This paper gives a heuristic argument generalizing Bennett's formula to block quantization where a vector of random variables is quantized. The approach is again based on the asymptotic situation where N , the number of quantized output vectors, is very large. Using the resulting heuristic formula, an optimization is performed leading to an expression for the minimum quantizing noise attainable for any block quantizer of a given block size k . The results are consistent with Zador's results and specialize to known results for the one- and two-dimensional cases and for the case of infinite block length (k \rightarrow \infty) . The same heuristic approach also gives an alternate derivation of a bound of Elias for multidimensional quantization. Our approach leads to a rigorous method for obtaining upper bounds on the minimum distortion for block quantizers. In particular, for k = 3 we give a tight upper bound that may in fact be exact. The idea of representing a block quantizer by a block "compressor" mapping followed with an optimal quantizer for uniformly distributed random vectors is also explored. It is not always possible to represent an optimal quantizer with this block companding model.

936 citations


Journal ArticleDOI
TL;DR: A coding theorem for the discrete memoryless broadcast channel is proved for the case where no common message is to he transmitted and the result is tight for broadcast channels having one deterministic component.
Abstract: A coding theorem for the discrete memoryless broadcast channel is proved for the case where no common message is to he transmitted. The theorem is a generalization of the results of Cover and van der Meulen on this problem. The result is tight for broadcast channels having one deterministic component

739 citations


Journal ArticleDOI
TL;DR: If the sequences are the outputs of two correlated memoryless binary sources, then in some cases the rate of this information may be substantially less than the joint entropy of the two sources.
Abstract: How much separate information about two random binary sequences is needed in order to tell with small probability of error in which positions the two sequences differ? If the sequences are the outputs of two correlated memoryless binary sources, then in some cases the rate of this information may be substantially less than the joint entropy of the two sources. This result is implied by the solution of the source coding problem with two separately encoded side information sources for a special class of source distributions.

413 citations


Journal ArticleDOI
TL;DR: The structure of punctured convolutional cedes is described, and it is indicated how their use simplifies the design of maximum likelihood decoders.
Abstract: The structure of punctured convolutional cedes is described, and it is indicated how their use simplifies the design of maximum likelihood decoders. The best codes of this class for rates 2/3 and 3/4 are tabulated and performance curves are given for these codes.

327 citations



Journal ArticleDOI
TL;DR: In this paper, Delsarte's linear programming bound is compared with Lov\acute{a}sz's \theta -function bound (an upper bound on the Shannon capacity of a graph).
Abstract: Delsarte's linear programming bound (an upper bound on the cardinality of cliques in association schemes) is compared with Lov\acute{a}sz's \theta -function bound (an upper bound on the Shannon capacity of a graph). The two bounds can be treated in a uniform fashion. Delsarte's linear programming bound can be generalized to a bound \theta \prime(G) on the independence number \propto(G) of an arbitrary graph G , such that \theta \prime(G) \leq \theta(G) . On the other hand, if the edge set of G is a union of classes of a symmetric association scheme, \theta(G) may be calculated by linear programming, For such graphs the product \theta(G) . \theta(G) is equal to the number of vertices of G .

249 citations


Journal ArticleDOI
TL;DR: The capacity region is established for those discrete memoryless broadcast channels p(y,z \mid x) for which I(X;Y) \geq I (X;Z) holds for all Input distributions.
Abstract: The capacity region is established for those discrete memoryless broadcast channels p(y,z \mid x) for which I(X;Y) \geq I(X;Z) holds for all Input distributions. The capacity region for this class of channels resembles the capacity region for degraded message sets considered by Korner and Marton.

233 citations


Journal ArticleDOI
TL;DR: It is shown that the mean-square difference between the probability of error for the nde and its deleted estimate is bounded by A/ \sqrt{n} where A is an explicitly given constant depending only on M and the potential function.
Abstract: In the discrimination problem the random variable \theta , known to take values in {1, \cdots ,M} , is estimated from the random vector X . All that is known about the joint distribution of (X, \theta) is that which can be inferred from a sample (X_{1}, \theta_{1}), \cdots ,(X_{n}, \theta_{n}) of size n drawn from that distribution. A discrimination nde is any procedure which determines a decision \hat{ \theta} for \theta from X and (X_{1}, \theta_{1}) , \cdots , (X_{n}, \theta_{n}) . For rules which are determined by potential functions it is shown that the mean-square difference between the probability of error for the nde and its deleted estimate is bounded by A/ \sqrt{n} where A is an explicitly given constant depending only on M and the potential function. The O(n ^{-1/2}) behavior is shown to be the best possible for one of the most commonly encountered rules of this type.

Journal ArticleDOI
TL;DR: In this paper, photoemissive detection of arbitrary quantized radiation fields is studied with incorporation of the nontrivial effects of detector quantum efficiency and the increase in homodyne signal-to-noise ratio obtained by use of TCS radiation yields significant performance gains in both linear modulation and antipodal signal detection.
Abstract: In Part I of this three-part study, it was shown that novel quantum states, called two-photon coherent states (TCS), have significant potential for improving free-space optical communications. Because TCS radiation does not possess a classical analog, i.e., its diagonal P -representation is highly singular, the semiclassical conditional Poisson process model for direct detection is not applicable to TCS reception. In this paper, photoemissive detection of arbitrary quantized radiation fields is studied with incorporation of the nontrivial effects of detector quantum efficiency. General theorems are derived permitting the application of classical point process results to the detection and estimation of signals in arbitrary quantum states. These general theorems are applied to determining the performance of TCS optical communication systems that employ direct, heterodyne, or homodyne detection in binary decision as well as in linear modulation problems. It is shown that the use of TCS radiation with direct detection or heterodyne detection results in minimal performance increments over comparable coherent-state systems. Homodyne detection, however, can achieve the full TCS signal-to-noise ratio improvement predicted in Part I of this study. The increase in homodyne signal-to-noise ratio obtained by use of TCS radiation yields significant performance gains in both linear modulation and antipodal signal detection.

Journal ArticleDOI
TL;DR: Coding schemes for the binary memoryless T -user adder channel are investigated and it is shown that these codes can be used to construct multilevel codes suitable for use on the additive white Gaussian noise channel.
Abstract: Coding schemes for the binary memoryless T -user adder channel are investigated in this paper. First upper and lower bounds on the capacity sum, which are asymptotically tight with increasing T , are derived for the noiseless case. Second, a class of T -user uniquely decodable codes with rates, asymptotically in T , equal to the maximal achievable values is constructed. A decoding algorithm for these codes is also presented. Next, a class of error-correcting codes for the noisy T -user adder channel is constructed. It is shown that these codes can he used to construct multilevel codes suitable for use on the additive white Gaussian noise channel.

Journal ArticleDOI
TL;DR: The answers to several problems of Lov\hat{a}sz concerning the Shannon capacity of a graph are shown to be negative.
Abstract: The answers to several problems of Lov\hat{a}sz concerning the Shannon capacity of a graph are shown to be negative.

Journal ArticleDOI
TL;DR: A new series for the interpolation of band-limited functions is found by using an approximation to tho prolate spheroidal wave function as a convergence factor and it is shown that this bound is lower than other known bounds in many cases of interest.
Abstract: A new series for the interpolation of band-limited functions is found by using an approximation to tho prolate spheroidal wave function as a convergence factor. A truncation error bound is found for the central interpolation of band-limited functions using the new series. It is shown that this bound is lower than other known bounds in many cases of interest.

Journal ArticleDOI
TL;DR: A modified Schalkwijk-Barron transmission scheme is presented for channels with noiseless feedback and the new result obtained shows that the modified scheme also attains high reliability functions for DMC's.
Abstract: A modified Schalkwijk-Barron transmission scheme is presented for channels with noiseless feedback. The modified scheme employs blockwise decision and a fixed length transmission in place of Viterbi's sequential decision feedback. The reliability functions of the modified scheme are derived for the additive white Gaussian noise (AWGN) channel and discrete memoryless channels (DMC's). For the AWGN channel the result obtained is asymptotically the same as in the case of the Schalkwijk-Barron scheme. On the other hand, the new result obtained for DMC's shows that the modified scheme also attains high reliability functions for DMC's.

Journal ArticleDOI
TL;DR: A two-stage iterative algorithm for selecting a subset of a training set of samples for use in a condensed nearest neighbor (CNN) decision rule is introduced.
Abstract: A two-stage iterative algorithm for selecting a subset of a training set of samples for use in a condensed nearest neighbor (CNN) decision rule is introduced. The proposed method uses the concept of mutual nearest neighborhood for selecting samples close to the decision line. The efficacy of the algorithm is brought out by means of an example.

Journal ArticleDOI
TL;DR: In the discrimination problem the random variable \theta, known to take values in {1,\ldots,M} , is estimated from the random vector X taking values in {\bfR}^{d} .
Abstract: In the discrimination problem the random variable \theta , known to take values in {1 ,\ldots ,M} , is estimated from the random vector X taking values in {\bfR}^{d} . Ali that is known about the joint distribution of (X,O) is that which can be inferred from a sample (X_{1} , \theta_{1}, \ldots , (X_{n}, \theta_{n}) of size n drawn from that distribution. A discrimination rule is any procedure which determines a decision \hat{\theta} for \theta from X and (X_{1},\theta_{1}) , \ldots , (X_{n}, \theta_{n}) . The rule is called k -local if the decision \hat{\theta} depends only on X and the pairs (X_{i}, \theta_{i}) ,for which X_{i} is one of the k closest to X from X_{1} , \ldots ,X_{n} . If L_{n} denotes the probability of error for a k -local rule given the sample, then estimates \hat{L}_{n} of L_{n} , are determined for which P {| \hat{L}_{n} - L_{n} \geq \epsilon} \exp (- Bn) , where A and B are positive constants depending only on d , M , and \epsilon .

Journal ArticleDOI
TL;DR: All self-dual codes over GF(3) and GF(4) of length 16 are found and several new extremal self- dual codes are given which have length greater than 16.
Abstract: All self-dual codes over GF(3) and GF(4) of length 16 are found. The self-dual codes of shorter length are described in a concise and systematic notation. A number of new techniques ("promotion" and "demotion," "tag olng? and "subtraction") are given for constructing codes. Finally, several new extremal self-dual codes are given which have length greater than 16.

Journal ArticleDOI
TL;DR: In this paper, the construction of BCH codes over finite fields was shown to be similar to that for BCH code over finite integer rings, where the generator polynomials are derived by factorization of x n −1 over the unit ring of an appropriate extension of the Finite integer ring.
Abstract: Bose-Chadhuri-Hocquenghem (BCH) codes with symbols from an arbitrary finite integer ring are derived in terms of their generator polynomials. Tile derivation is based on the factorization of x^{n}-1 over the unit ring of an appropriate extension of the Finite integer ring. The construction is thus shown to be similar to that for BCH codes over finite fields.

Journal ArticleDOI
TL;DR: Algorithms are presented for encoding and decoding strings of characters as real binary fractions, using registers of fixed precision, and have storage requirements and computation time O(n \log_{2}N) for string length n and alphabet size N.
Abstract: Algorithms are presented for encoding and decoding strings of characters as real binary fractions, using registers of fixed precision. The encoding is left to right and does not require blocking. The algorithms have storage requirements O(N) and computation time O(n \log_{2}N) for string length n and alphabet size N .

Journal ArticleDOI
TL;DR: Asymptotically coincident upper and lower bounds on the exponent of the largest possible probability of the correct decoding of block codes are given for all rates above capacity.
Abstract: Asymptotically coincident upper and lower bounds on the exponent of the largest possible probability of the correct decoding of block codes are given for all rates above capacity. The lower bound sharpens Omura's bound. The upper bound is proved by a new and simple combinatorial argument.

Journal ArticleDOI
TL;DR: An analytical technique, which employs a Markov ratio limit theorem, is presented for the derivation of the delay-throughput performance curves of dynamic demand-assignment reservation schemes.
Abstract: Reservation and TDMA schemes are studied for governing the access-control discipline for a network of terminals communicating through a multi-access broadcast channel. A single repeater is employed to allow a fully connected network structure. A channel can be characterized as inducing a Iow propagation-delay value, as for terrestrial radio or fine networks, or as being associated with a higher propagation-delay value, as for a satellite communication channel. A synchronized (slotted) communication medium is considered. Messages are composed of a random number of packets, governed by an arbitrary message-length distribution. The process describing the number of reserved message arrivals within each time frame is assumed to be a sequence of i.i.d, random variables, governed by an arbitrary distribution. (A Poisson arrival stream thus becomes a special case.) The reservation access-control disciplines studied in this paper employ message-switching distributed-control procedures. The performance of each access-control scheme is evaluated according to its delay-throughput function. In particular, schemes are developed to adapt their structure, or protocol, dynamically to the underlying fluctuating network traffic-flow values. A fixed-reservation access-control (FRAC) discipline is studied, employing a fixed periodic pattern of reservation and service periods. The reservation periods are used for the transmission of reservation packets as well as for the integrated service of other groups of network stations. The latter stations can access the channel during these periods, using any proper access control procedure. As a special case, message-delay distributions and moments under a TDMA scheme are obtained. Using dynamic estimates of the underlying message traffic parameters, a dynamic fixed-reservation access-control (DFRAC) scheme is obtained. An analytical technique, which employs a Markov ratio limit theorem, is presented for the derivation of the delay-throughput performance curves of dynamic demand-assignment reservation schemes. To illustrate its application, asynchronous reservation demand-assignment (ARDA) schemes are developed to adapt automatically to the underlying network traffic characteristics. Such schemes establish reservation slots dynamically according to observed network service demands and queue sizes.

Journal ArticleDOI
TL;DR: The singie-letter characterization obtained for this region simplifies the computation of the capacity region of this related class of DBC's.
Abstract: A single-letter characterization of the capacity region for a class of discrete degraded interference channels (IFC's) is determined. This class of degraded IFC's has an additive structure similar to that of a class of degraded Gaussian IFC's. The capacity region for this class of degraded IFC's is shown to be also the capacity region of a related class of degraded broadcast channels (DBC's). The singie-letter characterization obtained for this region simplifies the computation of the capacity region of this related class of DBC's.

Journal ArticleDOI
TL;DR: The algorithm for developing the rational approximations is based on continued fraction techniques and is virtually equivalent to an algorithm employed by Berlekamp for decoding BCH codes.
Abstract: Theorems are presented concerning the optimality of rational approximations using non-Archimedean norms. The algorithm for developing the rational approximations is based on continued fraction techniques and is virtually equivalent to an algorithm employed by Berlekamp for decoding BCH codes. Several variations of the continued fraction technique and Berlekamp's algorithm are illustrated on a common example.

Journal ArticleDOI
TL;DR: Two specific problems are discussed: the use of previous decisions, which leads to a weighted generalization of feedback decoding, and the extension of replication decoding to nonsystematic codes.
Abstract: Any symbol in a redundant code can be recovered when it belongs to certain erasure patterns. Several alternative expressions of a given symbol, to be referred to as its replicas, can therefore be computed in terms of other ones. Decoding is interpreted as decoding upon a received symbol, given itself and a number of such replicas, expressed in terms of other received symbols. For linear q-ary (n,k) block codes, soft-decision demodulation and memoryless channels, the maximum-likelihood decision rule on a given symbol is formulated in terms of r \leqn - k linearly independent replicas from the parity-check equations. All replicas deriving from the r selected replicas by linear combination are actually taken into account in this decision rule. Its implementation can be direct; use transformations or a sequential circuit implementing a trellis representation of the parity-check matrix. If r = n - k , decoding is optimum, in the sense of symbol-by-symbol maximum-likelihoed. Simplification results in the transformed and sequential implementations when r . If the selected replicas are disjoint, generalized ( q -ary, weighted) threshold decoding results. The decoding process can easily be modffied in order to provide word-by-word maximum-likelihood decoding. Convolutional codes are briefly considered. Two specific problems are discussed: the use of previous decisions, which leads to a weighted generalization of feedback decoding, and the extension of replication decoding to nonsystematic codes.

Journal ArticleDOI
Jr. C. Johnson1
TL;DR: Hyperstability, a concept from nonlinear stability theory, is used to develop a real-time adaptive recursive filter useful in a nonstationary environment.
Abstract: Hyperstability, a concept from nonlinear stability theory, is used to develop a real-time adaptive recursive filter useful in a nonstationary environment.

Journal ArticleDOI
TL;DR: It is shown that in the idealized model the above stability results apply with only minor modifications, however, nonmixing input in conjunction with certain types of noise lead to bounded input - unbounded output, i.e., instability.
Abstract: A variety of theoretical results are derived for a well-known class of discrete-time adaptive filters. First the following idealized identification problem is considered: a discrete-time system has vector input x(t) and scalar output z(t)= h \' x(t) where h is an unknown time-invariant coefficient vector. The filter considered adjusts an estimate vector \hat{h}(t) in a control loop according to \hat{h}(t + \Delta t) = \hat{h}(t) + K[z(t) - \hat{z} (t)]x(t) , where \hat{z}( t)= \hat{h}( t) \' x( t) and K is the control loop gain. The effectiveness of the filter is determined by the convergence properties of the misalignment vector r(t) = h - \hat{h}(t) . It is shown that a certain nondegeneracy "mixing" condition on the Input { x(t)} is necessary and sufficient for the exponential convergence of the misalignment. Qualitatively identical upper and lower bounds are derived for the rate of convergence. Situations where noise is present in z(t) and x(t) and the coefficient vector h is time-varying are analyzed. Nonmixing inputs are also considered, and it is shown that in the idealized model the above stability results apply with only minor modifications. However, nonmixing input in conjunction with certain types of noise lead to bounded input - unbounded output, i.e., instability.

Journal ArticleDOI
G. Brassard1
TL;DR: Evidence is given for the difficulty of an eventual proof of computational security for cryptosystems based on one-way functions, such as the one proposed by Diffie and Hellman.
Abstract: Evidence is given for the difficulty of an eventual proof of computational security for cryptosystems based on one-way functions, such as the one proposed by Diffie and Hellman. A proof of NP-completeness for the cryptanalytic effort would imply NP=CoNP.

Journal ArticleDOI
TL;DR: A large family of linear orthogonal invertible scrambling transformations is described that result in a negligible expansion of bandwidth and can therefore serve as building blocks in a secure communication system.
Abstract: The problem of scrambling a discrete-time analog sequence for the purpose of privacy encoding is studied. A large family of linear orthogonal invertible scrambling transformations is described that result in a negligible expansion of bandwidth and can therefore serve as building blocks in a secure communication system.