Showing papers in "IEEE Transactions on Information Theory in 1986"
TL;DR: The results show that the proposed multiuser detectors afford important performance gains over conventional single-user systems, in which the signal constellation carries the entire burden of complexity required to achieve a given performance level.
Abstract: Consider a Gaussian multiple-access channel shared by K users who transmit asynchronously independent data streams by modulating a set of assigned signal waveforms. The uncoded probability of error achievable by optimum multiuser detectors is investigated. It is shown that the K -user maximum-likelihood sequence detector consists of a bank of single-user matched filters followed by a Viterbi algorithm whose complexity per binary decision is O(2^{K}) . The upper bound analysis of this detector follows an approach based on the decomposition of error sequences. The issues of convergence and tightness of the bounds are examined, and it is shown that the minimum multiuser error probability is equivalent in the Iow-noise region to that of a single-user system with reduced power. These results show that the proposed multiuser detectors afford important performance gains over conventional single-user systems, in which the signal constellation carries the entire burden of complexity required to achieve a given performance level.
2,300 citations
TL;DR: A "coordinate recurrence" method for solving sparse systems of linear equations over finite fields is described and a probabilistic algorithm is shown to exist for finding the determinant of a square matrix.
Abstract: A "coordinate recurrence" method for solving sparse systems of linear equations over finite fields is described. The algorithms discussed all require O(n_{1}(\omega + n_{1})\log^{k}n_{1}) field operations, where n_{1} is the maximum dimension of the coefficient matrix, \omega is approximately the number of field operations required to apply the matrix to a test vector, and the value of k depends on the algorithm. A probabilistic algorithm is shown to exist for finding the determinant of a square matrix. Also, probabilistic algorithms are shown to exist for finding the minimum polynomial and rank with some arbitrarily small possibility of error.
617 citations
TL;DR: In this paper, the problem of determining the minimum number of vertex guards that can see an n -wall simply connected art gallery is shown to be NP-hard, and it can be modified to show that the problems of finding edge guards and point guards in a simply connected polygonal region are also NP-Hard.
Abstract: We study the computational complexity of the art gallery problem originally posed by Klee, and its variations. Specifically, the problem of determining the minimum number of vertex guards that can see an n -wall simply connected art gallery is shown to be NP-hard. The proof can be modified to show that the problems of determining the minimum number of edge guards and the minimum number of point guards in a simply connected polygonal region are also NP-hard. As a byproduct, the problem of decomposing a simple polygon into a minimum number of star-shaped polygons such that their union is the original polygon is also shown to be NP-hard.
441 citations
TL;DR: Although suboptimum in a rate-distortion sense, because the PVQ can encode large-dimensional vectors, it offers significant reduction in rose distortion compared with the optimum Lloyd-Max scalar quantizer, and provides an attractive alternative to currently available vector quantizers.
Abstract: The geometric properties of a memoryless Laplacian source are presented and used to establish a source coding theorem. Motivated by this geometric structure, a pyramid vector quantizer (PVQ) is developed for arbitrary vector dimension. The PVQ is based on the cubic lattice points that lie on the surface of an L -dimensional pyramid and has simple encoding and decoding algorithms. A product code version of the PVQ is developed and generalized to apply to a variety of sources. Analytical expressions are derived for the PVQ mean square error (mse), and simulation results are presented for PVQ encoding of several memoryless sources. For large rate and dimension, PVQ encoding of memoryless Laplacian, gamma, and Gaussian sources provides rose improvements of 5.64, 8.40 , and 2.39 dB, respectively, over the corresponding optimum scalar quantizer. Although suboptimum in a rate-distortion sense, because the PVQ can encode large-dimensional vectors, it offers significant reduction in rose distortion compared with the optimum Lloyd-Max scalar quantizer, and provides an attractive alternative to currently available vector quantizers.
350 citations
TL;DR: Coding schemes in which each codeword contains equally many zeros and ones are constructed in such a way that they can be efficiently encoded and decoded.
Abstract: Coding schemes in which each codeword contains equally many zeros and ones are constructed in such a way that they can be efficiently encoded and decoded.
299 citations
TL;DR: A new class of statistical problems is introduced, involving the presence of communication constraints on remotely collected data, when the statistician has direct access to Y data but can be informed about X data only at a preseribed finite rate R.
Abstract: A new class of statistical problems is introduced, involving the presence of communication constraints on remotely collected data. Bivariate hypothesis testing, H_{0}: P_{XY} against H_{1}: P_{\={XY}} , is considered when the statistician has direct access to Y data but can be informed about X data only at a preseribed finite rate R . For any fixed R the smallest achievable probability of an error of type 2 with the probability of an error of type 1 being at most \epsilon is shown to go to zero with an exponential rate not depending on \epsilon as the sample size goes to infinity. A single-letter formula for the exponent is given when P_{\={XY}} = P_{X} \times P_{Y} (test against independence), and partial results are obtained for general P_{\={XY}} . An application to a search problem of Chernoff is also given.
272 citations
TL;DR: To use probabilistic functions of a Markov chain to model certain parameterizations of the speech signal, an estimation technique of Liporace is extended to the eases of multivariate mixtures, such as Gaussian sums, and products of mixtures.
Abstract: To use probabilistic functions of a Markov chain to model certain parameterizations of the speech signal, we extend an estimation technique of Liporace to the eases of multivariate mixtures, such as Gaussian sums, and products of mixtures. We also show how these problems relate to Liporace's original framework.
244 citations
TL;DR: The proposed picture compressibility is shown to possess the properties that one would expect and require of a suitably defined concept of two-dimensional entropy for arbitrary probabilistic ensembles of infinite pictures.
Abstract: Distortion-free compressibility of individual pictures, i.e., two-dimensional arrays of data, by finite-state encoders is investigated. For every individual infinite picture I , a quantity \rho(I) is defined, called the compressibility of I , which is shown to be the asymptotically attainable lower bound on the compression ratio that can be achieved for I by any finite-state information-lossless encoder. This is demonstrated by means of a constructive coding theorem and its converse that, apart from their asymptotic significance, might also provide useful criteria for finite and practical data-compression tasks. The proposed picture compressibility is also shown to possess the properties that one would expect and require of a suitably defined concept of two-dimensional entropy for arbitrary probabilistic ensembles of infinite pictures. While the definition of \rho(I) allows the use of different machines for different pictures, the constructive coding theorem leads to a universal compression scheme that is asymptotically optimal for every picture. The results are readily extendable to data arrays of any finite dimension.
217 citations
TL;DR: Here an information-theoretic proof of the blowing-up lemma, generalizing it to continuous alphabets, is given.
Abstract: The blowing-up lemma says that if the probability with respect to a product measure of a set A\subseteq {\cal X}^{n} ({\cal X} finite, n large) is not exponentially small, then its l_{n} -neighborhood has probability almost one for some l_{n} = O(n) . Here an information-theoretic proof of the blowing-up lemma, generalizing it to continuous alphabets, is given.
216 citations
TL;DR: Shannon's self-information of a string is generalized to its complexity relative to the class of finite-state-machine (FSM) defined sources by a theorem stating that, asymptotically, the mean complexity provides a tight lower bound for the mean length of all so-called regular codes.
Abstract: Shannon's self-information of a string is generalized to its complexity relative to the class of finite-state-machine (FSM) defined sources. Unlike an earlier generalization, the new one is valid for both short and long strings. The definition is justified in part by a theorem stating that, asymptotically, the mean complexity provides a tight lower bound for the mean length of all so-called regular codes. This also generalizes Shannon's noiseless coding theorem. For a large subclass of FSM sources a simple algorithm is described for computing the complexity.
210 citations
TL;DR: A new lower bound for the minimum distance of cyclic codes that includes earlier bounds (i.e., BCH bound, HT bound, Roos bound) is created and can be even stronger than the first one.
Abstract: The main result is a new lower bound for the minimum distance of cyclic codes that includes earlier bounds (i.e., BCH bound, HT bound, Roos bound). This bound is related to a second method for bounding the minimum distance of a cyclic code, which we call shifting. This method can be even stronger than the first one. For all binary cyclic codes of length (with two exceptions), we show that our methods yield the true minimum distance. The two exceptions at the end of our list are a code and its even-weight subcode. We treat several examples of cyclic codes of length \geq 63 .
TL;DR: Ami are used to obtain new and faster decoding algorithms for the Gosset lattice E_{8} , the Golay code the Leech lattice and some general methods are given for constructing such algorithms.
Abstract: Two kinds of algorithms are considered. 1) If *** is a binary code of length n , a "soft decision" decoding algorithm for *** changes an arbitrary point of R^{n} into a nearest codeword (nearest in Euclidean distance). 2) Similarly, a decoding algorithm for a lattice \Lambda in R^{n} changes an arbitrary point of R^{n} into a closest lattice point. Some general methods are given for constructing such algorithms, ami are used to obtain new and faster decoding algorithms for the Gosset lattice E_{8} , the Golay code the Leech lattice.
TL;DR: In this article, upper bounds on the decoder error probability for Reed-Solomon codes were derived, and it was shown that for a t error-correcting RS code, if more than t errors occur, the probability of decoder failure is less than 1/t.
Abstract: Upper bounds On the decoder error probability for Reed-Solomon codes are derived. By definition, "decoder error" occurs when the decoder finds a codeword other than the transitted codeword; this is in contrast to "decoder failure," which occurs when the decoder fails to find any codeword at all. These results imply, for example, that for a t error-correcting Reed-Solomon code of length q - 1 over GF (q) , if more than t errors occur, the probability of decoder error is less than 1/t! .
TL;DR: The performance of frequency-hop transmission in a packet communication network is analyzed and new measures of "local" performance are defined and evaluated for networks of this type, and new concepts that are important in the design of these networks are introduced.
Abstract: The performance of frequency-hop transmission in a packet communication network is analyzed. Satellite multiple-access broadcast channels for packet switching and terrestrial packet radio networks are the primary examples of the type of network considered. An analysis of the effects of multiple-access interference in frequency-hop radio networks is presented. New measures of "local" performance are defined and evaluated for networks of this type, and new concepts that are important in the design of these networks are introduced. In particular, error probabilities and local throughput are evaluated for a frequency-hop radio network which incorporates the standard slotted and unslotted ALOHA channel-access protocols, asynchronous frequency hopping, and Reed-Solomon error-control coding. The performance of frequency-hop multiple access with error-control coding is compared with the performance of conventional ALOHA random access using narrow-band radios.
TL;DR: The generalized Lloyd algorithm for vector quantizer design is analyzed as a descent algorithm for nonlinear programming and a well-known convergence theorem is applied to show that iterative applications of the algorithm produce a sequence of quantizers that approaches the set of fixed-point quantizers.
Abstract: The generalized Lloyd algorithm for vector quantizer design is analyzed as a descent algorithm for nonlinear programming. A broad class of convex distortion functions is considered and any input distribution that has no singular-continuous part is allowed. A well-known convergence theorem is applied to show that iterative applications of the algorithm produce a sequence of quantizers that approaches the set of fixed-point quantizers. The methods of the theorem are extended to sequences of algorithms, yielding results on the behavior of the algorithm when an unknown distribution is approximated by a training sequence of observations. It is shown that as the length of the training sequence grows large that 1) fixed-point quantizers for the training sequence approach the set of fixed-point quantizers for the true distribution, and 2) limiting quantizers produced by the algorithm with the training sequence distribution perform no worse than limiting quantizers produced by the algorithm with the true distribution.
TL;DR: Lexicographic codes, or lexicodes, are defined by various versions of the greedy algorithm and are closely related to the theory of certain impartial games, which leads to a number of surprising properties.
Abstract: Lexicographic codes, or lexicodes, are defined by various versions of the greedy algorithm. The theory of these codes is closely related to the theory of certain impartial games, which leads to a number of surprising properties. For example, lexicodes over an alphabet of size B=2^{a} are closed under addition, while if B = 2^{2^{a}} the lexicodes are closed under multiplication by scalars, where addition and multiplication are in the nim sense explained in the text. Hamming codes and the binary Golay codes are lexieodes. Remarkably simple constructions are given for the Steiner systems S(5,6,12) and S(5,8,24) . Several record-breaking constant weight codes are also constructed.
TL;DR: Recursive estimation of the univariate probability density function f(x) for stationary processes \{X_{j}\} is considered and Quadratic-mean convergence and asymptotic normality for density estimators f_{n}( x) are established for strong mixing and for asymPTotically uncorrelated processes.
Abstract: Recursive estimation of the univariate probability density function f(x) for stationary processes \{X_{j}\} is considered. Quadratic-mean convergence and asymptotic normality for density estimators f_{n}(x) are established for strong mixing and for asymptotically uncorrelated processes \{X_{j}\} . Recent results for nonrecursive density estimators are extended to the recursive case.
TL;DR: A number of upper and lower bounds are obtained for K(n, R) , the minimal number of codewords in any binary code of length n and covering radius R, and an upper bound is given for the density of a covering code over any alphabet.
Abstract: A number of upper and lower bounds are obtained for K(n, R) , the minimal number of codewords in any binary code of length n and covering radius R . Several new constructions are used to derive the upper bounds, including an amalgamated direct sum construction for nonlinear codes. This construction works best when applied to normal codes, and we give some new and stronger conditions which imply that a linear code is normal. An upper bound is given for the density of a covering code over any alphabet, and it is shown that K(n + 2, R + 1) \leq K(n, R) holds for sufficiently large n .
TL;DR: The capacity of the arbitrarily varying channels with states sequence known to the sender is determined with the help of an elimination technique and a robustification technique, demonstrating the power of these techniques.
Abstract: The capacity of the arbitrarily varying channels with states sequence known to the sender is determined. The result is obtained with the help of an elimination technique and a robustification technique. It demonstrates once more the power of these techniques.
TL;DR: An application of error-correcting codes to "write-once" memories (WOM's) as defined by Rivest and Shamir is studied and large classes of "WOM codes" that are easily decodable are obtained.
Abstract: An application of error-correcting codes to "write-once" memories (WOM's) as defined by Rivest and Shamir is studied. Large classes of "WOM codes" that are easily decodable are obtained. In particular, a construction allowing three successive writings of 11 bits on 23 positions is derived from the Golay code.
TL;DR: It is proved that the proposed test for detecting a change from one given stationary and ergodic stochastic process to another such process is asymptotically optimal in a mathematically precise sense.
Abstract: The problem of detecting a change from one given stationary and ergodic stochastic process to another such process is considered. It is assumed that both stochastic processes are processes with memory and that they are mutually independent. A sequential test is proposed and analyzed. It is proved that the proposed test is asymptotically optimal in a mathematically precise sense.
TL;DR: An approach for efficient utilization of fast Hadamard transform in decoding binary linear block codes is presented and the availability of these codewords in general, and particularly in some of the most frequently encountered codes, is discussed.
Abstract: An approach for efficient utilization of fast Hadamard transform in decoding binary linear block codes is presented. Computational gain is obtained by employing various types of concurring codewords, and memory reduction is also achieved by appropriately selecting rows for the generator matrix. The availability of these codewords in general, and particularly in some of the most frequently encountered codes, is discussed.
TL;DR: Upper bounds are presented on the redundancy of Huffman codes when the extreme probabilities P_{1} and P_{N} are known.
Abstract: New upper bounds on the redundancy of Huffman codes are provided. A bound that for 2/9 \leq P_{1} \leq 0.4 is sharper than the bound of Gallager, when the probability of the most likely source letter P_{1} is the only known probability is presented. The improved bound is the tightest possible for 1/3 \leq P_{1} \leq 0.4 . Upper bounds are presented on the redundancy of Huffman codes when the extreme probabilities P_{1} and P_{N} are known.
TL;DR: An interesting feature of the analysis is that magnitude codes that are catastrophic may perform better than those that are noncatastrophic.
Abstract: Calderbank, Heegard, and Ozarow [1] have suggested a method of designing codes for channels with intersymbol interference, such as the magnetic recording channel. These codes are designed to exploit intersymbol interference. The standard method is to minimize intersymbol interference by constraining the input to the channel using run-length limited sequences. Calderbank, Heegard, and Ozarow considered an idealized model of an intersymbol interference channel that leads to the problem of designing codes for a partial response channel with transfer function (1 - D^{N}) /2 , where the channel inputs are constrained to be \pm 1 . This problem is considered here. Channel inputs are generated using a nontrivial coset of a binary convolutional code. The coset is chosen to limit the zero-run length of the output of the channel and so maintain clock synchronization. The minimum squared Euclidean distance between outputs corresponding to distinct inputs is bounded below by the free distance of a second convolutional code which we call the magnitude code. An interesting feature of the analysis is that magnitude codes that are catastrophic may perform better than those that are noncatastrophic.
TL;DR: There exists a nontrivial cyclic extended Reed-Solomon code of length q over GF (q) if and only if q is a prime.
Abstract: It is shown that a cyclic code C of length q over GF (q) is the maximum distance separable if and only if either 1) q is a prime, in which case C is equivalent, up to a coordinate permutation, to an extended Reed-Solomon code, or 2) C is a trivial code of dimension k \in \{1, q - 1, q \} . Hence there exists a nontrivial cyclic extended Reed-Solomon code of length q over GF (q) if and only if q is a prime.
TL;DR: This work introduces exceptionally simple decoding algorithms for the two extended Golay codes based on recent methods of Conway and Curtis of finding the unique blocks containing five points in either the (5,8,24) Steiner system or the ( 5,6,12) Steiners system.
Abstract: We introduce exceptionally simple decoding algorithms for the two extended Golay codes The algorithms are based on recent methods of Conway and Curtis of finding the unique blocks containing five points in either the (5,8,24) Steiner system or the (5,6,12) Steiner system These decoding methods are simple enough to enable decoding extended Golay codes by hand Both of the methods involve relations between the extended Golay codes and other self-dual codes Proofs are given explaining these relationships and why the decoding methods work The decoding algorithms are explained and illustrated with many examples [3 , chap 12] has facts about the Mathieu group and some details about decoding the Golay codes
TL;DR: Both nonrecursive and recursive nonparametric regression estimates are studied and the rates of weak and strong convergence of kernel estimates, as well as corresponding multiple classification errors, are derived without assuming the existence of the density of the measurements.
Abstract: Both nonrecursive and recursive nonparametric regression estimates are studied. The rates of weak and strong convergence of kernel estimates, as well as corresponding multiple classification errors, are derived without assuming the existence of the density of the measurements. An application of the obtained results to a nonparametric Bayes predication is presented.
TL;DR: It is proved that the O(1/n) bound is optimal to within a constant factor and it is shown that linear estimation procedures are just as powerful (up to the measure of mean-square error) as arbitrary estimation procedures.
Abstract: Let \{X_{i}\}_{i=1}^{\infty} be a sequence of independent Bernoulli random variables with probability p that X_{i} = 1 and probability q=1-p that X_{i} = 0 for all i \geq 1 . Time-invariant finite-memory (i.e., finite-state) estimation procedures for the parameter p are considered which take X_{1}, \cdots as an input sequence. In particular, an n-state deterministic estimation procedure is described which can estimate p with mean-square error O(\log n/n) and an n -state probabilistic estimation procedure which can estimate p with mean-square error O(1/n) . It is proved that the O(1/n) bound is optimal to within a constant factor. In addition, it is shown that linear estimation procedures are just as powerful (up to the measure of mean-square error) as arbitrary estimation procedures. The proofs are based on an analog of the well-known matrix tree theorem that is called the Markov chain tree theorem.
TL;DR: In this paper, a probabilistic polynomial-time algorithm for computing the square root of a number x ∆ ∆ Z/P{\bf Z}, where P = 2^{S}Q + 1(Q odd, s > 0) is a prime number, is described.
Abstract: A probabilistic polynomial-time algorithm for computing the square root of a number x \in {\bf Z}/P{\bf Z} , where P = 2^{S}Q + 1(Q odd, s > 0) is a prime number, is described. In contrast to the Adleman, Manders, and Miller algorithm, this algorithm gets faster as s grows. As with the Berlekamp-Rabin algorithm, the expected running time of the algorithm is independent of x . However, the algorithm presented here is considerably faster for values of s greater than 2 .
TL;DR: It is shown that an (n, k, n - k + 1) generalized Reed-Solomon code can be extended by one digit while preserving the MDS property if and only if the resulting extended code is also a generalized Reed, Solomon code.
Abstract: An (n, k, d) linear code over F= GF (q) is said to be {\em maximum distance separable} (MDS) if d = n - k + 1 . It is shown that an (n, k, n - k + 1) generalized Reed-Solomon code such that 2\leq k \leq n - \lfloor (q - 1)/2 \rfloor (k
eq 3 {\rm if} q is even) can be extended by one digit while preserving the MDS property if and only if the resulting extended code is also a generalized Reed-Solomon code. It follows that a generalized Reed-Solomon code with k in the above range can be {\em uniquely} extended to a maximal MDS code of length q + 1 , and that generalized Reed-Solomon codes of length q + 1 and dimension 2\leq k \leq \lfloor q/2 \rfloor + 2 (k
eq 3 {\rm if} q is even) do not have MDS extensions. Hence, in cases where the (q + 1, k) MDS code is essentially unique, (n, k) MDS codes with n > q + 1 do not exist.