Showing papers in "IEEE Transactions on Information Theory in 1989"
TL;DR: Under the assumptions of symbol-synchronous transmissions and white Gaussian noise, the authors analyze the detection mechanism at the receiver, comparing different detectors by their bit error rates in the low-background-noise region and by their worst-case behavior in a near-far environment.
Abstract: Under the assumptions of symbol-synchronous transmissions and white Gaussian noise, the authors analyze the detection mechanism at the receiver, comparing different detectors by their bit error rates in the low-background-noise region and by their worst-case behavior in a near-far environment where the received energies of the users are not necessarily similar. Optimum multiuser detection achieves important performance gains over conventional single-user detection at the expense of computational complexity that grows exponentially with the number of users. It is shown that in the synchronous case the performance achieved by linear multiuser detectors is similar to that of optimum multiuser detection. Attention is focused on detectors whose linear memoryless transformation is a generalized inverse of the matrix of signature waveform crosscorrelations, and on the optimum linear detector. It is shown that the generalized inverse detectors exhibit the same degree of near-far resistance as the optimum multiuser detectors. The optimum linear detector is obtained. >
1,609 citations
TL;DR: Methods for the design and analysis of OOCs, using tools from projective geometry, the greedy algorithm, iterative constructions, algebraic coding theory, block design, and various other combinational disciplines, are discussed.
Abstract: An optical orthogonal code (OOC) is a family of (0,1) sequences with good auto- and cross-correlation properties, i.e the autocorrelation of each sequence exhibits the `thumbtack' shape and the cross correlation between any two sequences remains low throughout. The use of optical orthogonal codes enables a large number of asynchronous users to transmit information efficiently and reliably. The thumbtack-shaped autocorrelation facilitates the detection of the desired signal, and low-profile cross correlation reduces interference from unwanted signals. Theoretical upper and lower bounds on the maximum possible size of OOCs are derived. Methods for the design and analysis of OOCs, using tools from projective geometry, the greedy algorithm, iterative constructions, algebraic coding theory, block design, and various other combinational disciplines, are discussed
1,066 citations
TL;DR: Two complementary approaches are proposed which are intended to clarify the meaning of nonstationary spectra of fractional Brownian motions and reveal an underlying stationary structure relative to each time-scaling.
Abstract: Fractional Brownian motions (FBMs) provide useful models for a number of physical phenomena whose empirical spectra obey power laws of fractional order However, due to the nonstationary nature of these processes, the precise meaning of such spectra remains generally unclear Two complementary approaches are proposed which are intended to clarify this point The first one, based on a time-frequency analysis, takes into account the nonstationary nature of FBM and puts emphasis on time-averaged measurements; the second one, based on a time-scale analysis, is matched to self-similarity properties of FBM and reveals an underlying stationary structure relative to each time-scaling >
478 citations
TL;DR: An algorithm introduced by Breiman et al. (1984) in the context of classification and regression trees is reinterpreted and extended to cover a variety of applications in source coding and modeling in which trees are involved.
Abstract: An algorithm introduced by L. Breiman et al. (1984) in the context of classification and regression trees is reinterpreted and extended to cover a variety of applications in source coding and modeling in which trees are involved. These include variable-rate and minimum-entropy tree-structured vector quantization, minimum expected cost decision trees, variable-order Markov modeling, optimum bit allocation, and computer graphics and image processing using quadtrees. A concentration on the first of these and a detailed analysis of variable-rate tree-structured vector quantization are provided. It is found that variable-rate tree-structured vector quantization outperforms not only the fixed-rate variety but also full-search vector quantization. The successive approximation character of variable-rate tree-structured vector quantization permits it to degrade gracefully if the rate is reduced at the encoder. This has applications to the problem of buffer overflow. >
468 citations
TL;DR: It is shown that the performance of a system incorporating a decision-feedback decoding algorithm that completely recovers this capacity loss is determined by an equivalent genie-aided channel, the capacity of which equals that of the original channel.
Abstract: The Gilbert-Elliott channel, a varying binary symmetric channel, with crossover probabilities determined by a binary-state Markov process, is treated. In general, such a channel has a memory that depends on the transition probabilities between the states. A method of calculating the capacity of this channel is introduced and applied to several examples, and the question of coding is addressed. In the conventional usage of varying channels, a code suitable for memoryless channels is used in conjunction with an interleaver, with the decoder considering the deinterleaved symbol stream as the output of a derived memoryless channel. The transmission rate is limited by the capacity of this memoryless channel, which is often considerably less than the capacity of the original channel. A decision-feedback decoding algorithm that completely recovers this capacity loss is introduced. It is shown that the performance of a system incorporating such an algorithm is determined by an equivalent genie-aided channel, the capacity of which equals that of the original channel. The calculated random coding exponent of the genie-aided channel indicates a considerable increase in the cutoff rate over that of the conventionally derived memoryless channel. >
438 citations
TL;DR: A technique for analyzing time-series data from experiments is presented that provides estimates of four basic characteristics of a system: the measure-theoretic entropy; the accuracy of the measurements; the number of measurements necessary to specify a system state; the best delay time T to use in order to construct phase space portraits by the method of delays.
Abstract: A technique for analyzing time-series data from experiments is presented that provides estimates of four basic characteristics of a system: (1) the measure-theoretic entropy; (2) the accuracy of the measurements; (3) the number of measurements necessary to specify a system state; and (4) the best delay time T to use in order to construct phase space portraits by the method of delays. These characteristics are obtained by separating the entropy of measurements into a part due to noise and parts due to deterministic effects. For the technique to work, the noise associated with each measurement must be independent of the noise associated with all other measurements. An algorithm for implementing the analysis is presented with three examples. >
316 citations
TL;DR: The authors' main finding is that any object among doubly exponentially many objects can be identified in blocklength n with arbitrarily small error probability via a discrete memoryless channel (DMC), if randomization can be used for the encoding procedure.
Abstract: The authors' main finding is that any object among doubly exponentially many objects can be identified in blocklength n with arbitrarily small error probability via a discrete memoryless channel (DMC), if randomization can be used for the encoding procedure. A novel doubly exponential coding theorem is presented which determines the optimal R, that is, the identification capacity of the DMC as a function of its transmission probability matrix. This identification capacity is a well-known quantity, namely, Shannon's transmission capacity for the DMC. >
283 citations
TL;DR: A model of a packet radio network in which transmitters with range R are distributed according to a two-dimensional Poisson point process with density D is examined and it is shown that pi R/sup 2/D, the expected number of nearest neighbors of a transmitter, must grow logarithmically with the area of the network.
Abstract: A model of a packet radio network in which transmitters with range R are distributed according to a two-dimensional Poisson point process with density D is examined To ensure network connectivity, it is shown that pi R/sup 2/D, the expected number of nearest neighbors of a transmitter, must grow logarithmically with the area of the network For an infinite area there exists an infinite connected component with nonzero probability if pi R/sup 2/D>N/sub 0/, for some critical value N/sub 0/ It is shown that 2195 >
281 citations
TL;DR: Theorems concerning the entropy of a stationary ergodic information source are derived and used to obtain insight into the workings of certain data-compression coding schemes, in particular the Lempel-Siv data compression algorithm.
Abstract: Theorems concerning the entropy of a stationary ergodic information source are derived and used to obtain insight into the workings of certain data-compression coding schemes, in particular the Lempel-Siv data compression algorithm. >
279 citations
TL;DR: The signal detection problem is considered for the case in which distributed sensors are used and a global decision is desired, and a person-by-person optimization of the overall system is carried out.
Abstract: The signal detection problem is considered for the case in which distributed sensors are used and a global decision is desired. Local decisions from the sensors are fed to a data fusion center, which yields a global decision based on a fusion rule. A Bayesian formulation of the problem is considered, and a person-by-person optimization of the overall system is carried out. The special case of identical detectors with independent observations is considered as well. An illustrative example is presented. >
242 citations
TL;DR: An asymptotic equipartition theorem for nonstationary Gaussian processes is proved and it is proved that the feedback capacity C/sub FB/ in bits per transmission and the nonfeedback capacity C satisfy C > C >.
Abstract: The capacity of time-varying additive Gaussian noise channels with feedback is characterized. Toward this end, an asymptotic equipartition theorem for nonstationary Gaussian processes is proved. Then, with the aid of certain matrix inequalities, it is proved that the feedback capacity C/sub FB/ in bits per transmission and the nonfeedback capacity C satisfy C >
TL;DR: The decision problem of testing M hypotheses when the source is Kth-order Markov and there are M (or fewer) training sequences of length N and a single test sequence of length n is considered and it is shown that, for this configuration, only one training sequence is needed to achieve an asymptotically optimal test.
Abstract: The decision problem of testing M hypotheses when the source is Kth-order Markov and there are M (or fewer) training sequences of length N and a single test sequence of length n is considered. K, M, n, N are all given. It is shown what the requirements are on M, n, N to achieve vanishing (exponential) error probabilities and how to determine or bound the exponent. A likelihood ratio test that is allowed to produce a no-match decision is shown to provide asymptotically optimal error probabilities and minimum no-match decisions. As an important serial case, the binary hypotheses problem without rejection is discussed. It is shown that, for this configuration, only one training sequence is needed to achieve an asymptotically optimal test. >
TL;DR: It is shown that if the transmitters are assigned the same waveform, symbol asynchronism has no effect on the two-user capacity region of the white Gaussian channel which is equal to the Cover-Syner pentagon, whereas if the assigned waveforms are different, the symbol-asynchronous capacity region is no longer a pentagon.
Abstract: An equivalent discrete-time Gaussian channel parametrized by the signal cross-correlations is derived to obtain an equivalent channel model with discrete-time outputs. The main feature introduced by the lack of symbol synchronism is that the channel has memory. This is due to the overlap of each symbol transmitted by a user with two consecutive symbols transmitted by the other user. It is shown that if the transmitters are assigned the same waveform, symbol asynchronism has no effect on the two-user capacity region of the white Gaussian channel which is equal to the Cover-Syner pentagon, whereas if the assigned waveforms are different (e.g., code division multiple access), the symbol-asynchronous capacity region is no longer a pentagon. An alternative representation of the capacity region which results in a particularly compact characterization of the fundamental limits of the multiple-access channel in the region of signal-to-noise ratios is also considered. >
TL;DR: The authors show that coset codes and indeed any codes which rely on partitioning of the signal set are all subclasses of the proposed coding scheme, leading to reduced complexity in comparison with previously published schemes.
Abstract: Following V.V. Ginzburg (1984), a hierarchy of codes is proposed to match the geometric partitioning of a signal set. The authors show that coset codes (including Ungerboeck, lattice, and binary codes) and indeed any codes which rely on partitioning of the signal set are all subclasses of the proposed coding scheme. The combination of such codes in a multilevel scheme often leads to reduced complexity in comparison with previously published schemes. A variety of decoder structures is discussed. >
TL;DR: The authors consider how much performance advantage a fixed-dimensional vector quantizer can gain over a scalar quantizer and collects several results from high-resolution or asymptotic quantization theory to identify source and system characteristics that contribute to the vectorquantizer advantage.
Abstract: The authors consider how much performance advantage a fixed-dimensional vector quantizer can gain over a scalar quantizer. They collect several results from high-resolution or asymptotic (in rate) quantization theory and use them to identify source and system characteristics that contribute to the vector quantizer advantage. One well-known advantage is due to improvement in the space-filling properties of polytopes as the dimension increases. Others depend on the source's memory and marginal density shape. The advantages are used to gain insight into product, transform, lattice, predictive, pyramid, and universal quantizers. Although numerical prediction consistently overestimated gains in low rate (1 bit/sample) experiments, the theoretical insights may be useful even at these rates. >
TL;DR: A highly reliable, accurate, and efficient method of calculating the probability of detection, P/sub N/(X,Y), for N incoherently integrated samples, where X is the constant received signal-to-noise ratio of a single pulse and Y is the normalized threshold level.
Abstract: A highly reliable, accurate, and efficient method of calculating the probability of detection, P/sub N/(X,Y), for N incoherently integrated samples, where X is the constant received signal-to-noise ratio of a single pulse and Y is the normalized threshold level, is presented. The useful range of parameters easily exceeds most needs. On a VAX/11 computer with double precision calculations, better than 13-place absolute accuracy is normally achieved. There is a gradual loss of accuracy with increasing parameter values. For example, for N=10/sup 9/, and with both NX and Y near 10/sup 7/, the accuracy can drop to ten places. The function P/sub N/(X,Y) can be equated to the generalized Marcum Q-function, Q/sub m/( alpha , beta ). The corresponding limits on alpha and beta are roughly 4500 for the 13-place accuracy and 60000 for ultimate (INTEGER*4) limit. >
TL;DR: Entropy expressions for several continuous multivariate distributions are derived and the asymptotic distribution of the uniformly minimum variance unbiased estimator for multinormal entropy is obtained.
Abstract: Entropy expressions for several continuous multivariate distributions are derived. Point estimation of entropy for the multinormal distribution and for the distribution of order statistics from D.G. Weinman's (Ph.D dissertation, Ariz. State Univ., Tempe, AZ, 1966) exponential distribution is considered. The asymptotic distribution of the uniformly minimum variance unbiased estimator for multinormal entropy is obtained. Simulation results on convergence of the means and variances of these estimators are provided. >
TL;DR: The authors determine the admissible rate region R (D), where D is the distortion of the reconstruction of (Y/sub k/) and the binary Hamming case is investigated explicitly.
Abstract: The authors unify earlier investigations concerning the encoding of two correlated sources (X/sub k/), (Y/sub k/) by means of separate encoders. Decoding is done by a single decoder which receives the outputs from both encoders. The reconstruction of (X/sub k/) is required to be perfect in the usual Shannon sense. The authors determine the admissible rate region R (D), where D is the distortion of the reconstruction of (Y/sub k/). The binary Hamming case is investigated explicitly. >
TL;DR: The authors estimate the order of a finite Markov source based on empirically observed statistics and propose a universal asymptotically optimal test for the case where a given integer is known to be the upper bound of the true order.
Abstract: The authors estimate the order of a finite Markov source based on empirically observed statistics. The performance criterion adopted is to minimize the probability of underestimating the model order while keeping the overestimation probability exponent at a prescribed level. A universal asymptotically optimal test, in the sense just defined, is proposed for the case where a given integer is known to be the upper bound of the true order. For the case where such a bound is unavailable, an alternative rule based on the Lempel-Ziv data compression algorithm is shown to be asymptotically optimal also and computationally more efficient. >
TL;DR: In this paper, the capacity region of the discrete memoryless single-output two-way channel was shown to be bounded by the idea that no more dependence can be consumed than is produced.
Abstract: If in a transmission the inputs of a single-output two-way channel exhibit some interdependence, this dependence must have been created during earlier transmissions. The idea that no more dependence can be consumed than is produced is used to obtain new upper bounds to the capacity region of the discrete memoryless single-output two-way channel. With these upper bounds it is shown that C.E. Shannon' (1961) inner bound region is the capacity region for channels in a certain class, and the Zhang-Berger-Schalkwijk upper bound (1986) for Blackwell's multiplying channel is improved upon. >
TL;DR: The least storage and node computation required by a breadth-first tree or trellis decoder that corrects t errors over the binary symmetric channels is calculated.
Abstract: The least storage and node computation required by a breadth-first tree or trellis decoder that corrects t errors over the binary symmetric channels is calculated. Breadth-first decoders work with code paths of the same length, without backtracking. The Viterbi algorithm is an exhaustive trellis decoder of this type; other schemes look at a subset of the tree or trellis paths. For random tree codes, theorems about the asymptotic number of paths required and their depth are proved. For concrete convolutional codes, the worst case storage for t error sequences is measured. In both cases the optimal decoder storage has the same simple dependence on t. The M algorithm and algorithms proposed by G.J. Foschini (ibid., vol.IT-23, p.605-9, Sept. 1977) and by S.J. Simmons (PhD. diss., Queens Univ., Kingston, Ont., Canada) are optimal, or nearly so; they are all far more efficient than the Viterbi algorithm. >
TL;DR: A fast algorithm for searching a tree (FAST) is presented for computing the distance spectrum of convolutional codes and several of the listed encoders have distance spectra superior to those of any previously known codes of the same rate and memory.
Abstract: A fast algorithm for searching a tree (FAST) is presented for computing the distance spectrum of convolutional codes. The distance profile of a code is used to limit substantially the error patterns that have to be searched. The algorithm can easily be modified to determine the number of nonzero information bits of an incorrect path as well as the length of an error event. For testing systematic codes, a faster version of the algorithm is given. FAST is much faster than the standard bidirectional search. On a microVAX, d/sub infinity /=27 was verified for a rate R=1/2, memory M=25 code in 37 s of CPU time. Extensive tables of rate R=1/2 encoders are given. Several of the listed encoders have distance spectra superior to those of any previously known codes of the same rate and memory. A conjecture than an R=1/2 systematic convolutional code of memory 2M will perform as well as a nonsystematic convolutional code of memory M is given strong support. >
IBM1
TL;DR: Performing maximum-likelihood decoding in a linear block error-correcting code is shown to be equivalent to finding a global maximum of the energy function of a certain neural network, and the connection between maximization of polynomials over the n-cube is investigated.
Abstract: Several ways of relating the concept of error-correcting codes to the concept of neural networks are presented. Performing maximum-likelihood decoding in a linear block error-correcting code is shown to be equivalent to finding a global maximum of the energy function of a certain neural network. Given a linear block code, a neural network can be constructed in such a way that every codeword corresponds to a local maximum. The connection between maximization of polynomials over the n-cube and error-correcting codes is also investigated; the results suggest that decoding techniques can be a useful tool for solving such maximization problems. The results are generalized to both nonbinary and nonlinear codes. >
TL;DR: A decoding algorithm is constructed which turns out to be a generalization of the Peterson algorithm for decoding BCH decoder codes.
Abstract: A class of codes derived from algebraic plane curves is constructed. The concepts and results from algebraic geometry that were used are explained in detail; no further knowledge of algebraic geometry is needed. Parameters, generator and parity-check matrices are given. The main result is a decoding algorithm which turns out to be a generalization of the Peterson algorithm for decoding BCH decoder codes. >
TL;DR: A study is made of the identification problem in the presence of a noiseless feedback channel, and the second-order capacity C/sub f/ for deterministic (resp. randomized) encoding strategies is determined.
Abstract: A study is made of the identification problem in the presence of a noiseless feedback channel, and the second-order capacity C/sub f/ (resp. C/sub F/) for deterministic (resp. randomized) encoding strategies is determined. Several important phenomena are encountered. (1) Although feedback does not increase the transmission capacity of a discrete memoryless channel (DMC), it does increase the (second-order) identification capacity; (2) noise increases C/sub f/; (3) the structure of the new capacity formulas is simpler than C.E. Shannon's (1948) familiar formula. This has the effect that proofs of converses become easier than in the authors' previous work. >
TL;DR: A binary multiple-check generalization of the Wagner rule is presented, and two methods for its implementation, one of which resembles the suboptimal Forney-Chase algorithms, are described.
Abstract: Maximum-likelihood soft-decision decoding of linear block codes is addressed. A binary multiple-check generalization of the Wagner rule is presented, and two methods for its implementation, one of which resembles the suboptimal Forney-Chase algorithms, are described. Besides efficient soft decoding of small codes, the generalized rule enables utilization of subspaces of a wide variety, thereby yielding maximum-likelihood decoders with substantially reduced computational complexity for some larger binary codes. More sophisticated choice and exploitation of the structure of both a subspace and the coset representatives are demonstrated for the (24, 12) Golay code, yielding a computational gain factor of about 2 with respect to previous methods. A ternary single-check version of the Wagner rule is applied for efficient soft decoding of the (12, 6) ternary Golay code. >
TL;DR: It is shown that when the channel has memory, frame asynchronism rules out nonstationary inputs to achieve any point in the capacity region, thereby allowing only coding strategies that involve cooperation in the frequency domain but not in the time domain.
Abstract: The capacity region of frame-synchronous and frame-asynchronous, discrete, two-user multiple-access channels with finite memory is obtained. Frame synchronism refers to the ability of the transmitters to send their code words in unison. The absence of frame synchronism in memoryless multiple-access channels is known to result in the removal of the convex hull operation from the expression of the capacity region. It is shown that when the channel has memory, frame asynchronism rules out nonstationary inputs to achieve any point in the capacity region, thereby allowing only coding strategies that involve cooperation in the frequency domain but not in the time domain. This restriction drastically reduces the capacity region of some multiple-access channels with memory, and in particular the total capacity of the channel, which is invariant to frame asynchronism for memoryless channels. >
TL;DR: A theoretical basis is provided for multistage sigma-delta modulation (MSM), which is a cascade realization of several single-loop sigma/delta modulators with a linear combinatorial network, and the noise-shaping characteristic of MSM is investigated.
Abstract: A theoretical basis is provided for multistage sigma-delta modulation (MSM), which is a cascade realization of several single-loop sigma-delta modulators with a linear combinatorial network. Equations are derived describing the output and the quantization noise of MSM for an arbitrary input signal, and the noise-shaping characteristic of MSM is investigated. The spectral characteristics of an m-stage sigma-delta modulator with both DC and sinusoidal inputs are developed. For both types of inputs the binary quantizer noise of the mth (m>or=3) quantizer, which appears at the output as an mth order difference, is asymptotically white, uniformly distributed, and uncorrelated with the input level. It is also found that for an m-stage sigma-delta quantizer with either an ideal low-pass filter or a sinc/sup m+1/ filter decoder, the average quantization noise of the system is inversely proportional to the (2m+1)th power of the oversampling ratio. This implies that the high-order systems are favourable in terms of the trade-off between the quantization noise and oversampling ratio. Simulation results are presented to support the theoretical analysis. >
TL;DR: A special form of Cauchy matrix is used to obtain a tighter bound for the validity region of the maximum distance separable (MDS) conjecture and a new compact characterization of generalized Reed-Solomon codes.
Abstract: A special form of Cauchy matrix is used to obtain a tighter bound for the validity region of the maximum distance separable (MDS) conjecture and a new compact characterization of generalized Reed-Solomon codes. The latter is further used to obtain constructions and some existence results for long (2k, k) double-circulant MDS codes. >