Showing papers in "IEEE Transactions on Information Theory in 1990"
TL;DR: Two different procedures for effecting a frequency analysis of a time-dependent signal locally in time are studied and the notion of time-frequency localization is made precise, within this framework, by two localization theorems.
Abstract: Two different procedures for effecting a frequency analysis of a time-dependent signal locally in time are studied. The first procedure is the short-time or windowed Fourier transform; the second is the wavelet transform, in which high-frequency components are studied with sharper time resolution than low-frequency components. The similarities and the differences between these two methods are discussed. For both schemes a detailed study is made of the reconstruction method and its stability as a function of the chosen time-frequency density. Finally, the notion of time-frequency localization is made precise, within this framework, by two localization theorems. >
6,180 citations
TL;DR: A necessary and sufficient condition for blind deconvolution (without observing the input) of nonminimum-phase linear time-invariant systems (channels) is derived and several optimization criteria are proposed, and their solution is shown to correspond to the desired response.
Abstract: A necessary and sufficient condition for blind deconvolution (without observing the input) of nonminimum-phase linear time-invariant systems (channels) is derived. Based on this condition, several optimization criteria are proposed, and their solution is shown to correspond to the desired response. These criteria involve the computation only of second- and fourth-order moments, implying a simple tap update procedure. The proposed methods are universal in the sense that they do not impose any restrictions on the probability distribution of the (unobserved) input sequence. It is shown that in several important cases (e.g. when the additive noise is Gaussian), the proposed criteria are essentially unaffected. >
843 citations
TL;DR: A cryptanalytic attack on the use of short RSA secret exponents is described, which poses no threat to the normal case of RSA where the secret exponent is approximately the same size as the modulus.
Abstract: A cryptanalytic attack on the use of short RSA secret exponents is described. The attack makes use of an algorithm based on continued fractions that finds the numerator and denominator of a fraction in polynomial time when a close enough estimate of the fraction is known. The public exponent e and the modulus pq can be used to create an estimate of a fraction that involves the secret exponent d. The algorithm based on continued fractions uses this estimate to discover sufficiently short secret exponents. For a typical case where e>pq, GCD(p-1, q-1) is small, and p and q have approximately the same number of bits, this attack will discover secret exponents with up to approximately one-quarter as may bits as the modulus. Ways to combat this attack, ways to improve it, and two open problems are described. This attack poses no threat to the normal case of RSA where the secret exponent is approximately the same size as the modulus. This is because the attack uses information provided by the public exponent and, in the normal case, the public exponent can be chosen almost independently of the modulus. >
657 citations
TL;DR: It is shown that, if the input alphabet contains a zero-cost symbol, then the capacity per unit cost admits a simple expression as the maximum normalized divergence between two conditional output distributions.
Abstract: Memoryless communication channels with arbitrary alphabets where each input symbol is assigned a cost are considered. The maximum number of bits that can be transmitted reliably through the channel per unit cost is studied. It is shown that, if the input alphabet contains a zero-cost symbol, then the capacity per unit cost admits a simple expression as the maximum normalized divergence between two conditional output distributions. The direct part of this coding theorem admits a constructive proof via Stein's lemma on the asymptotic error probability of binary hypothesis tests. Single-user, multiple-access, and interference channels are studied. >
582 citations
TL;DR: The authors examine the relative entropy distance D/sub n/ between the true density and the Bayesian density and show that the asymptotic distance is (d/2)(log n)+c, where d is the dimension of the parameter vector.
Abstract: In the absence of knowledge of the true density function, Bayesian models take the joint density function for a sequence of n random variables to be an average of densities with respect to a prior. The authors examine the relative entropy distance D/sub n/ between the true density and the Bayesian density and show that the asymptotic distance is (d/2)(log n)+c, where d is the dimension of the parameter vector. Therefore, the relative entropy rate D/sub n//n converges to zero at rate (log n)/n. The constant c, which the authors explicitly identify, depends only on the prior density function and the Fisher information matrix evaluated at the true parameter value. Consequences are given for density estimation, universal data compression, composite hypothesis testing, and stock-market portfolio selection. >
517 citations
TL;DR: It is concluded that the channel-optimized vector quantizer design algorithm, if used carefully, can result in a fairly robust system with no additional delay.
Abstract: Several issues related to vector quantization for noisy channels are discussed. An algorithm based on simulated annealing is developed for assigning binary codewords to the vector quantizer code-vectors. It is shown that this algorithm could result in dramatic performance improvements as compared to randomly selected codewords. A modification of the simulated annealing algorithm for binary codeword assignment is developed for the case where the bits in the codeword are subjected to unequal error probabilities (resulting from unequal levels of error protection). An algorithm for the design of an optimal vector quantizer for a noisy channel is briefly discussed, and its robustness under channel mismatch conditions is studied. Numerical results for a stationary first-order Gauss-Markov source and a binary symmetric channel are provided. It is concluded that the channel-optimized vector quantizer design algorithm, if used carefully, can result in a fairly robust system with no additional delay. The case in which the communication channel is nonstationary (as in mobile radio channels) is studied, and some preliminary ideas for quantizer design are presented. >
509 citations
TL;DR: Exact formulas for quantizer noise spectra are developed and several results describing the behavior of quantization noise in a unified and simplified manner are discussed.
Abstract: Several results describing the behavior of quantization noise in a unified and simplified manner are discussed. Exact formulas for quantizer noise spectra are developed. They are applied to a variety of systems and inputs, including scalar quantization (PCM), dithered PCM, sigma-delta modulation, dithered sigma-delta modulation, two-stage sigma-delta modulation, and second-order sigma-delta modulation. >
472 citations
TL;DR: The known techniques for constructing constant weight codes are surveyed, and a table of (unrestricted) binary codes of length nl28 is given.
Abstract: A table of binary constant weight codes of length nl28 is presented. Explicit constructions are given for most of the 600 codes in the table; the majority of these codes are new. The known techniques for constructing constant weight codes are surveyed, and a table of (unrestricted) binary codes of length nl28 is given
459 citations
TL;DR: It is shown that the minimal distance d of a binary self-dual code of length n>or=74 is at most 2((n+6)/10).
Abstract: It is shown that the minimal distance d of a binary self-dual code of length n>or=74 is at most 2((n+6)/10). This bound is a consequence of some new conditions on the weight enumerator of a self-dual code obtained by considering a particular translate of the code, called its shadow. These conditions also enable one to find the highest possible minimal distance of a self-dual code for all n>or=60; to show that self-dual codes with d or=22, with d>or=8 exist precisely for n=24, 32 and n>or=26, and with d>or=10 exist precisely for n>or=46; and to show that there are exactly eight self-dual codes of length 32 with d=8. Several of the self-dual codes of length 34 have trivial group (this appears to be the smallest length where this can happen). >
384 citations
TL;DR: A Chapman-Robbins form of the Barankin bound is used to derive a multiparameter Cramer-Rao (CR) type lower bound on estimator error covariance when the parameter theta in R/sup n/ is constrained to lie in a subset of the parameter space.
Abstract: A Chapman-Robbins form of the Barankin bound is used to derive a multiparameter Cramer-Rao (CR) type lower bound on estimator error covariance when the parameter theta in R/sup n/ is constrained to lie in a subset of the parameter space. A simple form for the constrained CR bound is obtained when the constraint set Theta /sub C/, can be expressed as a smooth functional inequality constraint. It is shown that the constrained CR bound is identical to the unconstrained CR bound at the regular points of Theta /sub C/, i.e. where no equality constraints are active. On the other hand, at those points theta in Theta /sub C/ where pure equality constraints are active the full-rank Fisher information matrix in the unconstrained CR bound must be replaced by a rank-reduced Fisher information matrix obtained as a projection of the full-rank Fisher matrix onto the tangent hyperplane of the full-rank Fisher matrix onto the tangent hyperplane of the constraint set at theta . A necessary and sufficient condition involving the forms of the constraint and the likelihood function is given for the bound to be achievable, and examples for which the bound is achieved are presented. In addition to providing a useful generalization of the CR bound, the results permit analysis of the gain in achievable MSE performance due to the imposition of particular constraints on the parameter space without the need for a global reparameterization. >
350 citations
TL;DR: The authors determine the weights of the orthogonals of some binary linear codes; the Melas code of length, the irreducible cyclic binary codes of length 2/sup t/+1, and the extended binary Goppa codes defined by polynomials of degree two.
Abstract: Starting from results on elliptic curves and Kloosterman sums over the finite field GE(2/sup t/), the authors determine the weights of the orthogonals of some binary linear codes; the Melas code of length, the irreducible cyclic binary code of length 2/sup t/+1, and the extended binary Goppa codes defined by polynomials of degree two. >
TL;DR: Orthonormal wavelet bases are used to provide a new construction for nearly 1/f processes from a set of uncorrelated random variables.
Abstract: While so-called 1/f or scaling processes emerge regularly in modeling a wide range of natural phenomena, as yet no entirely satisfactory framework has been described for the analysis of such processes. Orthonormal wavelet bases are used to provide a new construction for nearly 1/f processes from a set of uncorrelated random variables. >
TL;DR: A systematic approach to partitioning L*MPSK signal sets that is based on block coding is used and an encoder system approach is developed that incorporates the design of a differential precoder, a systematic convolutional encoder, and a signal set mapper.
Abstract: A 2L-dimensional multiple phase-shift keyed (L*MPSK) signal set is obtained by forming the Cartesian product of L two-dimensional MPSK signal sets. A systematic approach to partitioning L*MPSK signal sets that is based on block coding is used. An encoder system approach is developed. It incorporates the design of a differential precoder, a systematic convolutional encoder, and a signal set mapper. Trellis-coded L*4PSK, L*8PSK, and L*16PSK modulation schemes are found for 1 >
TL;DR: In this paper, a nonequiprobable signaling scheme for the Gaussian channel based on finite-dimensional lattices is proposed, where a signal constellation, Omega, is partitioned into T subconstellations Omega /sub 0/,..., Omega/sub tau -1/ of equal size by scaling a basic region R. Signal points in the same subconstellation are used equiprobably.
Abstract: Signaling schemes for the Gaussian channel based on finite-dimensional lattices are considered. The signal constellation consists of all lattice points within a region R, and the shape of this region determines the average signal power. Spherical signal constellations minimize average signal power, and in the limit as N to infinity , the shape gain of the N-sphere over the N-cube approaches pi e/6 approximately=1.53 dB. A nonequiprobable signaling scheme is described that approaches this full asymptotic shape gain in any fixed dimension. A signal constellation, Omega is partitioned into T subconstellations Omega /sub 0/, . . ., Omega /sub tau -1/ of equal size by scaling a basic region R. Signal points in the same subconstellation are used equiprobably, and a shaping code selects the subconstellation Omega /sub i/ with frequency f/sub i/. Shaping codes make it possible to achieve any desired fractional bit rate. The schemes presented are compared with equiprobable signaling schemes based on Voronoi regions of multidimensional lattices. For comparable shape gain and constellation expansion ratio, the peak to average power ratio of the schemes presented is superior. Furthermore, a simple table lookup is all that is required to address points in the constellations. It is also shown that it is possible to integrate coding and nonequiprobable signaling within a common multilevel framework. >
[...]
TL;DR: For certain parameters the power spectral density exhibits 1/f-type behavior over a substantial range of frequencies, so that the process serves as a source of 1/ f/sup alpha / shot noise for alpha in the range 0 >.
Abstract: The behavior of power-law shot noise, for which the associated impulse response functions assume a decaying power-law form, is explored. Expressions are obtained for the moments, moment generating functions, amplitude probability density functions, autocorrelation functions, and power spectral densities for a variety of parameters of the process. For certain parameters the power spectral density exhibits 1/f-type behavior over a substantial range of frequencies, so that the process serves as a source of 1/f/sup alpha / shot noise for alpha in the range 0 >
TL;DR: It is shown how large-deviations theory can be applied to construct an asymptotically efficient simulation distribution for Monte Carlo simulation using the importance sampling technique.
Abstract: It is shown how large-deviations theory can be applied to construct an asymptotically efficient simulation distribution for Monte Carlo simulation using the importance sampling technique. A sufficient and necessary condition is given for the asymptotic efficiency of the candidate simulation distributions. This is done in the multidimensional setting that is required by many practical simulation problems. The result obtained is applied primarily in two areas. First, the generalization of previous work dealing with functionals of Markov chains is discussed. A second area of application is the simulation of nonlinear systems with Gaussian inputs. As an example of a system with Gaussian inputs, the simulation of a digital communication channel with nonlinear ISI (intersymbol interference) characteristic of satellite data links is considered. In the case of linear ISI, the asymptotically efficient exponentially twisted distribution turns out to agree with a method previously proposed by D. Lu and K. Yao (1988). The large-deviations point of view provides some useful insight on how to extent the Lu and Yao method to nonlinear channels. >
TL;DR: Three distinct upper bounds on the size of an OOC are presented that, for many values of the parameter set (n, omega , lambda ), improve upon the tightest previously known bound.
Abstract: A technique for constructing optimal OOCs (optical orthogonal codes) is presented. It provides the only known family of optimal (with respect to family size) OOCs having lambda =2. The parameters (n, omega , lambda ) are respectively (p/sup 2m/-1, p/sup m/+1,2), where p is any prime and the family size is p/sup m/-2. Three distinct upper bounds on the size of an OOC are presented that, for many values of the parameter set (n, omega , lambda ), improve upon the tightest previously known bound. >
TL;DR: A linear technique for combining equalization and coset codes on partial response channels with additive white Gaussian noise is developed and performance advantages are presented with respect to zero-forcing decision feedback methods that use the same coset code on the same partial response channel.
Abstract: A linear technique for combining equalization and coset codes on partial response channels with additive white Gaussian noise is developed. The technique, vector coding, uses a set of transmit filters or 'vectors' to partition the channel into an independent set of parallel intersymbol interference (ISI)-free channels for any given finite (or infinite) block length. The optimal transmit vectors for such channel partitioning are shown to be the eigenvectors of the channel covariance matrix for the specified block length, and the gains of the individual channels are the eigenvalues. An optimal bit allocation and energy distribution, are derived for the set of parallel channels, under an accurate extension of the continuous approximation for power in optimal multidimensional signal sets for constellations with unequal signal spacing in different dimensions. Examples are presented that demonstrate performance advantages with respect to zero-forcing decision feedback methods that use the same coset code on the same partial response channel. Only resampling the channel at an optimal rate and assuming no errors in the feedback path will bring the performance of the decision feedback methods up to the level of the vector coded system. >
TL;DR: The following problem is addressed: given that the peripheral encoders that satisfy capacity constraints are scalar quantizers, how should they be designed in order that the central test to be performed on their output indices is most powerful?
Abstract: In a decentralized hypothesis testing network, several peripheral nodes observe an environment and communicate their observations to a central node for the final decision. The presence of capacity constraints introduces theoretical and practical problems. The following problem is addressed: given that the peripheral encoders that satisfy these constraints are scalar quantizers, how should they be designed in order that the central test to be performed on their output indices is most powerful? The scheme is called cooperative design-separate encoding since the quantizers process separate observations but have a common goal; they seek to maximize a system-wide performance measure. The Bhattacharyya distance of the joint index space as such a criterion is suggested, and a design algorithm to optimize arbitrarily many quantizers cyclically is proposed. A simplified version of the algorithm, namely an independent design-separate encoding scheme, where the correlation is either absent or neglected for the sake of simplicity, is outlined. Performances are compared through worked examples. >
IBM1
TL;DR: The problem of maximum-likelihood decoding of linear block codes is known to be hard but the fact that the problem remains hard even if the code is known in advance, and can be preprocessed for as long as desired in order to device a decoding algorithm, is shown.
Abstract: The problem of maximum-likelihood decoding of linear block codes is known to be hard. The fact that the problem remains hard even if the code is known in advance, and can be preprocessed for as long as desired in order to device a decoding algorithm, is shown. The hardness is based on the fact that existence of a polynomial-time algorithm implies that the polynomial hierarchy collapses. Thus, some linear block codes probably do not have an efficient decoder. The proof is based on results in complexity theory that relate uniform and nonuniform complexity classes. >
TL;DR: A two-message protocol is described, and its worst case performance is investigated.
Abstract: The reduction in communication achievable by interaction is investigated. The model assumes two communicators: an informant having a random variable X, and a recipient having a possibly dependent random variable Y. Both communicators want the recipient to learn X with no probability of error, whereas the informant may or may not learn Y. To that end, they alternate in transmitting messages comprising finite sequences of bits. Messages are transmitted over an error-free channel and are determined by an agreed-upon, deterministic protocol for (X,Y) (i.e. a protocol for transmitting X to a person who knows Y). A two-message protocol is described, and its worst case performance is investigated. >
TL;DR: An optimal class of distances satisfying an orthogonality condition analogous to that enjoyed by linear projections in Hilbert space is derived and possess the geometric properties of cross entropy useful in speech and image compression, pattern classification, and cluster analysis.
Abstract: Minimum distance approaches are considered for the reconstruction of a real function from finitely many linear functional values. An optimal class of distances satisfying an orthogonality condition analogous to that enjoyed by linear projections in Hilbert space is derived. These optimal distances are related to measures of distances between probability distributions recently introduced by C.R. Rao and T.K. Nayak (1985) and possess the geometric properties of cross entropy useful in speech and image compression, pattern classification, and cluster analysis. Several examples from spectrum estimation and image processing are discussed. >
TL;DR: A new quantity called the time-frequency coherence (TFC) is defined, and it is demonstrated that its properties are analogous to those possessed by the stationary coherence function.
Abstract: Consideration is given to the generalization of stationary cross spectral analysis methods to a class of nonstationary processes, specifically, the class of semistationary finite energy processes possessing sample functions that are of finite energy almost surely. A new quantity called the time-frequency coherence (TFC) is defined, and it is demonstrated that its properties are analogous to those possessed by the stationary coherence function. The problem of estimating the TFC by using elements of L. Cohen's (1966) class of joint time-frequency representations is investigated. It is shown that the only admissible estimators are those based on the class of time-frequency smoothed periodograms. Thus, the familiar procedure of segmentation and (smoothed) short-time Fourier analysis cannot be improved upon (within the framework considered) by the use of the higher-resolution nonparametric time-frequency methods. Procedures for selection of the appropriate estimators and a possible application are suggested. >
TL;DR: It is shown that pseudocyclic MDS codes exist if and only if the multiplicative order of a divides (q-1)/n, and that when this condition is satisfied, such codes exist for all k.
Abstract: The (n, k) pseudocyclic maximum-distance-separable (MDS) codes modulo (x/sup n/-a) over GF(q) are considered. Suppose that n is a divisor of q+1. If n is odd, pseudocyclic MDS codes exist for all k. However, if n is even, nontrivial pseudocyclic MDS codes exist for odd k (but not for even k) if a is a quadratic residue in GF(q), and they exist for even k (but not for odd k) if a is not a quadratic residue in GF(q). Also considered is the case when n is a divisor of q-1, and it is shown that pseudocyclic MDS codes exist if and only if the multiplicative order of a divides (q-1)/n, and that when this condition is satisfied, such codes exist for all k. If the condition is not satisfied, every pseudocyclic code of length n is the result of interleaving a shorter pseudocyclic code. >
TL;DR: A code construction for a T active users out of N multiple-access system (TANMAS) is discussed and it is shown that this set can be identified uniquely (in addition to unique decodability) provided that at most T/2 users are active simultaneously.
Abstract: A code construction for a T active users out of N multiple-access system (TANMAS) is discussed. The multiple-access channel (MAC) that is used in the TANMAS is a discrete-time noiseless real adder channel used without feedback with N real or binary inputs. Each input may be affected by an unknown, slowly varying channel gain and by an unknown, slowly varying channel offset. A set of N codes is constructed such that the sum of codewords of any known set of T or less active users is uniquely decodable. The sum rate of the codes approaches one from above if the input alphabet of the MAC is binary, and it approaches one from below if the input symbols are real numbers. For the case when the set of active users is unknown. For the case when the set of active users is unknown, it is shown that this set can be identified uniquely (in addition to unique decodability) provided that at most T/2 users are active simultaneously. The sum rate is then reduced to 1/2, which is approached from above and from below, respectively, for binary and for real channel inputs. A simple decoding algorithm which operates over the reals is given for the case when the set of active users is known. It is pointed out that one of the applications of the given codes is in hybrid multiple-access systems that use both multiple-access coding and collision resolution. >
TL;DR: Existence conditions and recursive construction procedures for sets of periodic complementary binary sequences are given and relationships to sets of aperiodic complementary Binary sequences and to perfect binary arrays are noted.
Abstract: Existence conditions and recursive construction procedures for sets of periodic complementary binary sequences are given Relationships to sets of aperiodic complementary binary sequences and to perfect binary arrays, whose two-dimensional periodic autocorrelation function is a delta function, are noted The connections of periodic complementary binary sequences and difference families are given Sets of periodic complementary binary sequences, which result from computer search, are presented A diagram showing what is currently known about the existence of periodic complementary binary sequences with P >
TL;DR: It is shown that the difference between the average entropy of the individual order statistics and the entropy of a member of the original independent identically distributed (IID) population is a constant, regardless of theOriginal distribution.
Abstract: The entropy of a sequence of random variables under order restrictions is examined. A theorem that shows the amount of entropy reduction when the sequence is ordered is presented. Upper and lower bounds to the entropy reduction and conditions under which they are achieved are derived. Some interesting properties of the entropy of the individual order statistics are also presented. It is shown that the difference between the average entropy of the individual order statistics and the entropy of a member of the original independent identically distributed (IID) population is a constant, regardless of the original distribution. Finally, the entropies of the individual order statistics are found to be symmetric about the median when the probability density function (PDF) of the original IID sequence is symmetric about its mean. >
TL;DR: Two applications are considered where the probability of the error patterns being linearly dependent decreases exponentially with r, and a decoding techinque is described for which complexity increases no greater than as n/sup 3/, for any choice of code.
Abstract: Code symbols are treated as vectors in an r-dimensional vector space F/sup r/ over a field F. Given any (n, k) linear block code over F with minimum distance d, it is possible to derive an (n, k) code with symbols over F/sup r/, also with minimum distance d, which can correct any pattern of d-2 or fewer symbol errors for which the symbol errors as vectors are linearly independent. This is about twice the bound on the number of errors guaranteed to be correctable. Furthermore, if the error vectors are linearly dependent and d-2 or fewer in number, the existence of dependence can always be detected. A decoding techinque is described for which complexity increases no greater than as n/sup 3/, for any choice of code. For the two applications considered, situations are described where the probability of the error patterns being linearly dependent decreases exponentially with r. >
TL;DR: An analytic confirmation of an observation made by G. Ungerboeck (1965) that approximately optimal performance on a Gaussian channel with capacity C can be obtained with about 2/sup C+1/ levels is given.
Abstract: An analytic confirmation of an observation made by G. Ungerboeck (1965) that approximately optimal performance on a Gaussian channel with capacity C can be obtained with about 2/sup C+1/ levels is given. In particular, the results imply that by using about 2/sup C-1/ levels, a rate of nearly C-1 bit can be achieved, and that by using 2/sup C+1/ levels a rate of about C-0.4 bits can be achieved. >
TL;DR: The main result is to show that the minimum distance between received signals is the same as the pulse energy for rates of transmission about 25% beyond the Nyquist rate, which is the best possible result.
Abstract: The degradation suffered when pulses satisfying the Nyquist criterion are used to transmit binary data at a rate faster than the Nyquist rate over the ideal band-limited (brick-wall) channel is studied. The minimum distance between received signals is used as a performance criterion. It is well-known that when Nyquist pulses (i.e. pulses satisfying the Nyquist criterion) are sent at the Nyquist rate, the minimum distance between signal points is the same as the pulse energy. The main result is to show that the minimum distance between received signals is the same as the pulse energy for rates of transmission about 25% beyond the Nyquist rate, which is the best possible result. In fact, one can even identify the precise error event and signaling rate that causes the minimum distance to be no longer equal to the pulse energy. The mathematical formulation of the problem is analyzed. >