Showing papers in "IEEE Transactions on Information Theory in 1992"
TL;DR: It is proven that the local maxima of the wavelet transform modulus detect the locations of irregular structures and provide numerical procedures to compute their Lipschitz exponents.
Abstract: The mathematical characterization of singularities with Lipschitz exponents is reviewed. Theorems that estimate local Lipschitz exponents of functions from the evolution across scales of their wavelet transform are reviewed. It is then proven that the local maxima of the wavelet transform modulus detect the locations of irregular structures and provide numerical procedures to compute their Lipschitz exponents. The wavelet transform of singularities with fast oscillations has a particular behavior that is studied separately. The local frequency of such oscillations is measured from the wavelet transform modulus maxima. It has been shown numerically that one- and two-dimensional signals can be reconstructed, with a good approximation, from the local maxima of their wavelet transform modulus. As an application, an algorithm is developed that removes white noises from signals by analyzing the evolution of the wavelet transform maxima across scales. In two dimensions, the wavelet transform maxima indicate the location of edges in images. >
4,064 citations
TL;DR: Adapted waveform analysis uses a library of orthonormal bases and an efficiency functional to match a basis to a given signal or family of signals, and relies heavily on the remarkable orthogonality properties of the new libraries.
Abstract: Adapted waveform analysis uses a library of orthonormal bases and an efficiency functional to match a basis to a given signal or family of signals. It permits efficient compression of a variety of signals, such as sound and images. The predefined libraries of modulated waveforms include orthogonal wavelet-packets and localized trigonometric functions, and have reasonably well-controlled time-frequency localization properties. The idea is to build out of the library functions an orthonormal basis relative to which the given signal or collection of signals has the lowest information cost. The method relies heavily on the remarkable orthogonality properties of the new libraries: all expansions in a given library conserve energy and are thus comparable. Several cost functionals are useful; one of the most attractive is Shannon entropy, which has a geometric interpretation in this context. >
3,307 citations
TL;DR: Two examples of jointly shiftable transforms that are simultaneously shiftable in more than one domain are explored and the usefulness of these image representations for scale-space analysis, stereo disparity measurement, and image enhancement is demonstrated.
Abstract: One of the major drawbacks of orthogonal wavelet transforms is their lack of translation invariance: the content of wavelet subbands is unstable under translations of the input signal. Wavelet transforms are also unstable with respect to dilations of the input signal and, in two dimensions, rotations of the input signal. The authors formalize these problems by defining a type of translation invariance called shiftability. In the spatial domain, shiftability corresponds to a lack of aliasing; thus, the conditions under which the property holds are specified by the sampling theorem. Shiftability may also be applied in the context of other domains, particularly orientation and scale. Jointly shiftable transforms that are simultaneously shiftable in more than one domain are explored. Two examples of jointly shiftable transforms are designed and implemented: a 1-D transform that is jointly shiftable in position and scale, and a 2-D transform that is jointly shiftable in position and orientation. The usefulness of these image representations for scale-space analysis, stereo disparity measurement, and image enhancement is demonstrated. >
1,448 citations
TL;DR: If pictures can be characterized by their membership in the smoothness classes considered, then wavelet-based methods are near-optimal within a larger class of stable transform-based, nonlinear methods of image compression.
Abstract: A novel theory is introduced for analyzing image compression methods that are based on compression of wavelet decompositions. This theory precisely relates (a) the rate of decay in the error between the original image and the compressed image as the size of the compressed image representation increases (i.e., as the amount of compression decreases) to (b) the smoothness of the image in certain smoothness classes called Besov spaces. Within this theory, the error incurred by the quantization of wavelet transform coefficients is explained. Several compression algorithms based on piecewise constant approximations are analyzed in some detail. It is shown that, if pictures can be characterized by their membership in the smoothness classes considered, then wavelet-based methods are near-optimal within a larger class of stable transform-based, nonlinear methods of image compression. Based on previous experimental research it is argued that in most instances the error incurred in image compression should be measured in the integral sense instead of the mean-square sense. >
1,038 citations
TL;DR: A detailed second-order analysis is carried out for wavelet coefficients of FBM, revealing a stationary structure at each scale and a power-law behavior of the coefficients' variance from which the fractal dimension of F BM can be estimated.
Abstract: Fractional Brownian motion (FBM) offers a convenient modeling for nonstationary stochastic processes with long-term dependencies and 1/f-type spectral behavior over wide ranges of frequencies. Statistical self-similarity is an essential feature of FBM and makes natural the use of wavelets for both its analysis and its synthesis. A detailed second-order analysis is carried out for wavelet coefficients of FBM. It reveals a stationary structure at each scale and a power-law behavior of the coefficients' variance from which the fractal dimension of FBM can be estimated. Conditions for using orthonormal wavelet decompositions as approximate whitening filters are discussed, consequences of discretization are considered, and some connections between the wavelet point of view and previous approaches based on length measurements (analysis) or dyadic interpolation (synthesis) are briefly pointed out. >
934 citations
Journal Article•
699 citations
CNET1
TL;DR: The goal of this work is to develop guidelines for implementing discrete and continuous wavelet transforms efficiently, and to compare the various algorithms obtained and give an idea of possible gains by providing operation counts.
Abstract: Several algorithms are reviewed for computing various types of wavelet transforms: the Mallat algorithm (1989), the 'a trous' algorithm, and their generalizations by Shensa. The goal of this work is to develop guidelines for implementing discrete and continuous wavelet transforms efficiently, and to compare the various algorithms obtained and give an idea of possible gains by providing operation counts. Most wavelet transform algorithms compute sampled coefficients of the continuous wavelet transform using the filter bank structure of the discrete wavelet transform. Although this general method is already efficient, it is shown that noticeable computational savings can be obtained by applying known fast convolution techniques, such as the FFT (fast Fourier transform), in a suitable manner. The modified algorithms are termed 'fast' because of their ability to reduce the computational complexity per computed coefficient from L to log L (within a small constant factor) for large filter lengths L. For short filters, smaller gains are obtained: 'fast running FIR (finite impulse response) filtering' techniques allow one to achieve typically 30% savings in computations. >
639 citations
TL;DR: The behavior of the continuous wavelet and Gabor coefficients in the asymptotic limit using stationary phase approximations are investigated and it is shown how, under some additional assumptions, these coefficients allow the extraction of some characteristics of the analyzed signal, such as frequency and amplitude modulation laws.
Abstract: The behavior of the continuous wavelet and Gabor coefficients in the asymptotic limit using stationary phase approximations are investigated. In particular, it is shown how, under some additional assumptions, these coefficients allow the extraction of some characteristics of the analyzed signal, such as frequency and amplitude modulation laws. Applications to spectral line estimations and matched filtering are briefly discussed. >
592 citations
TL;DR: An analytically tractable framework is presented to describe mechanical and neural processing in the early stages of the auditory system, and algorithms are developed to assess the integrity of the acoustic spectrum at all processing stages.
Abstract: An analytically tractable framework is presented to describe mechanical and neural processing in the early stages of the auditory system. Algorithms are developed to assess the integrity of the acoustic spectrum at all processing stages. The algorithms employ wavelet representations, multiresolution processing, and the method of convex projections to construct a close replica of the input stimulus. Reconstructions using natural speech sounds demonstrate minimal loss of information along the auditory pathway. Close inspection of the final auditory patterns reveals spectral enhancements and noise suppression that have close perceptual correlates. The functional significance of the various auditory processing stages is discussed in light of the model, together with their potential applications in automatic speech recognition and low bit-rate data compression. >
503 citations
TL;DR: It is shown that the distribution of word frequencies for randomly generated texts is very similar to Zipf's law observed in natural languages such as English.
Abstract: It is shown that the distribution of word frequencies for randomly generated texts is very similar to Zipf's law observed in natural languages such as English. The facts that the frequency of occurrence of a word is almost an inverse power law function of its rank and the exponent of this inverse power law is very close to 1 are largely due to the transformation from the word's length to its rank, which stretches an exponential function to a power law function. >
502 citations
TL;DR: Among the topics discussed are sampling in multiple dimensions, multidimensional perfect reconstruction filter banks, the two-channel case in several dimensions, the synthesis of multiddimensional filter Banks, and the design of compactly supported wavelets.
Abstract: New results on multidimensional filter banks and their connection to multidimensional nonseparable wavelets are presented. Among the topics discussed are sampling in multiple dimensions, multidimensional perfect reconstruction filter banks, the two-channel case in multiple dimensions, the synthesis of multidimensional filter banks, and the design of compactly supported wavelets. >
TL;DR: It is shown that the generalized chirp-like sequences of odd length have the optimum crosscorrelation function under certain conditions.
Abstract: A new general class of polyphase sequences with ideal periodic autocorrelation function is presented. The new class of sequences is based on the application of Zadoff-Chu polyphase sequences of length N=sm/sup 2/, where s and m are any positive integers. It is shown that the generalized chirp-like sequences of odd length have the optimum crosscorrelation function under certain conditions. Finally, recently proposed generalized P4 codes are derived as a special case of the generalized chirp-like sequence. >
[...]
TL;DR: Trellis shaping, a method of selecting a minimum-weight sequence from an equivalence class of possible transmitted sequences by a search through the trellis diagram of a shaping convolutional code C/sub s/.
Abstract: The author discusses trellis shaping, a method of selecting a minimum-weight sequence from an equivalence class of possible transmitted sequences by a search through the trellis diagram of a shaping convolutional code C/sub s/. Shaping gains on the order of 1 dB may be obtained with simple four-state shaping codes and with moderate constellation expansion. The shaping gains obtained with more complicated codes approach the ultimate shaping gain of 1.53 dB. With a feedback-free syndrome-former for C/sub s/, transmitted data can be recovered without catastrophic error propagation. Constellation expansion and peak-to-average energy ratio may be effectively limited by peak constraints. With lattice-theoretic constellations, the shaping operation may be characterized as a decoding of an initial sequence in a channel trellis code by a minimum-distance decoder for a shaping trellis code based on the shaping convolutional code, and the set of possible transmitted sequences is then the set of code sequences in the channel trellis code that lie in the Voronoi region of the trellis shaping code. >
TL;DR: An efficient prediction procedure based on the incremental parsing procedure of the Lempel-Ziv data compression algorithm is shown to achieve asymptotically the finite-state predictability of an infinite sequence.
Abstract: The problem of predicting the next outcome of an individual binary sequence using finite memory is considered. The finite-state predictability of an infinite sequence is defined as the minimum fraction of prediction errors that can be made by any finite-state (FS) predictor. It is proven that this FS predictability can be achieved by universal sequential prediction schemes. An efficient prediction procedure based on the incremental parsing procedure of the Lempel-Ziv data compression algorithm is shown to achieve asymptotically the FS predictability. Some relations between compressibility and predictability are discussed, and the predictability is proposed as an additional measure of the complexity of a sequence. >
TL;DR: The construction of orthonormal bases for L/sup 2/(R/sup n/) is based on the notion of multiresolution analysis and reveals an interesting connection between the theory of compactly supported wavelet bases and the Theory of self-similar tilings.
Abstract: Orthonormal bases for L/sup 2/(R/sup n/) are constructed that have properties that are similar to those enjoyed by the classical Haar basis for L/sup 2/(R). For example, each basis consists of appropriate dilates and translates of a finite collection of 'piecewise constant' functions. The construction is based on the notion of multiresolution analysis and reveals an interesting connection between the theory of compactly supported wavelet bases and the theory of self-similar tilings. >
TL;DR: It is shown that the discrete wavelet coefficients of fractional Brownian motion at different scales are correlated and that their auto- and cross-correlation functions decay hyperbolically fast at a rate much faster than that of the autocorrelation of the fractionalBrownian motion itself.
Abstract: It is shown that the discrete wavelet coefficients of fractional Brownian motion at different scales are correlated and that their auto- and cross-correlation functions decay hyperbolically fast at a rate much faster than that of the autocorrelation of the fractional Brownian motion itself. The rate of decay of the correlation function in the wavelet domain is primarily determined by the number of vanishing moments of the analyzing wavelet. >
TL;DR: An event-detection pitch detector based on the dyadic wavelet transform is described and examples are provided that demonstrate the superior performance of the pitch detector in comparison with classical pitch detectors that use the autocorrelation and the cepstrum methods to estimate the pitch period.
Abstract: An event-detection pitch detector based on the dyadic wavelet transform is described. The proposed pitch detector is suitable for both low-pitched and high-pitched speakers and is robust to noise. Examples are provided that demonstrate the superior performance of the pitch detector in comparison with classical pitch detectors that use the autocorrelation and the cepstrum methods to estimate the pitch period. >
TL;DR: The classical Shannon sampling theorem is extended to the subspaces used in the multiresolution analysis in wavelet theory, and is first shown to have a Riesz basis formed from the reproducing kernels.
Abstract: The classical Shannon sampling theorem is extended to the subspaces used in the multiresolution analysis in wavelet theory. Under weak hypotheses, these subspaces are first shown to have a Riesz basis formed from the reproducing kernels. These in turn are used to construct the sampling sequences. Examples are given. >
TL;DR: It is shown how the wavelet transform directly suggests a modeling paradigm for multiresolution stochastic modeling and related notions of multiscale stationarity in which scale plays the role of a time-like variable.
Abstract: An overview is provided of the several components of a research effort aimed at the development of a theory of multiresolution stochastic modeling and associated techniques for optimal multiscale statistical signal and image processing. A natural framework for developing such a theory is the study of stochastic processes indexed by nodes on lattices or trees in which different depths in the tree or lattice correspond to different spatial scales in representing a signal or image. In particular, it is shown how the wavelet transform directly suggests such a modeling paradigm. This perspective then leads directly to the investigation of several classes of dynamic models and related notions of multiscale stationarity in which scale plays the role of a time-like variable. The investigation of models on homogeneous trees is emphasized. The framework examined here allows for consideration, in a very natural way, of the fusion of data from sensors with differing resolutions. Also, thanks to the fact that wavelet transforms do an excellent job of 'compressing' large classes of covariance kernels, it is seen that these modeling paradigms appear to have promise in a far broader context than one might expect. >
TL;DR: It is proven that these B-spline wavelets converge to Gabor functions (modulated Gaussian) pointwise and in all L/sub p/-norms with 1 > 1.
Abstract: A family of nonorthogonal polynomial spline wavelet transforms is considered. These transforms are fully reversible and can be implemented efficiently. The corresponding wavelet functions have a compact support. It is proven that these B-spline wavelets converge to Gabor functions (modulated Gaussian) pointwise and in all L/sub p/-norms with 1 >
TL;DR: Two families of four-phase sequences are constructed using irreducible polynomials over Z/sub 4/ using maximum nontrivial correlation magnitude C/sub max/ > as guide.
Abstract: Two families of four-phase sequences are constructed using irreducible polynomials over Z/sub 4/. Family A has period L=2/sup r/-1. size L+2. and maximum nontrivial correlation magnitude C/sub max/ >
TL;DR: A deterministic annealing approach is suggested to search for the optimal vector quantizer given a set of training data and the resulting codebook is independent of the codebook used to initialize the iterations.
Abstract: A deterministic annealing approach is suggested to search for the optimal vector quantizer given a set of training data. The problem is reformulated within a probabilistic framework. No prior knowledge is assumed on the source density, and the principle of maximum entropy is used to obtain the association probabilities at a given average distortion. The corresponding Lagrange multiplier is inversely related to the 'temperature' and is used to control the annealing process. In this process, as the temperature is lowered, the system undergoes a sequence of phase transitions when existing clusters split naturally, without use of heuristics. The resulting codebook is independent of the codebook used to initialize the iterations. >
TL;DR: It is found that the frequency hopping schemes are inherently superior and their performance is not dependent on the synchronization of the hopping times for the different users.
Abstract: Results on the modeling of interference in a radio communication network and performance measures for the link as a function of distance are presented. It is assumed that a transmitter-receiver pair in a radio network is affected by a set of interferers, using the same modulation and power, whose positions are modeled as a Poisson field in the plane. Assuming a 1/r/sup gamma / propagation power loss law, the probability distributions for the noise at the receiver are found to be the stable distributions. Results are given for the probability of symbol error and link capacity as a function of the distance between the transmitter and receiver for direct sequence and frequency hopping spread spectrum schemes. It is found that the frequency hopping schemes are inherently superior and their performance is not dependent on the synchronization of the hopping times for the different users. >
TL;DR: Two techniques for finding the discrete orthogonal wavelet of support less than or equal to some given integer that leads to the best approximation to a given finite support signal up to a desired scale are presented.
Abstract: Two techniques for finding the discrete orthogonal wavelet of support less than or equal to some given integer that leads to the best approximation to a given finite support signal up to a desired scale are presented. The techniques are based on optimizing certain cost functions. The first technique consists of minimizing an upper bound that is derived on the L/sub 2/ norm of error in approximating the signal up to the desired scale. It is shown that a solution to the problem of minimizing that bound does exist and it is explained how the constrained minimization over the parameters that define discrete finite support orthogonal wavelets can be turned into an unconstrained one. The second technique is based on maximizing an approximation to the norm of the projection of the signal on the space spanned by translates and dilates of the analyzing discrete orthogonal wavelet up to the desired scale. Both techniques can be implemented much faster than the optimization of the L/sub 2/ norm of either the approximation to the given signal up to the desired scale or that of the error in that approximation. >
TL;DR: A composite upper bound on the redundancy as a function of the quantizer resolution that leads to a tighter bound in the high rate (low distortion) case is presented.
Abstract: Uniform quantization with dither, or lattice quantization with dither in the vector case, followed by a universal lossless source encoder (entropy coder), is a simple procedure for universal coding with distortion of a source that may take continuously many values. The rate of this universal coding scheme is examined, and a general expression is derived for it. An upper bound for the redundancy of this scheme, defined as the difference between its rate and the minimal possible rate, given by the rate distortion function of the source, is derived. This bound holds for all distortion levels. Furthermore, a composite upper bound on the redundancy as a function of the quantizer resolution that leads to a tighter bound in the high rate (low distortion) case is presented. >
TL;DR: It is shown that cyclically permutable codes provide a natural solution to the problem of constructing protocol-sequence sets for the M-active-out-of-T-users collision channel without feedback.
Abstract: A general theorem is proved showing how to obtain a constant-weight binary cyclic code from a p-ary linear cyclic code, where p is a prime, by using a representation of GF(p) as cyclic shifts of a binary p-tuple. Based on this theorem, constructions are given for four classes of binary constant-weight codes. The first two classes are shown to achieve the Johnson upper bound on minimum distance asymptotically for long block lengths. The other two classes are shown similarly to meet asymptotically the low-rate Plotkin upper bound on minimum distance. A simple method is given for selecting virtually the maximum number of cyclically distinct codewords with full cyclic order from Reed-Solomon codes and from Berlekamp-Justesen maximum-distance-separable codes. Two correspondingly optimum classes of constant-weight cyclically permutable codes are constructed. It is shown that cyclically permutable codes provide a natural solution to the problem of constructing protocol-sequence sets for the M-active-out-of-T-users collision channel without feedback. >
TL;DR: Trellis precoding channel capacity can be approached essentially as closely on any strictly bandlimited, high signal-to-noise ratio Gaussian channel as on the ideal channel, using the same coding techniques.
Abstract: On a linear Gaussian channel with intersymbol interference (ISI), trellis precoding is a method that achieves the equalization performance of Tomlinson-Harashima (TH) precoding, the coding gain of any known lattice-type coset code, and a considerable shaping gain. Trellis precoding may be viewed as a generalization of trellis shaping to Gaussian ISI channels, or, alternatively, as a generalization of TH precoding, with coded modulation that achieves shaping gain. With trellis precoding channel capacity can be approached essentially as closely on any strictly bandlimited, high signal-to-noise ratio Gaussian channel as on the ideal channel, using the same coding techniques. For first- and second-order FIR and IIR (finite and infinite impulse response) channels, it is shown that shaping gains close to 1 dB can be obtained with a two-dimensional four-state trellis code. Trellis precoding is quite practical whenever channel information is available at the transmitter. >
TL;DR: A strategy for embedding information into a dy-homogeneous waveform on multiple time-scales is developed, called fractal modulation, which is potentially well-suited for use with noisy channels of simultaneously unknown duration and bandwidth.
Abstract: A potentially important family of self-similar signals based upon a deterministic scale-invariance characterization is introduced. These signals, which are referred to as 'dy-homogeneous' signals because they generalize the well-known homogeneous functions, have highly convenient representations in terms of orthonormal wavelet bases. In particular, wavelet representations can be exploited to construct orthonormal self-similar bases for these signals. The spectral and fractal characteristics of dy-homogeneous signals make them appealing candidates for use in a number of applications. As one potential example, their use in a communications-based context is considered. Specifically, a strategy for embedding information into a dy-homogeneous waveform on multiple time-scales is developed. This multirate modulation strategy, called fractal modulation, is potentially well-suited for use with noisy channels of simultaneously unknown duration and bandwidth. >
TL;DR: A Wiener system, i.e., a system in which a linear dynamic part is followed by a nonlinear and memoryless one, is identified and a nonparametric algorithm recovering the characteristic from input-output observations of the whole system is proposed.
Abstract: A Wiener system, i.e., a system in which a linear dynamic part is followed by a nonlinear and memoryless one, is identified. No parametric restriction is imposed on the functional form of the nonlinear characteristic of the memoryless subsystem, and a nonparametric algorithm recovering the characteristic from input-output observations of the whole system is proposed. Its consistency is shown and the rate of convergence is given. An idea for identification of the impulse response of the linear subsystem is proposed. Results of numerical simulation are also presented. >
TL;DR: The authors derive, implement, and demonstrate a computational approach for the measurement of emergent image frequencies that is cast as an ill-posed extremum problem regularized by the stabilizing term, leading to an iterative constraint propagation algorithm.
Abstract: The authors derive, implement, and demonstrate a computational approach for the measurement of emergent image frequencies. Measuring emergent signal frequencies requires spectral measurements accurate in both frequency and time or space, conflicting requirements that are shown to be balanced by a generalized uncertainty relationship. Such spectral measurements can be obtained from the responses of multiple wavelet-like channel filters that sample the signal spectrum, and that yield a locus of possible solutions for each locally emergent frequency. It is shown analytically that this locus of solutions is maximally localized in both space and frequency if the channel filters used are Gabor wavelets. A constrained solution is obtained by imposing a stabilizing term that develops naturally from the assumptions on the signal. The measurement of frequencies is then cast as an ill-posed extremum problem regularized by the stabilizing term, leading to an iterative constraint propagation algorithm. The technique is demonstrated by application to a variety of 2-D textured images. >