Showing papers in "IEEE Transactions on Signal Processing in 1993"

Matching pursuits with time-frequency dictionaries

Abstract: The authors introduce an algorithm, called matching pursuit, that decomposes any signal into a linear expansion of waveforms that are selected from a redundant dictionary of functions. These waveforms are chosen in order to best match the signal structures. Matching pursuits are general procedures to compute adaptive signal representations. With a dictionary of Gabor functions a matching pursuit defines an adaptive time-frequency transform. They derive a signal energy distribution in the time-frequency plane, which does not include interference terms, unlike Wigner and Cohen class distributions. A matching pursuit isolates the signal structures that are coherent with respect to a given dictionary. An application to pattern extraction from noisy signals is described. They compare a matching pursuit decomposition with a signal expansion over an optimized wavepacket orthonormal basis, selected with the algorithm of Coifman and Wickerhauser see (IEEE Trans. Informat. Theory, vol. 38, Mar. 1992). >

Embedded image coding using zerotrees of wavelet coefficients

Abstract: The embedded zerotree wavelet algorithm (EZW) is a simple, yet remarkably effective, image compression algorithm, having the property that the bits in the bit stream are generated in order of importance, yielding a fully embedded code The embedded code represents a sequence of binary decisions that distinguish an image from the "null" image Using an embedded coding algorithm, an encoder can terminate the encoding at any point thereby allowing a target rate or target distortion metric to be met exactly Also, given a bit stream, the decoder can cease decoding at any point in the bit stream and still produce exactly the same image that would have been encoded at the bit rate corresponding to the truncated bit stream In addition to producing a fully embedded bit stream, the EZW consistently produces compression results that are competitive with virtually all known compression algorithms on standard test images Yet this performance is achieved with a technique that requires absolutely no training, no pre-stored tables or codebooks, and requires no prior knowledge of the image source The EZW algorithm is based on four key concepts: (1) a discrete wavelet transform or hierarchical subband decomposition, (2) prediction of the absence of significant information across scales by exploiting the self-similarity inherent in images, (3) entropy-coded successive-approximation quantization, and (4) universal lossless data compression which is achieved via adaptive arithmetic coding >

B-spline signal processing. I. Theory

Abstract: The use of continuous B-spline representations for signal processing applications such as interpolation, differentiation, filtering, noise reduction, and data compressions is considered. The B-spline coefficients are obtained through a linear transformation, which unlike other commonly used transforms is space invariant and can be implemented efficiently by linear filtering. The same property also applies for the indirect B-spline transform as well as for the evaluation of approximating representations using smoothing or least squares splines. The filters associated with these operations are fully characterized by explicitly evaluating their transfer functions for B-splines of any order. Applications to differentiation, filtering, smoothing, and least-squares approximation are examined. The extension of such operators for higher-dimensional signals such as digital images is considered. >

Energy separation in signal modulations with application to speech analysis

Abstract: An efficient solution to the fundamental problem of estimating the time-varying amplitude envelope and instantaneous frequency of a real-valued signal that has both an AM and FM structure is provided. Nonlinear combinations of instantaneous signal outputs from the energy operator are used to separate its output energy product into its AM and FM components. The theoretical analysis is done first for continuous-time signals. Then several efficient algorithms are developed and compared for estimating the amplitude envelope and instantaneous frequency of discrete-time AM-FM signals. These energy separation algorithms are used to search for modulations in speech resonances, which are modeled using AM-FM signals to account for time-varying amplitude envelopes and instantaneous frequencies. The experimental results provide evidence that bandpass-filtered speech signals around speech formants contain amplitude and frequency modulations within a pitch period. >

B-spline signal processing. II. Efficiency design and applications

Abstract: For pt.I see ibid., vol.41, no.2, p.821-33 (1993). A class of recursive filtering algorithms for the efficient implementation of B-spline interpolation and approximation techniques is described. In terms of simplicity of realization and reduction of computational complexity, these algorithms compare favorably with conventional matrix approaches. A filtering interpretation (low-pass filter followed by an exact polynomial spline interpolator) of smoothing spline and least-squares approximation methods is proposed. These techniques are applied to the design of digital filters for cubic spline signal processing. An efficient implementation of a smoothing spline edge detector is proposed. It is also shown how to construct a cubic spline image pyramid that minimizes the loss of information in passage from one resolution level to the next. In terms of common measures of fidelity, this data structure appears to be superior to the Gaussian/Laplacian pyramid. >

On amplitude and frequency demodulation using energy operators

Abstract: It is shown that the nonlinear energy-tracking signal operator Psi (x)=(dx/dt)/sup 2/-xd/sup 2/x/dt/sup 2/ and its discrete-time counterpart can estimate the AM and FM modulating signals. Specifically, Psi can approximately estimate the amplitude envelope of AM signals and the instantaneous frequency of FM signals. Bounds are derived for the approximation errors, which are negligible under general realistic conditions. These results, coupled with the simplicity of Psi , establish the usefulness of the energy operator for AM and FM signal demodulation. These ideas are then extended to a more general class of signals that are sine waves with a time-varying amplitude and frequency and thus contain both an AM and an FM component; for such signals it is shown that Psi can approximately track the product of their amplitude envelope and their instantaneous frequency. The theoretical analysis is done for both continuous- and discrete-time signals. >

A local update strategy for iterative reconstruction from projections

Abstract: A method for Bayesian reconstruction which relies on updates of single pixel values, rather than the entire image, at each iteration is presented. The technique is similar to Gauss-Seidel (GS) iteration for the solution of differential equations on finite grids. The computational cost per iteration of the GS approach is found to be approximately equal to that of gradient methods. For continuously valued images, GS is found to have significantly better convergence at modes representing high spatial frequencies. In addition, GS is well suited to segmentation when the image is constrained to be discretely valued. It is shown that Bayesian segmentation using GS iteration produces useful estimates at much lower signal-to-noise ratios than required for continuously valued reconstruction. The convergence properties of gradient ascent and GS for reconstruction from integral projections are analyzed, and simulations of both maximum-likelihood and maximum a posteriori cases are included. >

Discrete time techniques for time delay estimation

Abstract: Basic aspects of time delay estimation (TDE) based on sampled signals are investigated. The direct cross-correlation method is analyzed and compared to the average square difference function (ASDF) and the (addition only based) average magnitude difference function (AMDF) estimators, Their relative accuracy is theoretically evaluated, and previous empirical results are explained. It is shown that both the ASDF- and the AMDF-based estimators outperform the direct cross-correlation based estimator for medium-high signal-to-noise ratios. Moreover, the AMDF-based estimator, which avoids any multiplications, significantly reduces the computational complexity of the estimation procedure while offering only a moderate performance loss with respect to the ASDF based estimator. >

On the convergence behavior of the LMS and the normalized LMS algorithms

Abstract: It is shown that the normalized least mean square (NLMS) algorithm is a potentially faster converging algorithm compared to the LMS algorithm where the design of the adaptive filter is based on the usually quite limited knowledge of its input signal statistics. A very simple model for the input signal vectors that greatly simplifies analysis of the convergence behavior of the LMS and NLMS algorithms is proposed. Using this model, answers can be obtained to questions for which no answers are currently available using other (perhaps more realistic) models. Examples are given to illustrate that even quantitatively, the answers obtained can be good approximations. It is emphasized that the convergence of the NLMS algorithm can be speeded up significantly by employing a time-varying step size. The optimal step-size sequence can be specified a priori for the case of a white input signal with arbitrary distribution. >

A stochastic gradient adaptive filter with gradient adaptive step size

Abstract: The step size of this adaptive filter is changed according to a gradient descent algorithm designed to reduce the squared estimation error during each iteration. An approximate analysis of the performance of the adaptive filter when its inputs are zero mean, white, and Gaussian noise and the set of optimal coefficients are time varying according to a random-walk model is presented. The algorithm has very good convergence speed and low steady-state misadjustment. The tracking performance of these algorithms in nonstationary environments is relatively insensitive to the choice of the parameters of the adaptive filter and is very close to the best possible performance of the least mean square (LMS) algorithm for a large range of values of the step size of the adaptation algorithm. Several simulation examples demonstrating the good properties of the adaptive filters as well as verifying the analytical results are also presented. >

Discrete Gabor transform

Abstract: A feasible algorithm for implementing the Gabor expansion, the coefficients of which are computed by the discrete Gabor transform (DGT), is presented. For a given synthesis window and sampling pattern, computing the auxiliary biorthogonal function of the DGT is nothing more than solving a linear system. The DGT presented applies for both finite as well as infinite sequences. By exploiting the nonuniqueness of the auxiliary biorthogonal function at oversampling an orthogonal like DGT is obtained. As the discrete Fourier transform (DFT) is a discrete realization of the continuous-time Fourier transform, similarly, the DGT introduced provides a feasible vehicle to implement the useful Gabor expansion. >

Theory of regular M-band wavelet bases

Abstract: Orthonormal M-band wavelet bases have been constructed and applied by several authors. This paper makes three main contributions. First, it generalizes the minimal length K-regular 2-band wavelets of Daubechies (1988) to the M-band case by deriving explicit formulas for K-regular M-band scaling filters. Several equivalent characterizations of K-regularity are given and their significance explained. Second, two approaches to the construction of the (M-1) wavelet filters and associated wavelet bases are described; one relies on a state-space characterization with a novel technique to obtain the unitary wavelet filters; the other uses a factorization approach. Third, this paper gives a set of necessary and sufficient condition on the M-band scaling filter for it to generate an orthonormal wavelet basis. The conditions are very similar to those obtained by Cohen (1990) and Lawton (1990) for 2-band wavelets. >

A signal-dependent time-frequency representation: optimal kernel design

Abstract: A new time-frequency distribution (TFD) that adapts to each signal and so offers a good performance for a large class of signals is introduced. The design of the signal-dependent TFD is formulated in Cohen's class as an optimization problem and results in a special linear program. Given a signal to be analyzed, the solution to the linear program yields the optimal kernel and, hence, the optimal time-frequency mapping for that signal. A fast algorithm has been developed for solving the linear program, allowing the computation of the signal-dependent TFD with a time complexity on the same order as a fixed-kernel distribution. Besides this computational efficiency, an attractive feature of the optimization-based approach is the ease with which the formulation can be customized to incorporate application-specific knowledge into the design process. >

On-line estimation of hidden Markov model parameters based on the Kullback-Leibler information measure

Abstract: Sequential or online hidden Markov model (HMM) signal processing schemes are derived, and their performance is illustrated by simulation. The online algorithms are sequential expectation maximization (EM) schemes and are derived by using stochastic approximations to maximize the Kullback-Leibler information measure. The schemes can be implemented either as filters or fixed-lag or sawtooth-lag smoothers. They yield estimates of the HMM parameters including transition probabilities, Markov state levels, and noise variance. In contrast to the offline EM algorithm (Baum-Welch scheme), which uses the fixed-interval forward-backward scheme, the online schemes have significantly reduced memory requirements and improved convergence, and they can estimate HMM parameters that vary slowly with time or undergo infrequent jump changes. Similar techniques are used to derive online schemes for extracting finite-state Markov chains imbedded in a mixture of white Gaussian noise (WGN) and deterministic signals of known functional form with unknown parameters. >

Tilings of the time-frequency plane: construction of arbitrary orthogonal bases and fast tiling algorithms

Abstract: The authors consider expansions which give arbitrary orthonormal tilings of the time-frequency plane. These differ from the short-time Fourier transform, wavelet transform, and wavelet packets tilings in that they change over time. They show how this can be achieved using time-varying orthogonal tree structures, which preserve orthogonality, even across transitions. The method is based on the construction of boundary and transition filters; these allow us to construct essentially arbitrary tilings. Time-varying modulated lapped transforms are a special case, where both boundary and overlapping solutions are possible with filters obtained by modulation. They present a double-tree algorithm which for a given signal decides on the best binary segmentation in both time and frequency. That is, it is a joint optimization of time and frequency splitting. The algorithm is optimal for additive cost functions (e.g., rate-distortion), and results in time-varying best bases, the main application of which is for compression of nonstationary signals. Experiments on test signals are presented. >

Linear phase paraunitary filter banks: theory, factorizations and designs

Abstract: M channel maximally decimated filter banks have been used in the past to decompose signals into subbands. The theory of perfect-reconstruction filter banks has also been studied extensively. Nonparaunitary systems with linear phase filters have also been designed. The authors study paraunitary systems in which each individual filter in the analysis synthesis banks has linear phase. Specific instances of this problem have been addressed by other authors, and linear phase paraunitary systems have been shown to exist. This property is often desirable for several applications, particularly in image processing. They begin by answering several theoretical questions pertaining to linear phase paraunitary systems. Next, they develop a minimal factorization for a large class of such systems. This factorization will be proved to be complete for even M. Further, they structurally impose the additional condition that the filters satisfy pairwise mirror-image symmetry in the frequency domain. This significantly reduces the number of parameters to be optimized in the design process. They then demonstrate the use of these filter banks in the generation of M-band orthonormal wavelets. Several design examples are also given to validate the theory. >

Efficient computation of the DFT with only a subset of input or output points

Abstract: Ways of efficiently computing the discrete Fourier transform (DFT) when the number of input and output data points differ are discussed. The two problems of determining whether the length of the input sequence or the length of the output sequence is reduced can be found to be duals of each other, and the same methods can, to a large extent, be used to solve both. The algorithms utilize the redundancy in the input or output to reduce the number of operations below those of the fast Fourier transform (FFT) algorithms. The usual pruning method is discussed, and an efficient algorithm, called transform decomposition, is introduced. It is based on a mixture of a standard FFT algorithm and the Horner polynomial evaluation scheme equivalent to the one in Goertzel's algorithms. It requires fewer operations and is more flexible than pruning. The algorithm works for power-of-two and prime-factor algorithms, as well as for real-input data. >

Wavelets and recursive filter banks

Abstract: It is shown that infinite impulse response (IIR) filters lead to more general wavelets of infinite support than finite impulse response (FIR) filters. A complete constructive method that yields all orthogonal two channel filter banks, where the filters have rational transfer functions, is given, and it is shown how these can be used to generate orthonormal wavelet bases. A family of orthonormal wavelets that have a maximum number of disappearing moments is shown to be generated by the halfband Butterworth filters. When there is an odd number of zeros at pi it is shown that closed forms for the filters are available without need for factorization. A still larger class of orthonormal wavelet bases having the same moment properties and containing the Daubechies and Butterworth filters as the limiting cases is presented. It is shown that it is possible to have both linear phase and orthogonality in the infinite impulse response case, and a constructive method is given. It is also shown how compactly supported bases may be orthogonalized, and bases for the spline function spaces are constructed. >

Measuring the Fractal Dimension of Signals: Morphological Covers and Iterative Optimization

Abstract: Fractals can model many classes of time-series data. The fractal dimension is an important characteristic of fractals that contains information about their geometrical structure at multiple scales. The covering methods are a class of efficient approaches to measure the fractal dimension of an arbitrary fractal signal by creating multiscale covers around the signal’s graph. In this paper we develop a general method that uses multiscale morphological operations with varying structuring elements to unify and extend the theory and digital implementations of covering methods. It is theoretically established that, for the fractal dimension computation, covering one-dimensional signals with planar sets is equivalent to morphologically transforming the signal by one-dimensional functions, which reduces the computational complexity from quadratic in the signal’s length to linear. Then a morphological covering algorithm is developed and applied to discrete-time signals synthesized from Weierstrass functions, fractal interpolation functions, and fractional Brownian motion. Further, for deterministic parametric fractals depending on a single parameter related to their dimension, we develop an optimization method that starts from an initial estimate and iteratively con- verges to the true fractal dimension by searching in the parameter space and minimizing a distance between the original signal and all such signals from the same class. Experimental results are also provided to demonstrate the good performance of the developed methods.

Low bit rate transparent audio compression using adapted wavelets

Abstract: Describes a novel wavelet based audio synthesis and coding method. The method uses optimal adaptive wavelet selection and wavelet coefficients quantization procedures together with a dynamic dictionary approach. The adaptive wavelet transform selection and transform coefficient bit allocation procedures are designed to take advantage of the masking effect in human hearing. They minimize the number of bits required to represent each frame of audio material at a fixed distortion level. The dynamic dictionary greatly reduces statistical redundancies in the audio source. Experiments indicate that the proposed adaptive wavelet selection procedure by itself can achieve almost transparent coding of monophonic compact disk (CD) quality signals (sampled at 44.1 kHz) at bit rates of 64-70 kilobits per second (kb/s). The combined adaptive wavelet selection and dynamic dictionary coding procedures achieve almost transparent coding of monophonic CD quality signals at bit rates of 48-66 kb/s. >

The scale representation

Abstract: The authors considers "scale" a physical attribute of a signal and develop its properties. He presents an operator which represents scale and study its characteristics and representation. This allows one to define the scale transform and the energy scale density spectrum which is an indication of the intensity of scale values in a signal. He obtains explicit expressions for the mean scale, scale bandwidth, instantaneous scale, and scale group delay. Furthermore, he derives expressions for mean time, mean frequency, duration, frequency bandwidth in terms of the scale variable. The short-time transform is defined and used to obtain the conditional value of scale for a given time. He shows that as the windows narrows one obtains instantaneous scale. Convolution and correlation theorems for scale are derived. A formulation is devised for studying linear scale-invariant systems. He derives joint representations of time-scale and frequency-scale, General classes for each are presented using the same methodology as for the time-frequency case. As special cases the joint distributions of Marinovich-Altes (1978, 1986) and Bertrand-Bertrand (1984) are recovered. Also, joint representations of the three quantities, time-frequency-scale are devised. A general expression for the local scale autocorrelation function is given. Uncertainty principles for scale and time and scale and frequency are derived. >

Adaptive Bayesian equalizer with decision feedback

Abstract: A Bayesian solution is derived for digital communication channel equalization with decision feedback. This is an extension of the maximum a posteriori probability symbol-decision equalizer to include decision feedback. A novel scheme utilizing decision feedback that not only improves equalization performance but also reduces computational complexity greatly is proposed. It is shown that the Bayesian equalizer has a structure equivalent to that of the radial basis function network, the latter being a one-hidden-layer artificial neural network widely used in pattern classification and many other areas of signal processing. Two adaptive approaches are developed to realize the Bayesian solution. The maximum-likelihood Viterbi algorithm and the conventional decision feedback equalizer are used as two benchmarks to asses the performance of the Bayesian decision feedback equalizer. >

A unified texture model based on a 2-D Wold-like decomposition

Abstract: A unified texture model that is applicable to a wide variety of texture types found in natural images is presented. This model leads to the derivation of texture analysis and synthesis algorithms designed to estimate the texture parameters and to reconstruct the original texture field from these parameters. The texture field is assumed to be a realization of a regular homogeneous random field, which is characterized in general by a mixed spectral distribution. The texture field is orthogonally decomposed into a purely indeterministic component and a deterministic component. The deterministic component is further orthogonally decomposed into a harmonic component, and a generalized-evanescent component. Both analytical and experimental results show that the deterministic components should be parameterized separately from the purely indeterministic component. The model is very efficient in terms of the number of parameters required to faithfully represent textures. Reconstructed textures are practically indistinguishable from the originals. >

Waveform-preserving blind estimation of multiple independent sources

Abstract: The problem of blind estimation of source signals is to estimate the source signals without knowing the characteristics of the transmission channel. It is shown that the minimum-variance unbiased estimates can be obtained if and only if the transmission channel can be identified blindly. It is shown that the channel can be blindly identified if and only if there is not more than one Gaussian source. This condition suggests that waveform-preserving blind estimation can be achieved over a wide range of signal processing applications, including those cases in which the source signals have identical nonGaussian distributions. The constructive proof of the necessary and sufficient condition serves as a foundation for the development of waveform-preserving blind signal estimation algorithms. Examples are presented to demonstrate the applications of the theoretical results. >

AM-FM energy detection and separation in noise using multiband energy operators

Abstract: This paper develops a multiband or wavelet approach for capturing the AM-FM components of modulated signals immersed in noise. The technique utilizes the recently-popularized nonlinear energy operator Psi (s)=(s)/sup 2/-ss to isolate the AM-FM energy, and an energy separation algorithm (ESA) to extract the instantaneous amplitudes and frequencies. It is demonstrated that the performance of the energy operator/ESA approach is vastly improved if the signal is first filtered through a bank of bandpass filters, and at each instant analyzed (via Psi and the ESA) using the dominant local channel response. Moreover, it is found that uniform (worst-case) performance across the frequency spectrum is attained by using a constant-Q, or multiscale wavelet-like filter bank. The elementary stochastic properties of Psi and of the ESA are developed first. The performance of Psi and the ESA when applied to bandpass filtered versions of an AM-FM signal-plus-noise combination is then analyzed. The predicted performance is greatly improved by filtering, if the local signal frequencies occur in-band. These observations motivate the multiband energy operator and ESA approach, ensuring the in-band analysis of local AM-PM energy. In particular, the multi-bands must have the constant-Q or wavelet scaling property to ensure uniform performance across bands. The theoretical predictions and the simulation results indicate that improved practical strategies are feasible for tracking and identifying AM-FM components in signals possessing pattern coherencies manifested as local concentrations of frequencies. >

Time-varying system identification and model validation using wavelets

Abstract: Parametric identification of time-varying (TV) systems is possible if each TV coefficient can be expanded onto a finite set of basis sequences. The problem then becomes time invariant with respect to the parameters of the expansion. The authors address the question of selecting this set of basis sequences. They advocate the use of a wavelet basis because of its flexibility in capturing the signal's characteristics at different scales, and discuss how to choose the optimal wavelet basis for a given system trajectory. They also develop statistical tests to keep only the basis sequences that significantly contribute to the description of the system's time-variation. By formulating the problem as a regressor selection problem, they apply an P-test and an AIC based approach for multiresolution analysis of TV systems. The resulting algorithm can estimate TV AR or ARMAX models and determine their orders. They apply this algorithm to both synthetic and real speech data and compare it with the Kalman filtering TV parameter estimator. >

Perfect reconstruction filter banks with rational sampling factors

Abstract: An open problem, namely, how to construct perfect reconstruction filter banks with rational sampling factors, is solved. Such filter banks have N branches, each one having a sampling factor of p i/qi, and their sum equals one. In this way, the well-known theory of filter banks with uniform band splitting is extended to allow for nonuniform divisions of the spectrum. This can be very useful in the analysis of speech and music. The theory relies on two transforms. The first transform leads to uniform filter banks having polyphase components as individual filters. The other results in a uniform filter bank containing shifted versions of same filters. This, in turn, introduces dependencies in design, and is left for future work. As an illustration, several design examples for the (2/3, 1/3) case are given. Filter banks are then classified according to the possible ways in which they can be built. It is shown that some cases cannot be solved even with ideal filters (with real coefficients)

Multiscale representations of Markov random fields

Abstract: Recently, a framework for multiscale stochastic modeling was introduced based on coarse-to-fine scale-recursive dynamics defined on trees. This model class has some attractive characteristics which lead to extremely efficient, statistically optimal signal and image processing algorithms. The authors show that this model class is also quite rich. In particular, they describe how 1-D Markov processes and 2-D Markov random fields (MRFs) can be represented within this framework. The recursive structure of 1-D Markov processes makes them simple to analyze, and generally leads to computationally efficient algorithms for statistical inference. On the other hand, 2-D MRFs are well known to be very difficult to analyze due to their noncausal structure, and thus their use typically leads to computationally intensive algorithms for smoothing and parameter identification. In contrast, their multiscale representations are based on scale-recursive models and thus lead naturally to scale-recursive algorithms, which can be substantially more efficient computationally than those associated with MRF models. In 1-D, the multiscale representation is a generalization of the midpoint deflection construction of Brownian motion. The representation of 2-D MRFs is based on a further generalization to a "midline" deflection construction. The exact representations of 2-D MRFs are used to motivate a class of multiscale approximate MRF models based on one-dimensional wavelet transforms. They demonstrate the use of these latter models in the context of texture representation and, in particular, they show how they can be used as approximations for or alternatives to well-known MRF texture models. >

LUM filters: a class of rank-order-based filters for smoothing and sharpening

Abstract: A new class of rank-order-based filters, called lower-upper-middle (LUM) filters, is introduced. The output of these filters is determined by comparing a lower- and an upper-order statistic to the middle sample in the filter window. These filters can be designed for smoothing and sharpening, or outlier rejection. The level of smoothing done by the filter can range from no smoothing to that of the median filter. This flexibility allows the LUM filter to be designed to best balance the tradeoffs between noise smoothing and signal detail preservation. LUM filters for enhancing edge gradients can be designed to be insensitive to low levels of additive noise and to remove impulsive noise. Furthermore, LUM filters do not cause overshoot or undershoot. Some statistical and deterministic properties of the LUM filters are developed, and a number of experimental results are presented to illustrate the performance. These experiments include applications to 1D signals and to images. >