Showing papers on "Recursive least squares filter published in 1995"

Projection approximation subspace tracking

Abstract: Subspace estimation plays an important role in a variety of modern signal processing applications. We present a new approach for tracking the signal subspace recursively. It is based on a novel interpretation of the signal subspace as the solution of a projection like unconstrained minimization problem. We show that recursive least squares techniques can be applied to solve this problem by making an appropriate projection approximation. The resulting algorithms have a computational complexity of O(nr) where n is the input vector dimension and r is the number of desired eigencomponents. Simulation results demonstrate that the tracking capability of these algorithms is similar to and in some cases more robust than the computationally expensive batch eigenvalue decomposition. Relations of the new algorithms to other subspace tracking methods and numerical issues are also discussed. >

Adaptive Processing: The Least Mean Squares Approach with Applications in Transmission

Abstract: Transmission Systems Theory of Adaptive Transversal (FIR) Filters Implementation Considerations Tracking the Time-Variations Adaptive Recursive (IIR) Filters The Case of Independent Input Vectors

Fast orthogonal least squares algorithm for efficient subset model selection

Abstract: An efficient implementation of the orthogonal least squares algorithm for subset model selection is derived. Computational complexity of the algorithm is examined and the result shows that this new fast orthogonal least squares algorithm significantly reduces computational requirements. >

Isolation of resonance in acoustic backscatter from elastic targets using adaptive estimation schemes

Abstract: The problem of underwater target detection and classification from acoustic backscatter is the central focus of this paper. It has been shown that at certain frequencies the acoustic backscatter from elastic targets exhibits certain resonance behavior which closely relates to the physical properties of the target such as dimension, thickness, and composition. Several techniques in both the time domain and frequency domain have been developed to characterize the resonance phenomena in acoustic backscatter from spherical or cylindrical thin shells. The purpose of this paper is to develop an automated approach for identifying the presence of resonance in the acoustic backscatter from an unknown target by isolating the resonance part from the specular contribution. An adaptive transversal filter structure is used to estimate the specular part of the backscatter and consequently the error signal would provide an estimate of the resonance part. An important aspect of this scheme Lies in the fact that it does not require an underlying model for the elastic return. The adaptation rule is based upon fast Recursive Least Squares (RLS) learning. The approach taken in this paper is general in the sense that it can be applied to targets of unknown geometry and thickness and, further, does not require any a priori information about the target and/or the environment. Test results on acoustic data are presented which indicate the effectiveness of the proposed approach.

Performance analysis of general tracking algorithms

Abstract: A general family of tracking algorithms for linear regression models is studied. It includes the familiar least mean square gradient approach, recursive least squares, and Kalman filter based estimators. The exact expressions for the quality of the obtained estimates are complicated. Approximate, and easy-to-use, expressions for the covariance matrix of the parameter tracking error are developed. These are applicable over the whole time interval, including the transient, and the approximation error can be explicitly calculated. >

Principal component extraction using recursive least squares learning

Abstract: A new neural network-based approach is introduced for recursive computation of the principal components of a stationary vector stochastic process. The neurons of a single-layer network are sequentially trained using a recursive least squares squares (RLS) type algorithm to extract the principal components of the input process. The optimality criterion is based on retaining the maximum information contained in the input sequence so as to be able to reconstruct the network inputs from the corresponding outputs with minimum mean squared error. The proof of the convergence of the weight vectors to the principal eigenvectors is also established. A simulation example is given to show the accuracy and speed advantages of this algorithm in comparison with the existing methods. Finally, the application of this learning algorithm to image data reduction and filtering of images degraded by additive and/or multiplicative noise is considered. >

Adaptive filtering algorithms for stereophonic acoustic echo cancellation

Abstract: It is likely that stereophonic (and more generally, multichannel) sound pick-up, transmission and diffusion will be implemented in future teleconference systems to provide the users with enhanced quality. Therefore, adequate solutions must be found to solve the problem of stereophonic acoustic echo which will occur in such systems. We explain in this paper the difference between the mono and two-channel systems and the behavior of the two-channel classical adaptive algorithms in comparison with the same algorithms in the mono-channel case. Also, we outline a new NLMS-like algorithm derived from the two-channel RLS algorithm as a first member of a family of improved two-channel adaptive filters.

Exact expectation analysis of the LMS adaptive filter

Abstract: In almost all analyses of the least mean square (LMS) adaptive filter, it is assumed that the filter coefficients are statistically independent of the input data currently in filter memory, an assumption that is incorrect for shift-input data. We present a method for deriving a set of linear update equations that can be used to predict the exact statistical behavior of a finite-impulse-response (FIR) LMS adaptive filter operating upon finite-time correlated input data. Using our method, we can derive exact bounds upon the LMS step size to guarantee mean and mean-square convergence. Our equation-deriving procedure is recursive and algorithmic, and we describe a program written in the MAPLE symbolic-manipulation software package that automates the derivation for arbitrarily-long adaptive filters operating on input data with stationary statistics. Using our analysis, we present a search algorithm that determines the exact step size mean-square stability bound for a given filter length and input correlation statistics. Extensive computer simulations indicate that the exact analysis is more accurate than previous analyses in predicting adaptation behavior. Our results also indicate that the exact step size bound is much more stringent than the bound predicted by the independence assumption analysis for correlated input data.

Disposable audio processor for use with implanted hearing devices

Abstract: The echo canceller is designed to be placed between a hands-free acoustical interface and a communications network. It comprises a plurality of processing paths connected in parallel and each allocated to one of a plurality of adjacent sub-bands taken from the spectrum band of the output signal. Each path comprises an analysis filter receiving the echo-containing signal for transmission after correction, a second analysis filter receiving the incoming signal coming from the network, and feeding an adaptive filter that supplies an estimated echo in the respective sub-band to the subtractive input of the subtracter and a synthesis filter. The adaptive filters in at least some of the sub-bands implement a QR decomposition RLS algorithm on the incoming signal, using the fast version thereof, with or without recursive order.

Nonlinear time series modelling and prediction using Gaussian RBF networks with enhanced clustering and RLS learning

Abstract: An improved clustering and recursive least squares (RLS) learning algorithm for Gaussian radial basis function (RBF) networks is described for modelling and predicting nonlinear time series Significant performance gain can be achieved with a much smaller network compared with the usual clustering and RLS method

Optimal tracking of time-varying channels: a frequency domain approach for known and new algorithms

Abstract: In this paper, we developed a systematic frequency domain approach to analyze adaptive tracking algorithms for fast time-varying channels. The analysis is performed with the help of two new concepts, a tracking filter and a tracking error filter, which are used to calculate the mean square identification error (MSIE). First, we analyze existing algorithms, the least mean squares (LMS) algorithm, the exponential windowed recursive least squares (EW-RLS) algorithm and the rectangular windowed recursive least squares (RW-RLS) algorithm. The equivalence of the three algorithms is demonstrated by employing the frequency domain method. A unified expression for the MSIE of all three algorithms is derived. Secondly, we use the frequency domain analysis method to develop an optimal windowed recursive least squares (OW-RLS) algorithm. We derive the expression for the MSIE of an arbitrary windowed RLS algorithm and optimize the window shape to minimize the MSIE. Compared with an exponential window having an optimized forgetting factor, an optimal window results in a significant improvement in the h MSIE. Thirdly, we propose two types of robust windows, the average robust window and the minimax robust window. The RLS algorithms designed with these windows have near-optimal performance, but do not require detailed statistics of the channel. >

Fast projection algorithm and its step size control

Abstract: Of the many adaptive filtering algorithms, the normalized LMS (NLMS) algorithm is generally used in practice because of its simplicity. The computational complexity of the NLMS algorithm is low, however, convergence is very slow and tracking is poor for a colored input signal such as speech. The projection algorithm was proposed as a generalization of the NLMS algorithm. This paper provides a fast projection algorithm and a step size control to obtain the same steady-state excess mean squared error (MSE) for various projection orders. Computer simulations for colored noise and speech input signal confirm the effectiveness of the projection algorithm and the step size control.

Identification of linear parametrically varying systems

Abstract: Addresses the problem of identification of linear parametrically varying systems with one measurable varying parameter. Under the assumption of full state measurements, the authors show that the problem can be reduced to a set of n (dimension of state space) recursive least squares problems. Further, the authors show that these recursions do estimate the parameters of the original model accurately under certain assumptions on the parameter variations. In the case of noisy state measurements the authors set up the problem as a set of n instrument variable recursions. Once again the authors demonstrate strong consistency of estimates. Simulations are presented to illustrate the results.

Improvement of Observer/Kalman Filter Identification (OKID) by Residual Whitening

Abstract: This paper presents a time-domain method to identify a state space model of a linear system and its corresponding observer/Kalman filter from a given set of general input-output data. The identified filter has the properties that its residual is minimized in the least squares sense, orthogonal to the time-shifted versions of itself, and to the given input-output data sequence. The connection between the state space model and a particular auto-regressive moving average description of a linear system is made in terms of the Kalman filter and a deadbeat gain matrix. The procedure first identifies the Markov parameters of an observer system, from which a state space model of the system and the filter gain are computed. The developed procedure is shown to improve results obtained by an existing observer/Kalman filter identification method, which is based on an auto-regressive model without the moving average terms. Numerical and experimental results are presented to illustrate the proposed method.

A set of algorithms linking NLMS and block RLS algorithms

Abstract: This paper describes a set of block processing algorithms which contains as extremal cases the normalized least mean squares (NLMS) and the block recursive least squares (BRLS) algorithms. All these algorithms use small block lengths, thus allowing easy implementation and small input-output delay. It is shown that these algorithms require a lower number of arithmetic operations than the classical least mean squares (LMS) algorithm, while converging much faster. A precise evaluation of the arithmetic complexity is provided, and the adaptive behavior of the algorithm is analyzed. Simulations illustrate that the tracking characteristics of the new algorithm are also improved compared to those of the NLMS algorithm. The conclusions of the theoretical analysis are checked by simulations, illustrating that, even in the case where noise is added to the reference signal, the proposed algorithm allows altogether a faster convergence and a lower residual error than the NLMS algorithm. Finally, a sample-by-sample version of this algorithm is outlined, which is the link between the NLMS and recursive least squares (RLS) algorithms. >

Exponential stability of general tracking algorithms

Abstract: Tracking and adaptation algorithms are, from a formal point of view, nonlinear systems which depend on stochastic variables in a fairly complicated way. The analysis of such algorithms is thus quite complicated. A first step is to establish the exponential stability of these systems. This is of interest in its own right and a prerequisite for the practical use of the algorithm. It is also a necessary starting point to analyze the performance in terms of tracking and adaptation because that is how close the estimated parameters are to the time-varying true ones. In this paper we establish some general conditions for the exponential stability of a wide and common class of tracking algorithms. This includes least mean squares, recursive least squares, and Kalman filter based adaptation algorithms. We show how stability of an averaged (linear and deterministic) equation and stability of the actual algorithm are linked to each other under weak conditions on the involved stochastic processes. We also give explicit conditions for exponential stability of the most common algorithms. The tracking performance of the algorithms is studied in a companion paper. >

Fitting of circles and ellipses least squares solution

Abstract: Publisher Summary Fitting ellipses to given points in the plane is a problem that arises in many application areas, such as computer graphics, coordinate metrology, petroleum engineering, statistics. In the past, algorithms have been given which fit circles and ellipses in some least squares sense without minimizing the geometric distance to the given points. This chapter first presents algorithms that compute the ellipse, for which the sum of the squares of the distances to the given points is minimal. Note that the solution of this non-linear least squares problem is generally expensive. Further, the chapter gives an overview of linear least squares solutions which minimize the distance in some algebraic sense. Given only a few points, it can be seen that the geometric solution often differs significantly from algebraic solutions. The chapter also refines the algebraic method by iteratively solving weighted linear least squares. A criterion based on the singular value decomposition is shown to be essential for the quality of the approximation to the exact geometric solution.

Adaptive equalization for co-channel interference in a multipath fading environment

Abstract: The paper concerns the feasibility and achievable performance of adaptive filtering in an interference-limited multipath fading environment as encountered in indoor wireless communications. In a typical cellular radio application, the performance-limiting impairment is interference due to synchronous data streams from other co-channel and adjacent channel users (CCI and ACI). The receiver under consideration employs an adaptive fractionally spaced decision feedback equalizer (DFE) Which exploits the correlation of the cyclostationary interference to achieve superior performance relative to the worst case when the interference is stationary noise. The paper presents ideal calculations which confirm that significant DFE performance gains are potentially achievable by explicitly accounting for the cyclostationary CCI. Two adaptive DFE strategies are considered. One approach is to adapt the DFE directly using iterative algorithms such as least mean square (LMS) or recursive least squares (RLS). Another approach is to compute the minimum mean square error DFE using an RLS channel impulse response (CIR) estimate and a sample estimate of the CCI autocorrelation obtained from the CIR estimation error during training. The best approach for adaptive equalization, in terms of adaptation speed and system performance, is to employ an RLS DFE which does not explicitly estimate the CIR or the CCI autocorrelation. >

Analysis and implementation of the max-NLMS adaptive filter

Abstract: We provide an efficient implementation and a statistical analysis of the max-NLMS adaptive filter. This adaptive filter only adjusts the coefficient associated with the data element that has the maximum absolute value in the filter memory at each iteration. Our method for determining this maximum absolute data value requires many fewer compares and storage locations on average as compared to other techniques. We then provide statistical and stability analyses of the max-NLMS algorithm for several input data models. Theory and simulations show that the max-NLMS adaptive filter is statistically more efficient than other adaptive filters with similar computational complexity for some input signals; however, its stability behavior is very sensitive to skew in the input data probability distribution.

Experimental results with increased bandwidth analysis filters in oversampled, subband acoustic echo cancelers

Abstract: The motivation for adaptive filtering in subbands stems from two well-known problems in least-mean square full-band adaptive filtering. First, the convergence and tracking can be very slow if the input correlation matrix is ill conditioned, as in the case with speech input. Second, very high order adaptive filters are computationally expensive. One problem with adaptive filtering in subbands is the slow, asymptotic convergence associated with oversampled systems. Increasing the bandwidth of analysis filters relative to the synthesis filters is proposed to reduce the slow asymptotic convergence. The authors present experimental results illustrating the benefits of this modification. >

On the estimation of rigid body rotation from noisy data

Abstract: We derive an exact solution to the problem of estimating the rotation of a rigid body from noisy 3D image data. Our approach is based on total least squares (TLS), but unlike previous work involving TLS, we include the constraint that the transformation matrix should be orthonormal. It turns out that the solution to the estimation problem has the same form as if the data are not noisy, and thus the solution to the standard Procrustes problem can be applied.

Conversion of the delayed LMS algorithm into the LMS algorithm

Abstract: For some applications of adaptive finite impulse response (FIR) filtering, the adaptation algorithm can be implemented only with a delay in the coefficient update. It is well known that this has an adverse effect on the convergence behavior of the algorithm. It is shown in which way the delayed LMS (DLMS) algorithm can be transformed into the standard LMS algorithm at only slightly increased computational expense.

On-line spectral estimation of nonstationary time series based on AR model parameter estimation and order selection with a forgetting factor

Abstract: A new method for on-line spectral estimation of nonstationary time series via autoregressive (AR) model construction is proposed. The method consists of on-line parameter estimation based on the recursive least squares ladder estimation algorithm with a forgetting factor and on-line order determination based on AIC with some modifications. The effectiveness of the proposed method is demonstrated by computer simulation study and applying to the actual data of electroencephalogram (EEG). >

Robust Tool Wear Estimation With Radial Basis Function Neural Networks

Abstract: In this paper, a unified method for constructing dynamic models for tool wear from prior experiments is proposed. The model approximates flank and crater wear propagation and their effects on cutting force using radial basis function neural networks. Instead of assuming a structure for the wear model and identifying its parameters, only an approximate model is obtained in terms of radial basis functions. The appearance of parameters in a linear fashion motivates a recursive least squares training algorithm. This results in a model which is available as a monitoring tool for on-line application. Using the identified model, a state estimator is designed based on the upperbound covariance matrix. This filter includes the errors in modeling the wear process, and hence reduces filter divergence. Simulations using the neural network for different cutting conditions show good results. Addition of pseudo noise during state estimation is used to reflect inherent process variabilities. Estimation of wear under these conditions is also shown to be accurate. Simulations performed using experimental data similarly show good results. Finally, experimental implementation of the wear monitoring system reveals a reasonable ability of the proposed monitoring scheme to track flank wear.

Nonlinear distance and cruise control for passenger cars

Abstract: A methodology for designing an automated vehicle longitudinal velocity and distance controller is presented and applied to an automobile. nonlinearities. The controller consists of a three layer structure. In the first layer a linearization of the nonlinearities is done in order to achieve a simplified structure for controller design purposes. With respect to changes in the typical vehicle parameters-mass and aerodynamical drag-an adaptive controller structure is used and these parameters are estimated by a recursive least squares algorithm. Based on classical controlling techniques a linear acceleration controller is developed in the middle layer. A fuzzy controller is applied in the upper layer. This controller based on the linguistic description of comfort demands. Additionally a neural network is used in this layer instead of the fuzzy system. The complete structure is used in two different series fabricated vehicles and experimental results are shown for highway traffic and also for stop-go traffic on highway congestions. Additionally a neural network is trained by measurement data as a second approach to describe the comfort demands.