Showing papers on "Recursive least squares filter published in 1988"

Dynamic monocular machine vision

Abstract: A new approach to real-time machine vision in dynamic scenes is presented based on special hardware and methods for feature extraction and information processing. Using integral spatio-temporal models, it by-passes the nonunique inversion of the perspective projection by applying recursive least squares filtering. By prediction error feedback methods similar to those used in modern control theory, all spatial state variables including the velocity components are estimated. Only the last image of the sequence needs to be evaluated, thereby alleviating the real-time image sequence processing task.

Modified least squares algorithm incorporating exponential resetting and forgetting

Abstract: In this paper we present the general analysis of a class of least squares algorithms with emphasis on their dynamic performance particularly in the presence of poor excitation. The analysis is carried out in a deterministic framework and stresses geometrical interpretations. The core of this paper is the proposal and analysis of a new algorithm which incorporates exponential forgetting and resetting to an unprejudiced treatment of data when excitation is poor. The algorithm is particularly suitable for tracking time-varying parameters and is similar in computational complexity to the standard recursive least squares algorithm. The superior performance of the algorithm is verified via simulation studies.

A comparative study of several wind estimation algorithms for spaceborne scatterometers

Abstract: The authors compare the performance of seven wind-estimation algorithms, including the weighted least squares in the log domain, maximum-likelihood (ML), least squares, weighted least squares, adjustable weighted least squares, L1 norm, and least wind speed squares algorithms, for wind retrieval. For each algorithm, they present performance simulation results for the NASA scatterometer system planned to be launched in the 1990s. A relative performance merit based on the root-mean-square value of wind vector error is devised. It is found that performances of all algorithms are quite comparable. However, the results do indicate that the ML algorithm performs best for the 50-km wind resolution cell case and the L1 norm algorithm performs best for the 25-km wind resolution cell case. >

Tracking improvements in fast RLS algorithms using a variable forgetting factor

Abstract: The concept of a variable forgetting factor (VFF) is incorporated into fast recursive least-squares (FRLS) algorithms. Compromises in the data matrix that are needed to do this are examined. Both prewindowed and growing memory covariance algorithms are presented in transversal and lattice structures. Forgetting-factor adaptation schemes, which improve tracking performance over conventional FRLS algorithms, are suggested. Finally, the bias introduced by the use of the VFF is analyzed. >

A new method to stabilize fast RLS algorithms based on a first-order of the propagation of numerical errors

Abstract: An effective method to stabilize fast RLS algorithms is proposed. It is based on the analysis of the propagation of the numerical errors according to a first-order linear model. Two variables are shown to be responsible for the numerical instability. The proposed method modifies the numerical properties of these variables, while preserving the theoretical form of the algorithms. This method is applied to the FTF (fast transversal filter) and fast Kalman algorithms. Experimental results in floating- and fixed-point arithmetic show the efficiency of the method. >

On least squares methods for direction of arrival estimation in the presence of unknown noise fields

Abstract: The direction-of-arrival estimation of signal wavefronts in the presence of unknown noise fields is investigated. Generalizations of known criteria for both conditional and nonconditional maximum-likelihood estimates are developed. Numerical calculations show that the usual Gauss-Newton iteration for conditional maximum-likelihood estimates cannot give good results. Therefore, a related, relatively simple two-step least-squares estimate is constructed. Results of numerical experiments are presented and indicate that the two-step estimate has approximately the same power as the least-squares estimate using the exact noise correlation structure. >

Adaptive Filters and Equalisers

Abstract: 1 Introduction.- 1.1 Adaptive Signal Processing.- 1.2 The Adaptive Filter.- 1.3 Modes of Operation.- 1.4 Application of Adaptive Filters.- 1.5 Summary.- 2 Adaptive Fir Filter Algorithms.- 2.1 Introduction.- 2.2 Optimum Linear Estimation.- 2.2.1 The Optimum FIR Filter.- 2.2.2 FIR System Identification.- 2.3 Sampled Matrix Inversion.- 2.4 Least Squares Estimation.- 2.4.1 Recursive Least Squares.- 2.4.2 Data Windows.- 2.4.3 Fast Algorithms.- 2.4.4 Properties of the Least Squares Estimate.- 2.5 Stochastic Gradient Methods.- 2.5.1 The Least Mean Squares Algorithm.- 2.5.2 The Block Least Mean Squares Algorithm.- 2.6 Self-Orthogonalising Algorithms.- 2.6.1 The Sliding DFT Adaptive Filter.- 2.7 Summary and Complexity Comparison.- 3 Performance Comparisons.- 3.1 Introduction.- 3.2 System Identification.- 3.3 Channel Equalisation.- 3.4 Summary and Conclusions.- 4 A Self-Orthogonalising Block Adaptive Filter.- 4.1 Introduction.- 4.2 Theoretical Development.- 4.2.1 Comparison of Theory with Simulation.- 4.3 A Practical Algorithm.- 4.4 Computational Complexity.- 4.5 Simulation Results.- 4.6 Conclusions.- 5 The Infinite Impulse Response Linear Equaliser.- 5.1 Introduction.- 5.2 The Linear Equaliser.- 5.2.1 Structure of an IIR Equaliser.- 5.3 FIR and IIR Equaliser Performance.- 5.4 System Identification.- 5.4.1 Adaptive IIR Solutions.- 5.5 Conclusions.- 6 An Adaptive IIR Equaliser.- 6.1 Introduction.- 6.2 The Kalman Filter.- 6.3 The Kalman Filter as an IIR Equaliser.- 6.4 An Adaptive Kalman Equaliser.- 6.4.1 System Identification.- 6.4.2 Model Uncertainty.- 6.4.3 Verification of Compensation Technique.- 6.4.4 Comparison with an RLS FIR Equaliser.- 6.4.5 Computational Complexity.- 6.5 RLS System Identification.- 6.6 Conclusions.- 7 Conclusions.- 7.1 Summary.- 7.2 Limitations and Further Work.- Appendix A The Fast Kalman Algorithm.- Appendix B The RLS Lattice Algorithm.- Appendix C Circular and Linear Convolution.- References.

On System Identification in one and two Dimensions with Signal Processing Applications

Abstract: This thesis consists of four parts, with system identification as the common theme. The first part studies the asymptotic properties of two-dimensional identification methods. In the second part an approach to identification of time varying systems is presented. Part three applies system identification to the problem of transmission line protection. Finally part four deals with input estimation in speech coding.Part I is devoted to system identification in two dimensions. First we study the asymptotic properties of the estimates as the number of data tends to infinity. The main objective is to investigate what happens if the model order also tends to infinity. The focus is on frequency expressions of the extimation variance. The analysis covers both the least squares method for causal models, and the maximum likelihood method for noncausal models.In Part II we study one approach to identification of time varying sytems. The parameter variations are modelled as process noise in a state space model, and identified using adaptive Kalman filtering. A method for adaptive Kalman filtering is derived and analysed. The simulations indicate that this new approach is superior to previous methods based on adjusting the forgetting factor. The improvement is however gained at the price of a significant increase in computational complexity.Part III describes the use of recursive identification in protective relaying. The Fourier coefficients of voltage and current are estimated using recursive least squares identification. The estimates are then used to detect short circuits. The method is evaluated using data generated by the standard program EMTP.In Part IV a method for inverse glottal filtering is presented. The basis of the method is to use a parameterized model of the input signal, i.e. the glottal pulses. The algorithm simultaneously estimates the parameters of the input signal and the parameters of the system transfer function, the vocal tract model. The presentation is restricted to transfer functions of all-pole type.

Frequency Domain Aspects of Modeling and Control in Adaptive Systems

Abstract: In this thesis various aspects of modeling and control in adaptive systems are presented from a frequency domain viewpoint.The thesis consists of three parts, where the first part contains a general introduction and background information concerning the problems that will be treated. In the second part some recursive identification algorithms are studied with respect to their ability to track time-varying systems and their disturbance sensitivity. Simple and illustrative frequency domain expressions that describe these properties are derived using asymptotic methods. The algorithms that are treated are the constant gain gradient (LMS) algorithm, the recursive least squares algorithm with constant forgetting factor and the Kalman filter respectively. The behavior of these methods when applied to FIR and ARX systems are studied. In the third part of the thesis adaptive control based on low order models is studied. The adaptive control algorithm that is investigated is the recursive least squares algorithm combined with pole placement regulator design. Starting from frequency domain expressions, that describe how a low order model obtained by system identification approximates a higher order system, the consequences for adaptive control are investigated. It is shown how a proper use of filtering in the identification part of the adaptive regulator can improve the robustness properties of the adaptive regulator with respect to unmodelled dynamics.

Numerically stable fast recursive least-squares transversal filters

Abstract: The problem of numerical stability of fast recursive least-squares transversal filter (FTF) algorithms is addressed. The prewindowing case with exponential weighting is considered. A framework for the analysis of the error propagation in these algorithms is developed. Within this framework, it is shown that the computationally most efficient 7N form (dealt with by G. Carayanmis et al. (1983) and by J.M. Cioffi (1984)) is exponentially unstable. By introducing redundancy in this algorithm, feedback of numerical errors becomes possible. This leads to a numerically stable FTF algorithm with complexity 9N. The results are presented for the complex multichannel joint-process filtering problem. >

Digital protective relaying through recursive least-squares identification

Abstract: In the paper, parameter estimation is applied to the problem of transmission line protection. The Fourier coefficients of voltage and current are estimated using recursive least-squares identification. The estimates are then used to detect short circuits. The method is evaluated using data generated by the program EMTP.

An investigation of block-adaptive decision feedback equalization for frequency selective fading channels

Abstract: Block adaptation techniques are investigated by computer simulation for decision feedback equalization on a rapidly fading dispersive channel. Trade-offs between training sequence length and block length are found. Block adaptation supplemented by linear interpolation of coefficients is found to be an attractive alternative to more complex continuously-adapting recursive least squares adaption algorithms.

A fast recursive least-squares second order Volterra filter

Abstract: A fast, recursive least-squares (RLS) adaptive nonlinear filter is presented. The nonlinearity is modeled using a second-order Volterra-series expansion. The structure makes use of the ideas of fast RLS multichannel filters and has a computational complexity of O(N/sup 3/) multiplications. This compares with O(N/sup 6/) multiplications required for direct implementation. Simulation examples in which the filter is employed to identify nonlinear systems using noisy output observations are also presented. Further simplification to the structure through a simplified model is discussed. >

Adaptive time delay estimation with constraints

Abstract: The time (shift) delay parameter between two signals is modeled as a finite-impulse response filter whose coefficients are samples of a sinc function. The time-domain LMS (least-mean-squares) adaptive algorithm is used, but only the weight with the largest magnitude is updated, which involves less computation. The result is a faster adaptation and the elimination of interpolation needed in previous approaches to obtain nonintegral (multiples of sampling period) time-delay estimates. >

An RML algorithm for retrieval of sinusoids with cascaded notch filters

Abstract: To provide frequency estimation, adaptive notch filters implemented in a cascade of second-order cells can be used, along with strategies to track time-varying parameters. The estimated frequencies are then used to provide the estimation of amplitudes and phases in a recursive manner. The proposed RML (recursive maximum-likelihood) algorithm consists of two steps. The first step involves a maximum-likelihood algorithm to adapt the cascaded filter parameters which will provide the frequency estimates. The second step uses these estimates, and the amplitudes and phases are estimated by a recursive-least-squares algorithm The RML algorithm is asymptotically consistent and robust with respect to the neglected dynamics. In the case of time-varying signals its tracking capabilities ensure the goodness of the estimations. >

Stabilization of the RLS algorithm in the absence of persistent excitation

Abstract: Stability problems associated with recursive least-squares (RLS) algorithms due to lack persistency exciting input data are considered. It is shown by an example that quantization of data and finite-word length computations assist each other in destroying persistent excitation. A projection operator formalism is used to interpret this effect for the square-foot factorized autocorrelation matrix estimator. This estimator is modified such that only the single dimension observed through the current input data is updated. Thereby divergence of unobserved modes is prevented. This new O(N/sup 2/) RLS algorithm with selective memory is computationally simple and stable even for small values of the forgetting factor. >

A constrained two-dimensional adaptive digital filter with reduced computational complexity

Abstract: An adaptive 2-D digital filter is presented in which the filter is determined by a 1-D FIR (finite-impulse-response) prototype filter, and hence the computational complexity of the coefficient-update algorithm has an order of complexity that is similar to a 1-D adaptive filter. The convergence rate of the proposed structure is much faster than that of a direct-form 2-D LMS (least-mean-square) filter. Results of the hardware realization of the filter are presented that show its learning curve to be considerably less noisy than that of the 2-D direct-form LMS filter. >

Expectations, learning and the kalman filter

Abstract: Summary Our final comments can be relatively brief. In assessing the empirical importance of expectations variables in macroeconomic behavioural equations, a variety of “expectations models” should be used. To date, the Muth-rational expectations approach has dominated the empirical literature. A major drawback, however, is the lack of an explicit optimal learning process by agents. We have attempted to remedy this by bringing together various diverse strands in the expectations, statistics and engineering literature to formalize models that embody “optimal information extraction” by agents faced with a stochastic environment. The Khan filter provides a unified method of approaching these problems and in this paper we presented the Kalman filter in terms of the usual least squares approach familiar to applied economists. Relatively inexpensive econometric software which utilizes recursive estimation techniques has recently become available. It is hoped that this paper has provided a framework favourable to its use by applied economists particularly in investigating the role of expectations variables in economic models.

Adaptive harmonic noise cancellation with an application to distribution power line communications

Abstract: Adaptive LMS (least mean squares) filter are applied to cancel harmonic noise. Simulations were done using actual noise recorded at a distribution power substation. Two implementation methods are discussed. Using the characteristic properties of the noise, a constrained filter is proposed which greatly reduces computation without reducing its effectiveness. It is shown that both constrained and unconstrained adaptive noise cancellation filters can be used effectively. For either case, significant improvement in the error rate is obtained. >

Efficient block implementation of exact sequential least-squares problems

Abstract: An efficient blockwise algorithm, namely the block sequential least-squares (BSLS) algorithm, is presented for sequentially solving LS problems in realtime. The information is carried from block to block by iterating some correlation vectors. In the case of successive data blocks, the exactness of the BSLS algorithm is achieved at approximately the same computational requirement as characterizes the nonexact BFTF (block fast transversal filter) algorithm, which is significantly less than sample-by-sample RLS (recursive least squares) algorithms. However, the BSLS cannot accommodate the case of discontinuous blocks of data, which can be accommodated (at the expense of a nonexact solution) by the BFTF. It is shown that the BSLS algorithm allows efficient use of the FFT fast Fourier transform technique to make remarkable gains in computational complexity savings. Additionally, the BSLS algorithm can provide an improved numerical stability over the existing fast RLS algorithms. The numerical performance is illustrated by applications to adaptive equalization and online parameter identification. >

A fast ARMA transversal RLS filter algorithm

Abstract: A fast pole-zero (ARMA) transversal RLS (recursive least squares) algorithm is derived, using a geometric formulation and the concept of projection onto a vector subspace to derive a recursive solution. The algorithm estimates a parameter vector that contains both numerator and denominator coefficients of an unknown system transfer function, i.e. models an ARMA (pole-zero) process. The algorithm has a transversal filter structure, but is distinguished from previous multichannel transversal algorithms, wherein each input channel is constrained to have the same order; here the pole and zero orders can be independently and arbitrarily specified. The derivation of the algorithm uses permutation matrices similar to those in the ARMA fast Kalman algorithm, but achieves a significant reduction in computations when compared to that algorithm. It is shown that when the pole and zero orders of the ARMA process are correctly specified, the algorithm generates an extremely good estimate. Furthermore, if the poles and zeros are overspecified, it is shown that a spectral match is still achieved by mutual cancellation of superfluous poles and zeros. >

Adjusting filter coefficients

Abstract: In an adaptive filter (56) the coefficients of a transversal filter (64) are adjusted by a processing circuit (78) to reduce the difference between a signal applied to the filter input (55) and a reference signal applied to a further input (57) so that the filter transfer characteristic becomes such that it will convert a first signal into a second signal. The first and second signals are constituted by a signal occurring in a signal path through a communications receiver and an ideal version of this signal respectively. In order that a relatively simple but hence relatively slow algorithm can be employed by the processing circuit without requiring a correspondingly long first signal the first and second signals are not applied to the adaptive filter directly but only after convolution in respective transversal filters (49,62) with the output of a noise source (60). The adaptive filter is then used as an equalizer in the receiver.

A fast quasi-Newton adaptive filtering algorithm

Abstract: An adaptive filtering algorithm is introduced which is largely immune to the deleterious effects of colored inputs, yet requires only O(N) computation. Simulation results are presented which demonstrate that the proposed algorithm's performance is comparable to RLS (recursive least squares), and that it is quite robust with respect to finite-wordlength implementation. >

Convergence analysis of adaptive linear estimation for dependent stationary processes

Abstract: The convergence properties of an adaptive linear mean-square estimator that uses a modified LMS algorithm are established for generally dependent processes. Bounds on the mean-square error of the estimates of the filter coefficients and on the excess error of the estimate of the signal are derived for input processes which are either strong mixing or asymptotically uncorrelated. It is shown that the mean-square deviation is bounded by a constant multiple of the adaptation step size and that the same holds for the excess error of the signal estimation. The present findings extend earlier results in the literature obtained for independent and M-dependent input data. >

Deterministic convergence analysis of RLS estimators with different forgetting factors

Abstract: Three forgetting factors recursive-least-squares (RLS) algorithms, as well as the classical error-forgetting one, are considered. The basic assumptions are that the data-generation mechanism is deterministic, the unknown parameter vector is constant, and the observation vector is persistently exciting. It is possible to prove the convergence of the estimates supplied by the various algorithms to the true parameter vector. This conclusion does not mean that the algorithm possess tracking capabilities when the unknown parameter vector is time-varying. In this case, the exponential convergence in the case of constant unknown parameters is much more important. >