Showing papers on "Recursive least squares filter published in 2002"

Adaptive Nonlinear System Identification with Echo State Networks

Abstract: Echo state networks (ESN) are a novel approach to recurrent neural network training. An ESN consists of a large, fixed, recurrent "reservoir" network, from which the desired output is obtained by training suitable output connection weights. Determination of optimal output weights becomes a linear, uniquely solvable task of MSE minimization. This article reviews the basic ideas and describes an online adaptation scheme based on the RLS algorithm known from adaptive linear systems. As an example, a 10-th order NARMA system is adaptively identified. The known benefits of the RLS algorithms carry over from linear systems to nonlinear ones; specifically, the convergence rate and misadjustment can be determined at design time.

Characterization and adaptive filtering of motion artifacts in pulse oximetry using accelerometers

Abstract: Noise, in the form of motion artifact, often leads to false information and acts as a limiting factor in the analysis of pulse oximetric signals. We propose an adaptive filter for filtering motion artifacts from pulse oximetry signals, with accelerometer signals as noise references. We study two adaptive filtering schemes: (1) single-axis and (2) dual-axes stress tests; and apply both the LMS and RLS algorithms to each scheme to compare their effectiveness. Results show that a single-axis adaptive filter employing the RLS algorithm (N=32 and /spl lambda/=0.9999) is adequate to minimize motion artifact.

Efficient reinforcement learning using recursive least-squares methods

Abstract: The recursive least-squares (RLS) algorithm is one of the most well-known algorithms used in adaptive filtering, system identification and adaptive control. Its popularity is mainly due to its fast convergence speed, which is considered to be optimal in practice. In this paper, RLS methods are used to solve reinforcement learning problems, where two new reinforcement learning algorithms using linear value function approximators are proposed and analyzed. The two algorithms are called RLS-TD(λ) and Fast-AHC (Fast Adaptive Heuristic Critic), respectively. RLS-TD(λ) can be viewed as the extension of RLS-TD(0) from λ =0 to general 0≤ λ ≤1, so it is a multi-step temporal-difference (TD) learning algorithm using RLS methods. The convergence with probability one and the limit of convergence of RLS-TD(λ) are proved for ergodic Markov chains. Compared to the existing LS-TD(λ) algorithm, RLS-TD(λ) has advantages in computation and is more suitable for online learning. The effectiveness of RLS-TD(λ) is analyzed and verified by learning prediction experiments of Markov chains with a wide range of parameter settings. The Fast-AHC algorithm is derived by applying the proposed RLS-TD(λ) algorithm in the critic network of the adaptive heuristic critic method. Unlike conventional AHC algorithm, Fast-AHC makes use of RLS methods to improve the learning-prediction efficiency in the critic. Learning control experiments of the cart-pole balancing and the acrobot swing-up problems are conducted to compare the data efficiency of Fast-AHC with conventional AHC. From the experimental results, it is shown that the data efficiency of learning control can also be improved by using RLS methods in the learning-prediction process of the critic. The performance of Fast-AHC is also compared with that of the AHC method using LS-TD(λ). Furthermore, it is demonstrated in the experiments that different initial values of the variance matrix in RLS-TD(λ) are required to get better performance not only in learning prediction but also in learning control. The experimental results are analyzed based on the existing theoretical work on the transient phase of forgetting factor RLS methods.

Adaptive noise cancelling of motion artifact in stress ECG signals using accelerometer

Abstract: Electrocardiographic (ECG) signals obtained from stress examinations are diagnostically significant in detecting a number of heart diseases, which may not be apparent when the patient is at rest. However, the noise produced by the environment and by the patient often distorts the ECG data. Motion artifact, the most prevalent and difficult type of noise to filter in exercise ECG, corrupts the intelligibility of the desired signal thus reducing the reliability of the stress test. In this paper, the researchers aim to demonstrate a new adaptive filtering method for stress ECG signals. This noise cancellation scheme uses an accelerometer as a source of noise reference. Experiments involving single-axis and dual-axis motion sensors are conducted to evaluate the efficiency of this technique. The acquired real ECG and accelerometer data are simultaneously processed and analyzed using the two most widely used adaptive filtering algorithms, least mean squares (LMS) and recursive least squares (RLS). The results show that the proposed method can be adapted to effectively reduce motion artifact in stress ECG by just using a single-axis noise reference.

Application of equality constraints on variables during alternating least squares procedures

Abstract: We describe several methods of applying equality constraints while performing procedures that employ alternating least squares. Among these are mathematically rigorous methods of applying equality constraints, as well as approximate methods, commonly used in chemometrics, that are not mathematically rigorous. The rigorous methods are extensions of the methods described in detail in Lawson and Hanson's landmark text on solving least squares problems, which exhibit well-behaved least squares performance. The approximate methods tend to be easy to use and code, but they exhibit poor least squares behaviors and have properties that are not well understood. This paper explains the application of rigorous equality-constrained least squares and demonstrates the dangers of employing non-rigorous methods. We found that in some cases, upon initiating multivariate curve resolution with the exact basis vectors underlying synthetic data overlaid with noise, the approximate method actually results in an increase in the magnitude of residuals. This phenomenon indicates that the solutions for the approximate methods may actually diverge from the least squares solution. Copyright © 2002 John Wiley & Sons, Ltd.

System and method for adjusting combiner weights using an adaptive algorithm in a wireless communications system

Abstract: Method and apparatus to compute the combiner coefficients for wireless communication systems using an adaptive algorithm. One embodiment trains the weights on a signal known a priori that is time multiplexed with other signals, such as a pilot signal in a High Data Rate, HDR, system, wherein the signal is transmitted at full power. The adaptive algorithm recursively computes the weights during the pilot interval and applies the weights generated to the traffic signals. In one embodiment, the algorithm is a recursive least squares algorithm employing a transversal filter and weight calculation unit.

A unified approach to form error evaluation

Abstract: Evaluation of form error is a critical aspect of many manufacturing processes. Machines such as the coordinate measuring machine (CMM) often employ the technique of the least squares form fitting algorithms. While based on sound mathematical principles, it is well known that the method of least squares often overestimates the tolerance zone, causing good parts to be rejected. Many methods have been proposed in efforts to improve upon results obtained via least squares, including those, which result in the minimum zone tolerance value. However, these methods are mathematically complex and often computationally slow for cases where a large number of data points are to be evaluated. Extensive amount of data is generated where measurement equipment such as laser scanners are used for inspection, as well as in reverse engineering applications. In this report, a unified linear approximation technique is introduced for use in evaluating the forms of straightness, flatness, circularity, and cylindricity. Non-linear equation for each form is linearized using Taylor expansion, then solved as a linear program using software written in C++ language. Examples are taken from the literature as well as from data collected on a coordinate measuring machine for comparison with least squares and minimum zone results. For all examples, the new formulations are found to equal or better than the least squares results and provide a good approximation to the minimum zone tolerance.

An adaptive Volterra predistorter for the linearization of RF high power amplifiers

Abstract: An efficient digital baseband predistortion linearizer is presented to compensate for nonlinear distortions induced by RF high power amplifiers in wireless communication systems. The proposed approach utilizes an indirect learning architecture with a fast recursive least squares (RLS) filtering algorithm, implemented using V-vector algebra, to update the coefficients of a Volterra-based predistorter. There is no requirement for an initial identification of the nonlinear characteristics of HPA as in linearizers based on conventional pth-order inverse methods. Simulation results show that that good performance and low computational complexity are achieved in the linearization of both narrow and wide bandwidth systems.

On-line gyro-based, mass-property identification for thruster-controlled spacecraft using recursive least squares

Abstract: Spacecraft control, state estimation, and fault-detection-and-isolation systems are affected by unknown variations in the vehicle mass properties. It is often difficult to accurately measure inertia terms on the ground, and mass properties can change on-orbit as fuel is expended, the configuration changes, or payloads are added or removed. Recursive least squares-based algorithms that use gyro signals to identify the center of mass and inverse inertia matrix are presented. They are applied in simulation to 3 thruster-controlled vehicles: the X-38 and Mini-AERCam under development at NASA-JSC, and the S4, an air-bearing spacecraft simulator at the NASA-Ames Smart Systems Research Lab (SSRL).

Least Squares and Kalman Filtering on Forney Graphs

Abstract: General versions of Kalman filtering and recursive least-squares algorithms are derived as instances of the sum(mary)-product algorithm on Forney-style factor graphs.

Block adaptive filters with deterministic reference inputs for event-related signals: BLMS and BRLS

Abstract: Adaptive estimation of the linear coefficient vector in truncated expansions is considered for the purpose of modeling noisy, recurrent signals. Two different criteria are studied for block-wise processing of the signal: the mean square error (MSE) and the least squares (LS) error. The block LMS (BLMS) algorithm, being the solution of the steepest descent strategy for minimizing the MSE, is shown to be steady-state unbiased and with a lower variance than the LMS algorithm. It is demonstrated that BLMS is equivalent to an exponential averager in the subspace spanned by the truncated set of basis functions. The block recursive least squares (BRLS) solution is shown to be equivalent to the BLMS algorithm with a decreasing step size. The BRLS is unbiased at any occurrence number of the signal and has the same steady-state variance as the BLMS but with a lower variance at the transient stage. The estimation methods can be interpreted in terms of linear, time-variant filtering. The performance of the methods is studied on an ECG signal, and the results show that the performance of the block algorithms is superior to that of the LMS algorithm. In addition, measurements with clinical interest are found to be more robustly estimated in noisy signals.

Adaptive Identification and Control of Flow-Induced Cavity Oscillations

Abstract: Progress towards an adaptive self-tuning regulator (STR) for the cavity tone problem is discussed in this paper. Adaptive system identification algorithms were applied to an experimental cavity-flow tested as a prerequisite to control. In addition, a simple digital controller and a piezoelectric bimorph actuator were used to demonstrate multiple tone suppression. The control tests at Mach numbers of 0.275, 0.40, and 0.60 indicated approx. = 7dB tone reductions at multiple frequencies. Several different adaptive system identification algorithms were applied at a single freestream Mach number of 0.275. Adaptive finite-impulse response (FIR) filters of orders up to N = 100 were found to be unsuitable for modeling the cavity flow dynamics. Adaptive infinite-impulse response (IIR) filters of comparable order better captured the system dynamics. Two recursive algorithms, the least-mean square (LMS) and the recursive-least square (RLS), were utilized to update the adaptive filter coefficients. Given the sample-time requirements imposed by the cavity flow dynamics, the computational simplicity of the least mean squares (LMS) algorithm is advantageous for real-time control.

Short-term electrical load forecasting using least squares support vector machines

Abstract: This paper presents a least squares support vector machines (LS-SVM) approach to short-term electric load forecasting (STLF). The proposed algorithm is more robust and reliable as compared to the traditional approach when actual loads are forecasted and used as input variables. In order to provide the forecasted load, the LS-SVM interpolates among the load and temperature data in a training data set. Analysis of the experimental results proved that this approach can achieve greater forecasting accuracy than the traditional model.

Dynamic modelling of a single-link flexible manipulator: parametric and non-parametric approaches

Abstract: This paper presents an investigation into the development of parametric and non-parametric approaches for dynamic modelling of a flexible manipulator system. The least mean squares, recursive least squares and genetic algorithms are used to obtain linear parametric models of the system. Moreover, non-parametric models of the system are developed using a non-linear AutoRegressive process with eXogeneous input model structure with multi-layered perceptron and radial basis function neural networks. The system is in each case modelled from the input torque to hub-angle, hub-velocity and end-point acceleration outputs. The models are validated using several validation tests. Finally, a comparative assessment of the approaches used is presented and discussed in terms of accuracy, efficiency and estimation of the vibration modes of the system.

Adaptive RLS algorithm for blind source separation using a natural gradient

Abstract: By using the natural gradient on the Stiefel manifold to minimize a nonlinear principle component analysis criterion, this letter proposes a new adaptive recursive-least-squares (RLS) algorithm with prewhitening for blind source separation (BSS), which makes full use of the orthogonality constraint of the separating matrix. Simulations show that the new natural-gradient-based RLS algorithm has faster convergence than the existing least-mean-square algorithms and RLS algorithm for BSS.

A new variable length LMS algorithm: theoretical analysis and implementations

Abstract: This paper addresses the problem of finding the optimum length for the adaptive least mean square (LMS) filter. In almost all papers published in this field, the length of the adaptive filter is maintained constant and the values of the coefficients are modified such that the output mean squared error (MSE) is minimized. There are some practical applications where we need to have information about the length of the optimum Wiener solution. As an example in system identification, one needs to have not only accurate approximation of the coefficient values but also the number of the coefficients of the unknown system. Here we provide the theoretical analysis of the LMS algorithm where the length mismatch between the adaptive filter and the unknown filter is taken into account. Based on this theoretical analysis a new variable length LMS algorithm is introduced.

Using Robust Estimation Algorithms for Tracking Explicit Curves

Abstract: The context of this work is lateral vehicle control using a camera as a sensor. A natural tool for controlling a vehicle is recursive filtering. The well-known Kalman filtering theory relies on Gaussian assumptions on both the state and measure random variables. However, image processing algorithms yield measurements that, most of the time, are far from Gaussian, as experimentally shown on real data in our application. It is therefore necessary to make the approach more robust, leading to the so-called robust Kalman filtering. In this paper, we review this approach from a very global point of view, adopting a constrained least squares approach, which is very similar to the half-quadratic theory, and justifies the use of iterative reweighted least squares algorithms. A key issue in robust Kalman filtering is the choice of the prediction error covariance matrix. Unlike in the Gaussian case, its computation is not straightforward in the robust case, due to the nonlinearity of the involved expectation. We review the classical alternatives and propose new ones. A theoretical study of these approximations is out of the scope of this paper, however we do provide an experimental comparison on synthetic data perturbed with Cauchy-distributed noise.

Method and apparatus for adjusting receiver voltage threshold and phase sampling point using FEC counts

Abstract: Methods and apparatus are provided for adjusting slicing parameters, such as a voltage threshold and a phase sampling point, used in recovering 1's and 0's from a signal so as to reduce a bit error rate (BER) of the signal The BER is modelled as a second order polynomial of the slicing parameters The BER is repeatedly determined from a Forward Error Correction corrected bits counter For each BER measurement the model is updated, using for example a recursive least squares fit New values of the slicing parameters are then determined by carrying out an iteration of an optimization, such as a Levenberg-Marquardt optimization, using the model The new values of the slicing parameters are passed to a Clock and Data Recovery module Various conditions are checked before updating the model or determining the new values of the slicing parameters, such as changes in signal power or high BERs which exceed the error correction capabilities of the forward error correction

Variable forgetting factor linear least squares algorithm for frequency selective fading channel estimation

Abstract: In this article, the variable forgetting factor linear least squares algorithm is presented to improve the tracking capability of channel estimation. A linear channel model with respect to time change describes a time-varying channel more accurately than a conventional stationary channel model. To reduce the estimation error due to model mismatch, we incorporate the modified variable forgetting factor into the proposed algorithm. Compared to the existing algorithms-exponentially windowed recursive least squares algorithm with the optimal forgetting factor and linear least squares algorithm-the proposed method makes a remarkable improvement in a fast fading environment. The effects of channel parameters such as signal-to-noise ratio and fading rate are investigated by computer simulations.

Comparative tracking performance of the LMS and RLS algorithms for chirped narrowband signal recovery

Abstract: This paper studies the comparative tracking performance of the recursive least squares (RLS) and least mean square (LMS) algorithms for time-varying inputs, specifically for linearly chirped narrowband input signals in additive white Gaussian noise. It is shown that the structural differences in the implementation of the LMS and RLS weight updates produce regions where the LMS performance exceeds that of the RLS and other regions where the converse occurs. These regions are shown to be a function of the signal bandwidth and signal-to-noise ratio (SNR). LMS is shown to place a notch in the signal band of the mean lag filter, thus reducing the lag error and improving the tracking performance. For the chirped signal, it is shown that this produces smaller tracking error for small SNR. For high SNR, there is a region of signal bandwidth for which RLS will provide lower error than LMS, but even for these high SNR inputs, LMS always provides superior performance for very narrowband signals.

Fast computation of multi-input-multi-output decision feedback equalizer coefficients

Abstract: Multi-Input-Multi-Output (MIMO) Optimal Decision Feedback Equalizer (DFE) coefficients are determined from a channel estimate h by casting the MIMO DFE coefficient problem as a standard recursive least squares (RLS) problem and solving the RLS problem. In one embodiment, a fast recursive method, e.g., fast transversal filter (FTF) technique, then used to compute the Kalman gain of the RLS problem, which is then directly used to compute MIMO Feed Forward Equalizer (FFE) coefficients gopt. The complexity of a conventional FTF algorithm is reduced to one third of its original complexity by choosing the length of a MIMO Feed Back Equalizer (FBE) coefficients bopt (of the DFE) to force the FTF algorithm to use a lower triangular matrix. The MIMO FBE coefficients bop, are computed by convolving the MIMO FFE coefficients gopt with the channel impulse response h. In performing this operation, a convolution matrix that characterizes the channel impulse response h extended to a bigger circulant matrix. With the extended circulant matrix structure, the convolution of the MIMO FFE coefficients gopt with the channel impulse response h may be performed easily performed in the frequency domain.

Fast principal component extraction by a weighted information criterion

Abstract: Principal component analysis (PCA) is an essential technique in data compression and feature extraction, and there has been much interest in developing fast PICA algorithms. On the basis of the concepts of both weighted subspace and information maximization, this paper proposes a weighted information criterion (WINC) for searching the optimal solution of a linear neural network. We analytically show that the optimum weights globally asymptotically converge to the principal eigenvectors of a stationary vector stochastic process. We establish a dependent relation of choosing the weighting matrix on statistics of the input process through the analysis of stability of the equilibrium of the proposed criterion. Therefore, we are able to reveal the constraint on the choice of a weighting matrix. We develop two adaptive algorithms based on the WINC for extracting in parallel multiple principal components. Both algorithms are able to provide adaptive step size, which leads to a significant improvement in the learning performance. Furthermore, the recursive least squares (RLS) version of WINC algorithms has a low computational complexity O(Np), where N is the input vector dimension, and p is the number of desired principal components. In fact, the WINC algorithm corresponds to a three-layer linear neural network model capable of performing, in parallel, the extraction of multiple principal components. The WINC algorithm also generalizes some well-known PCA/PSA algorithms just by adjusting the corresponding parameters. Since the weighting matrix does not require an accurate value, it facilitates the system design of the WINC algorithm for practical applications. The accuracy and speed advantages of the WINC algorithm are verified through simulations.

Variants of the Greville Formula with Applications to Exact Recursive Least Squares

Abstract: In this paper, we present an order-recursive formula for the pseudoinverse of a matrix. It is a variant of the well-known Greville [SIAM Rev., 2 (1960), pp. 578--619] formula. Three forms of the proposed formula are presented for three different matrix structures. Compared with the original Greville formula, the proposed formulas have certain merits. For example, they reduce the storage requirements at each recursion by almost half; they are more convenient for deriving recursive solutions for optimization problems involving pseudoinverses. Regarding applications, using the new formulas, we derive recursive least squares (RLS) procedures which coincide exactly with the batch LS solutions to the problems of the unconstrained LS, weighted LS, and LS with linear equality constraints, respectively, including their simple and exact initializations. Compared with previous results, e.g., Albert and Sittler [J. Soc. Indust. Appl. Math. Ser. A Control, 3 (1965), pp. 384--417], our derivation of the explicit recursive formulas is much easier, and the recursions take a much simpler form. New findings include that the linear equality constrained LS and the unconstrained LS can have an identical recursion---their only difference is the initial conditions. In addition, some robustness issues, in particular, during the exact initialization of the RLS are studied.

Linear System Identification

Abstract: This chapter contains sections titled: The ARMAX Model and Variations Uniqueness Properties Model Identifiability Prediction Error Methods Instrumental Variable Methods Recursive Least Squares Algorithm Model Validation Summary References Recommended Exercises

Adaptive linearly constrained inverse QRD-RLS beamforming algorithm for moving jammers suppression

Abstract: A general, linearly constrained (LC) recursive least squares (RLS) array-beamforming algorithm, based on an inverse QR decomposition, is developed for suppressing moving jammers efficiently. In fact, by using the inverse QR decomposition-recursive least squares (QRD-RLS) algorithm approach, the least-squares (LS) weight vector can be computed without back substitution and is suitable for implementation using a systolic array to achieve fast convergence and good numerical properties. The merits of this new constrained algorithm are verified by evaluating the performance, in terms of the learning curve, to investigate the convergence property and numerical efficiency, and the output signal-to-interference-and-noise ratio. We show that our proposed algorithm outperforms the conventional linearly constrained LMS (LCLMS) algorithm, and the one using the fast linear constrained RLS algorithm and its modified version.

Fast computation of mimo decision feedback equalizer coefficients

Abstract: Multi-Input-Multi-Output (MIMO) Decision Feedback Equalizer (DFE) coefficients are determined from a channel estimate h by casting the MIMO DFE coefficient problem as a standard recursive least squares (RLS) problem and solving the RLS problem In one embodiment, a fast recursive method, eg, fast transversal filter (FTF) technique, then used to compute the Kalman gain of the RLS problem, which is then directly used to compute MIMO Feed Forward Equalizer (FFE) coefficients gopt The complexity of a conventional FTF algorithm is reduced to one third of its original complexity by choosing the length of a MIMO Feed Back Equalizer (FBE) coefficients bopt (of the DFE) to force the FTF algorithm to use a lower triangular matrix The MIMO FBE coefficients bopt are computed by convolving the MIMO FFE coefficients gopt with the channel impulse response h In performing this operation, a convolution matrix that characterizes the channel impulse response h is extended to a bigger circulant matrix With the extended circulant matrix structure, the convolution of the MIMO FFE coefficients gopt with the channel impulse response h may be performed easily performed in the frequency domain