Showing papers on "Recursive least squares filter published in 2006"

Distributed Recursive Least-Squares Strategies Over Adaptive Networks

Abstract: A distributed least-squares estimation strategy is developed by appealing to collaboration techniques that exploit the space-time structure of the data, achieving an exact recursive solution that is fully distributed. Each node is allowed to communicate with its immediate neighbor in order to exploit the spatial dimension, while it evolves locally to account for the time dimension as well. In applications where communication and energy resources are scarce, an approximate RLS scheme that is also fully distributed is proposed in order to decrease the communication burden necessary to implement distributed collaborative solution. The performance of the resulting algorithm tends to its exact counterpart in the mean-square sense as the forgetting factor lambda tends to unity. A spatial-temporal energy conservation argument is used to evaluate the steady-state performance of the individual nodes across the adaptive distributed network for the low communications RLS implementation. Computer simulations illustrate the results.

The iterated sigma point kalman filter with applications to long range stereo

Abstract: This paper investigates the use of statistical linearization to improve iterative non-linear least squares estimators. In particular, we look at improving long range stereo by filtering feature tracks from sequences of stereo pairs. A novel filter called the Iterated Sigma Point Kalman Filter (ISPKF) is developed from first principles; this filter is shown to achieve superior performance in terms of efficiency and accuracy when compared to the Extended Kalman Filter (EKF), Unscented Kalman Filter (UKF), and Gauss-Newton filter. We also compare the ISPKF to the optimal Batch filter and to a Gauss-Newton Smoothing filter. For the long range stereo problem the ISPKF comes closest to matching the performance of the full batch MLE estimator. Further, the ISPKF is demonstrated on real data in the context of modeling environment structure from long range stereo data.

A Sliding-Window Kernel RLS Algorithm and Its Application to Nonlinear Channel Identification

Abstract: In this paper we propose a new kernel-based version of the recursive least-squares (RLS) algorithm for fast adaptive nonlinear filtering. Unlike other previous approaches, we combine a sliding-window approach (to fix the dimensions of the kernel matrix) with conventional L2-norm regularization (to improve generalization). The proposed kernel RLS algorithm is applied to a nonlinear channel identification problem (specifically, a linear filter followed by a memoryless nonlinearity), which typically appears in satellite communications or digital magnetic recording systems. We show that the proposed algorithm is able to operate in a time-varying environment and tracks abrupt changes in either the linear filter or the nonlinearity.

Adaptive filtering primer with MATLAB

Abstract: INTRODUCTION Signal Processing An Example Outline of the Text DISCRETE-TIME SIGNAL PROCESSING Discrete Time Signals Transform-Domain Representation of Discrete-Time Signals The Z-Transform Discrete-Time Systems Problems Hints-Solutions-Suggestions RANDOM VARIABLES, SEQUENCES, AND STOCHASTIC PROCESSES Random Signals and Distributions Averages Stationary Processes Special Random Signals and Probability Density Functions Wiener-Khinchin Relations Filtering Random Processes Special Types of Random Processes Nonparametric Spectra Estimation Parametric Methods of power Spectral Estimation Problems Hints-Solutions-Suggestions WIENER FILTERS The Mean-Square Error The FIR Wiener Filter The Wiener Solution Wiener Filtering Examples Problems Hints-Solutions-Suggestions EIGENVALUES OF RX - PROPERTIES OF THE ERROR SURFACE The Eigenvalues of the Correlation Matrix Geometrical Properties of the Error Surface Problems Hints-Solutions-Suggestions NEWTON AND STEEPEST-DESCENT METHOD One-Dimensional Gradient Search Method Steepest-Descent Algorithm Problems Hints-Solutions-Suggestions THE LEAST MEAN-SQUARE (LMS) ALGORITHM Introduction Derivation of the LMS Algorithm Examples Using the LMS Algorithm Equation Performance Analysis of the LMS Algorithm Equation Learning Curve Complex Representation of LMS Algorithm Problems Hints-Solutions-Suggestions VARIATIONS OF LMS ALGORITHMS The Sign Algorithms Normalized LMS (NLMS) Algorithm Variable Step-Size LMS (VSLMS) Algorithm The Leaky LMS Algorithm Linearly Constrained LMS Algorithm Self-Correcting Adaptive Filtering (SCAF) Transform Domain Adaptive LMS Filtering Error Normalized LMS Algorithms Problems Hints-Solutions-Suggestions LEAST SQUARES AND RECURSIVE LEAST-SQUARES SIGNAL PROCESSING Introduction to Least Squares Least-Square Formulation Least-Squares Approach Orthogonality Principle Projection Operator Least-Squares Finite Impulse Response Filter Introduction to RLS Algorithm Problems Hints-Solutions-Suggestions ABBREVIATIONS BIBLIOGRAPHY APPENDIX A: MATRIX ANALYSIS INDEX

Bias compensation based recursive least-squares identification algorithm for MISO systems

Abstract: For multi-input single-output output-error systems, the least-squares (LS) estimates are biased. In order to obtain the unbiased estimates, we present a recursive LS identification algorithm based on a bias compensation technique. The basic idea is to eliminate the estimation bias by adding a correction term in the LS estimates, and further to derive a bias compensation based recursive LS algorithm. Finally, we test the proposed algorithms by simulation and show their effectiveness.

Gain Adaptation Beats Least Squares

Abstract: I present computational results suggesting that gainadaptation algorithms based in part on connectionist learning methods may improve over least squares and other classical parameter-estimation methods for stochastic time-varying linear systems. The new algorithms are evaluated with respect to classical methods along three dimensions: asymptotic error, computational complexity, and required prior knowledge about the system. The new algorithms are all of the same order of complexity as LMS methods, O(n), where n is the dimensionality of the system, whereas least-squares methods and the Kalman filter are O(n). The new methods also improve over the Kalman filter in that they do not require a complete statistical model of how the system varies over time. In a simple computational experiment, the new methods are shown to produce asymptotic error levels near that of the optimal Kalman filter and significantly below those of least-squares and LMS methods. The new methods may perform better even than the Kalman filter if there is any error in the filter’s model of how the system varies over time.

Adaptive Estimation of Autoregression Models with Time-Varying Variances

Abstract: Stable autoregressive models of known finite order are considered with martingale differences errors scaled by an unknown nonparametric time-varying function generating heterogeneity. An important special case involves structural change in the error variance, but in most practical cases the pattern of variance change over time is unknown and may involve shifts at unknown discrete points in time, continuous evolution or combinations of the two. This paper develops kernel-based estimators of the residual variances and associated adaptive least squares (ALS) estimators of the autoregressive coefficients. These are shown to be asymptotically efficient, having the same limit distribution as the infeasible generalized least squares (GLS). Comparisons of the efficient procedure and ordinary least squares (OLS) reveal that least squares can be extremely inefficient in some cases while nearly optimal in others. Simulations show that, when least squares work well, the adaptive estimators perform comparably well, whereas when least squares work poorly, major efficiency gains are achieved by the new estimators.

Generalized Recursive Algorithm for Adaptive Multiparameter Regression Application to Lead Acid, Nickel Metal Hydride, and Lithium-Ion Batteries

Abstract: We derive and implement an algorithm that can accommodate an arbitrary number of model parameters, thereby allowing for more complicated battery models to be employed in formulating model reference adaptive systems as part of an energy management scheme for systems employing batteries. We employ the (controls) methodology of weighted recursive least squares with exponential forgetting. The output from the adaptive algorithm is the battery state of charge (remaining energy), state of health (relative to the battery's nominal rating), and power capability. The adaptive characterization of lead acid, nickel metal hydride, and lithium-ion batteries is investigated with the algorithm. The algorithm works well for lithium-ion and lead-acid batteries; more work is needed on nickel metal hydride batteries.

Convex Combination of Adaptive Filters for a Variable Tap-Length LMS Algorithm

Abstract: A convex combination of adaptive filters is utilized to improve the performance of a variable tap-length least-mean-square (LMS) algorithm in a low signal-to-noise environment (SNRles0 dB). As shown by our simulations, the adaptation of the tap-length in the variable tap-length LMS algorithm is highly affected by the parameter choice and the noise level. Combination approaches can improve such adaptation by exploiting advantages of parallel adaptive filters with different parameters. Simulation results support the good properties of the proposed method

Adaptive Learning in Practice

Abstract: We analyse some practical aspects of implementing adaptive learning in the context of forward-looking linear models. In particular, we focus on how to set initial conditions for three popular algorithms, namely recursive least squares, stochastic gradient and constant gain learning. We propose three ways of initializing, one that uses randomly generated data, a second that is ad-hoc and a third that uses an appropriate distribution. We illustrate, via standard examples, that the behaviour and evolution of macroeconomic variables not only depend on the learning algorithm, but on the initial conditions as well. Furthermore, we provide a computing toolbox for analysing the quantitative properties of dynamic stochastic macroeconomic models under adaptive learning.

Internet-Based Bilateral Teleoperation Based on Wave Variable With Adaptive Predictor and Direct Drift Control

Abstract: In a conventional bilateral teleoperation, transmission delay over the Internet can potentially cause instability. A wave variable algorithm guarantees teleoperation stability under varying transmission delay at the cost of poor transient performance. Adding a predictor on the master side can reduce this undesirable side effect, but that would require a slave model. An inaccurate slave model used in the predictor as well as variations in transmission delay, both of which are likely under realistic situations, can result in steady-state errors. A direct drift control algorithm is used to drive this error to zero, regardless of the source of the error. A semi-adaptive predictor that can distinguish between free space and a rigid contact environment is used to provide a more accurate force feedback on the master side. A full adaptive predictor is also used that estimates the environmental force using recursive least squares with a forgetting factor. This research presents the experimental results and evaluations of the previously mentioned wavevariable-based methods under a realistic operation environment using a real master and slave. The algorithm proposed is innovative in that it takes advantage of the strengths of several control methods to build a promising bilateral teleoperation setup that can function under varying transmission delay, modeling error, and changing environment. Success could lead to practical applications in various fields, such as space-based remote control, and telesurgey.

Comparison of Adaptive Methods for Clutter Removal in PCL Radar

Abstract: The paper presents a comparison of classical adaptive filters for suppression of direct path interference and ground clutter in Passive Coherent Location (PCL) radar. The following algorithms were tested: Least Mean Squares, Recursive Least Squares and Least Square Lattice. The filtering methods were compared from the convergence rate, computational complexity and frequency filtering properties point of view.

Acoustic feedback cancellation for long acoustic paths using a nonstationary source model

Abstract: While several proactive acoustic feedback (Larsen-effect) cancellation schemes have been presented for speech applications with short acoustic feedback paths as encountered in hearing aids, these schemes fail with the long impulse responses inherent to, for instance, public address systems. We derive a new prediction error method (PEM)-based scheme (referred to as PEM-AFROW) which identifies both the acoustic feedback path and the nonstationary speech source model. A cascade of a short- and a long-term predictor removes the coloring and periodicity in voiced speech segments, which account for the unwanted correlation between the loudspeaker signal and the speech source signal. The predictors calculate row operations which are applied to prewhiten the speech source signal, resulting in a least squares system that is solved recursively by means of normalized least mean square or recursive least squares algorithms. Simulations show that this approach is indeed superior to earlier approaches whenever long acoustic channels are dealt with

Hybrid filtered error LMS algorithm: another alternative to filtered-x LMS

Abstract: The filtered-error LMS (FELMS) algorithms are widely used in multi-input and multi-output control (MIMO) active noise control (ANC) systems as an alternative to the filtered-x LMS (FXLMS) algorithms to reduce the computational complexity and memory requirements. However, the available FELMS algorithms introduce significant delays in updating the adaptive filter coefficients that slow the convergence rate. In this paper, we introduce a novel algorithm called the hybrid filtered-error LMS algorithm (HFELMS) which, while still a form of the FELMS algorithm, allows users to have some freedom to construct the error filter that guarantees its convergence with a sufficiently small step size. Without increasing the computational complexity, the proposed algorithm can improve the control system performance in one of several ways: 1) increasing the convergence rate without extra computation cost; 2) reducing the remaining noise mean square error (MSE); or 3) shaping the excess noise power. Simulation results show the effectiveness of the proposed method

Adaptive MAP receiver via the EM algorithm and message passings for MIMO-OFDM mobile communications

Abstract: This paper proposes two new types of maximum a posteriori probability (MAP) receivers for multiple-input-multiple-output and orthogonal frequency-division multiplexing mobile communications with a channel coding such as the low-density parity-check code. One proposed receiver employs the expectation-maximization algorithm so as to improve performance of approximated MAP detection. Differently from a conventional receiver employing the minimum mean-square estimation (MMSE) algorithm, it applies the recursive least squares (RLS) algorithm to the channel estimation in order to track a fast fading channel. For the purpose of further improvement, the other proposed receiver applies a new adaptive algorithm that can be derived from the message passing on factor graphs. The algorithm exploits all detected signals but one of targeted time, and can gain a considerable advantage over the MMSE and RLS. Computer simulations show that the first proposed receiver is superior in channel-tracking ability to the conventional receiver employing the MMSE. Furthermore, it is demonstrated that the second proposed receiver remarkably outperforms both the conventional and the first proposed ones.

Generalized RLS approach to the training of neural networks

Abstract: Recursive least square (RLS) is an efficient approach to neural network training. However, in the classical RLS algorithm, there is no explicit decay in the energy function. This will lead to an unsatisfactory generalization ability for the trained networks. In this paper, we propose a generalized RLS (GRLS) model which includes a general decay term in the energy function for the training of feedforward neural networks. In particular, four different weight decay functions, namely, the quadratic weight decay, the constant weight decay and the newly proposed multimodal and quartic weight decay are discussed. By using the GRLS approach, not only the generalization ability of the trained networks is significantly improved but more unnecessary weights are pruned to obtain a compact network. Furthermore, the computational complexity of the GRLS remains the same as that of the standard RLS algorithm. The advantages and tradeoffs of using different decay functions are analyzed and then demonstrated with examples. Simulation results show that our approach is able to meet the design goals: improving the generalization ability of the trained network while getting a compact network.

Adaptive Harmonic Steady-State Control for Disturbance Rejection

Abstract: We consider harmonic steady-state (HSS) control for active noise and vibration rejection when the system dynamics are unknown. After a brief review and analysis of the HSS control theory, we develop an adaptive control algorithm based on a recursive least squares algorithm that estimates the system dynamics. Active noise cancellation in an acoustic drum is demonstrated using the adaptive control algorithm. The results presented here unify and extend previous results on HSS control

Stereophonic acoustic echo cancellation employing selective-tap adaptive algorithms

Abstract: Stereophonic acoustic echo cancellation has generated much interest in recent years due to the nonuniqueness and misalignment problems that are caused by the strong interchannel signal coherence. In this paper, we introduce a novel adaptive filtering approach to reduce interchannel coherence which is based on a selective-tap updating procedure. This tap-selection technique is then applied to the normalized least-mean-square, affine projection and recursive least squares algorithms for stereophonic acoustic echo cancellation. Simulation results for the proposed algorithms have shown a significant improvement in convergence rate compared with existing techniques.

Adaptive Channel Estimation for OFDM Systems with Doppler spread

Abstract: In this paper, we propose adaptive channel estimation for Orthogonal Frequency Division Multiplexing (OFDM) in fast time-varying (TV) channels. A Basis Expansion Model (BEM) approach is used to capture the time variation of the channel within each OFDM block, and to reduce the estimator dimensionality. Capitalizing on the BEM structure and on a frequency domain training, two adaptive approaches are proposed, based on Kalman filtering and Recursive Least Squares (LS) methods, which exploit the time correlation of the channel between successive blocks and do not require any a-priori knowledge of the channel statistics. Simulation results show that, compared to classical Least Squares and statistically-aided Linear Minimum Mean Squared Error (LMMSE) approaches, the two proposed techniques effectively estimate the channel, adapt fast to its non stationary changes, thus enabling efficient TV channel equalization of the inter-carrier interference (ICI) induced in OFDM systems by high Doppler spreads.

Pairwise Optimal Weight Realization—Acceleration Technique for Set-Theoretic Adaptive Parallel Subgradient Projection Algorithm

Abstract: The adaptive parallel subgradient projection (PSP) algorithm was proposed in 2002 as a set-theoretic adaptive filtering algorithm providing fast and stable convergence, robustness against noise, and low computational complexity by using weighted parallel projections onto multiple time-varying closed half-spaces. In this paper, we present a novel weighting technique named pairwise optimal weight realization (POWER) for further acceleration of the adaptive PSP algorithm. A simple closed-form formula is derived to compute the projection onto the intersection of two closed half-spaces defined by a triplet of vectors. Using the formula inductively, the proposed weighting technique realizes a good direction of update. The resulting weights turn out to be pairwise optimal in a certain sense. The proposed algorithm has the inherently parallel structure composed of q primitive functions, hence its total computational complexity O(qrN) is reduced to O(rN) with q concurrent processors (r: a constant positive integer). Numerical examples demonstrate that the proposed technique for r=1 yields significantly faster convergence than not only adaptive PSP with uniform weights, affine projection algorithm, and fast Newton transversal filters but also the regularized recursive least squares algorithm

Decision-directed recursive least squares MIMO channels tracking

Abstract: A new approach for joint data estimation and channel tracking for multiple-input multiple-output (MIMO) channels is proposed based on the decision-directed recursive least squares (DD-RLS) algorithm. RLS algorithm is commonly used for equalization and its application in channel estimation is a novel idea. In this paper, after defining the weighted least squares cost function it is minimized and eventually the RLS MIMO channel estimation algorithm is derived. The proposed algorithm combined with the decision-directed algorithm (DDA) is then extended for the blind mode operation. From the computational complexity point of view being O(3) versus the number of transmitter and receiver antennas, the proposed algorithm is very efficient. Through various simulations, the mean square error (MSE) of the tracking of the proposed algorithm for different joint detection algorithms is compared with Kalman filtering approach which is one of the most well-known channel tracking algorithms. It is shown that the performance of the proposed algorithm is very close to Kalman estimator and that in the blind mode operation it presents a better performance with much lower complexity irrespective of the need to know the channel model.

A floating-point FPGA-based self-tuning regulator

Abstract: Recursive-least-square (RLS) algorithm is widely used in many areas with real-time implementation using digital signal processors. In this paper, the authors present a pure hardware implementation of a self-tuning regulator (STR) that uses a real-time RLS algorithm as the parameter estimator. The STR contains a controller design circuit and a controller circuit. Due to RLS computation-precision and dynamic-range requirements, the hardware implementation uses a floating-point format. The floating-point processing elements presented in this paper use parameterized design, where the number of exponents and mantissa bits can be changed as the data range and the accuracy of a specific application require. The strategies for overcoming the covariance matrix asymmetrical problem during the hardware computation and the covariance matrix resetting is introduced when the system is poorly exciting are presented. The design was verified with real-time experiments using a new testbed. The experiment results are presented.

A robust past algorithm for subspace tracking in impulsive noise

Abstract: The PAST algorithm is an effective and low complexity method for adaptive subspace tracking. However, due to the use of the recursive least squares (RLS) algorithm in estimating the conventional correlation matrix, like other RLS algorithms, it is very sensitive to impulsive noise and the performance can be degraded substantially. To overcome this problem, a new robust correlation matrix estimate, based on robust statistics concept, is proposed in this paper. It is derived from the maximum-likelihood (ML) estimate of a multivariate Gaussian process in contaminated Gaussian noise (CG) similar to the M-estimates in robust statistics. This new estimator is incorporated into the PAST algorithm for robust subspace tracking in impulsive noise. Furthermore, a new restoring mechanism is proposed to combat the hostile effect of long burst of impulses, which sporadically occur in communications systems. The convergence of this new algorithm is analyzed by extending a previous ordinary differential equation (ODE)-based method for PAST. Both theoretical and simulation results show that the proposed algorithm offers improved robustness against impulsive noise over the PAST algorithm. The performance of the new algorithm in nominal Gaussian noise is very close to that of the PAST algorithm.

Multi-mode vibration control of a flexible cantilever beam using adaptive resonant control

Abstract: In this paper, an adaptive resonant controller is used to attenuate multi-mode vibrations in a flexible cantilever beam structure with varying loading conditions. This controller is particularly designed for structures that are exposed to previously unmodelled dynamics. On-line estimation of the structure's natural frequencies is used to update the adaptive resonant controller's parameters. The estimation of the natural frequencies is achieved using a parallel set of second-order recursive least squares estimators, each of which is designed for a specific vibration mode of concern. To achieve the desired estimation accuracy for each mode frequency, a different sampling rate suitable for that mode is used for the corresponding estimator. Experiment results show that the proposed adaptive strategy can achieve better performance, as measured by attenuation level, over its fixed-parameter counterpart for a range of unmodelled dynamics.

Fast adaptive blind beamforming algorithm for antenna array in CDMA systems

Abstract: In this paper, the maximum signal-to-interference-plus-noise ratio (MSINR) beamforming problem in antenna-array CDMA systems is considered. In this paper, a modified MSINR criterion presented in a previous paper is interpreted as an unconstrained scalar cost function. By applying recursive least squares (RLS) to minimize the cost function, a novel blind adaptive beamforming algorithm to estimate the beamforming vector, which optimally combines the desired signal contributions from different antenna elements while suppressing noise and interference, is derived. Neither the knowledge of the channel conditions (fading coefficients, signature sequences and timing of interferers, statistics of other noises, etc.) nor training sequence is required. Compared with previously published adaptive beamforming algorithms based on the stochastic-gradient method, it has faster convergence and better tracking capability in the time-varying environment. Simulation results in various signal environments are presented to show the performance of the proposed algorithm.

A QRD-RLS-Based Predistortion Scheme for High-Power Amplifier Linearization

Abstract: A digital baseband predistortion (PD) scheme for high-power amplifier (HPA) linearization is proposed and analyzed in this brief. The proposed approach utilizes the QR-decomposition-based recursive least squares (QRD-RLS) algorithm to estimate the memoryless complex polynomial coefficients that characterize the HPA. The inverse polynomial model coefficients corresponding to the PD are similarly extracted using QRD-RLS. The performance of the proposed PD scheme is analyzed via simulations and compared with previously published techniques. Results show that the QRD-RLS-based solution offers improved performance over its comparatives

A Robust Variable Step Size Algorithm for Lms Adaptive Filters

Abstract: This work presents a modified version of the variable step size Kwong and Johnston's algorithm (VSS) for LMS adaptive filtering. The new proposal, called Robust Variable Step Size (RVSS), presents less sensitivity to the power of the measurement noise with only a very small increase in the computational complexity. A theoretical analysis demonstrates the main properties of the new algorithm. For white Gaussian input signals the RVSS presents the same performance than the original VSS in a noise free environment. Simulation results are provided, showing the better performance of the new algorithm. The RVSS should find application, for example, in telephony applications when double talking interferences are significant.

Least Squares SVM for Least Squares TD Learning

Abstract: We formulate the problem of least squares temporal difference learning (LSTD) in the framework of least squares SVM (LS-SVM). To cope with the large amount (and possible sequential nature) of training data arising in reinforcement learning we employ a subspace based variant of LS-SVM that sequentially processes the data and is hence especially suited for online learning. This approach is adapted from the context of Gaussian process regression and turns the unwieldy original optimization problem (with computational complexity being cubic in the number of processed data) into a reduced problem (with computional complexity being linear in the number of processed data). We introduce a QR decomposition based approach to solve the resulting generalized normal equations incrementally that is numerically more stable than existing recursive least squares based update algorithms. We also allow a forgetting factor in the updates to track non-stationary target functions (i.e. for the use with optimistic policy iteration). Experimental comparison with standard CMAC function approximation indicate that LS-SVMs are well-suited for online RL.