Showing papers on "Recursive least squares filter published in 2007"
30 Jul 2007
TL;DR: The message-passing approach to model-based signal processing is developed with a focus on Gaussian message passing in linear state-space models, which includes recursive least squares, linear minimum-mean-squared-error estimation, and Kalman filtering algorithms.
Abstract: The message-passing approach to model-based signal processing is developed with a focus on Gaussian message passing in linear state-space models, which includes recursive least squares, linear minimum-mean-squared-error estimation, and Kalman filtering algorithms. Tabulated message computation rules for the building blocks of linear models allow us to compose a variety of such algorithms without additional derivations or computations. Beyond the Gaussian case, it is emphasized that the message-passing approach encourages us to mix and match different algorithmic techniques, which is exemplified by two different approaches - steepest descent and expectation maximization - to message passing through a multiplier node.
517 citations
TL;DR: The estimation of sparse shallow-water acoustic communication channels and the impact of estimation performance on the equalization of phase coherent communication signals are investigated and a sparse channel estimation technique is developed based on the delay-Doppler-spread function representation of the channel.
Abstract: The estimation of sparse shallow-water acoustic communication channels and the impact of estimation performance on the equalization of phase coherent communication signals are investigated. Given sufficiently wide transmission bandwidth, the impulse response of the shallow-water acoustic channel is often sparse as the multipath arrivals become resolvable. In the presence of significant surface waves, the multipath arrivals associated with surface scattering fluctuate rapidly over time, in the sense that the complex gain, the arrival time, and the Dopplers of each arrival all change dynamically. A sparse channel estimation technique is developed based on the delay-Doppler-spread function representation of the channel. The delay-Doppler-spread function may be considered as a first-order approximation to the rapidly time-varying channel in which each channel component is associated with Doppler shifts that are assumed constant over an averaging interval. The sparse structure of the delay-Doppler-spread function is then exploited by sequentially choosing the dominant components that minimize a least squares error. The advantage of this approach is that it captures both the channel structure as well as its dynamics without the need of explicit dynamic channel modeling. As the symbols are populated with the sample Dopplers, the increase in complexity depends on the channel Doppler spread and can be significant for a severely Doppler-spread channel. Comparison is made between nonsparse recursive least squares (RLS) channel estimation, sparse channel impulse response estimation, and estimation using the proposed approach. The results are demonstrated using experimental data. In training mode, the proposed approach shows a 3-dB reduction in signal prediction error. In decision-directed mode, it improves significantly the robustness of the performance of the channel-estimate-based equalizer against rapid channel fluctuations.
426 citations
01 Jan 2007
TL;DR: In this paper, the message-passing approach to model-based signal processing is developed with a focus on Gaussian message passing in linear state-space models, which includes recursive least squares, linear minimum-mean-squared-error estimation, and Kalman filtering algorithms.
Abstract: The message-passing approach to model-based signal processing is developed with a focus on Gaussian message passing in linear state-space models, which includes recursive least squares, linear minimum-mean-squared-error estimation, and Kalman filtering algorithms. Tabulated mes- sage computation rules for the building blocks of linear models allow us to compose a variety of such algorithms without additional derivations or computations. Beyond the Gaussian case, it is emphasized that the message-passing approach encourages us to mix and match different algorithmic tech- niques, which is exemplified by two different approachesV steepest descent and expectation maximizationVto message passing through a multiplier node.
389 citations
TL;DR: This work examines “truncated” and “perturbed” Gauss-Newton methods where the inner linear least squares problem is not solved exactly, and two types of approximation used commonly in data assimilation.
Abstract: The Gauss-Newton algorithm is an iterative method regularly used for solving nonlinear least squares problems. It is particularly well suited to the treatment of very large scale variational data assimilation problems that arise in atmosphere and ocean forecasting. The procedure consists of a sequence of linear least squares approximations to the nonlinear problem, each of which is solved by an “inner” direct or iterative process. In comparison with Newton’s method and its variants, the algorithm is attractive because it does not require the evaluation of second-order derivatives in the Hessian of the objective function. In practice the exact Gauss-Newton method is too expensive to apply operationally in meteorological forecasting, and various approximations are made in order to reduce computational costs and to solve the problems in real time. Here we investigate the effects on the convergence of the Gauss-Newton method of two types of approximation used commonly in data assimilation. First, we examine “truncated” Gauss-Newton methods where the inner linear least squares problem is not solved exactly, and second, we examine “perturbed” Gauss-Newton methods where the true linearized inner problem is approximated by a simplified, or perturbed, linear least squares problem. We give conditions ensuring that the truncated and perturbed Gauss-Newton methods converge and also derive rates of convergence for the iterations. The results are illustrated by a simple numerical example. A practical application to the problem of data assimilation in a typical meteorological system is presented.
190 citations
01 May 2007
TL;DR: An online, sequential, anomaly detection algorithm that is suitable for use with multivariate data, based on the kernel version of the recursive least squares algorithm, that raises an alarm immediately upon encountering a deviation from the norm is described.
Abstract: High-speed backbones are regularly affected by various kinds of network anomalies, ranging from malicious attacks to harmless large data transfers. Different types of anomalies affect the network in different ways, and it is difficult to know a priori how a potential anomaly will exhibit itself in traffic statistics. In this paper we describe an online, sequential, anomaly detection algorithm, that is suitable for use with multivariate data. The proposed algorithm is based on the kernel version of the recursive least squares algorithm. It assumes no model for network traffic or anomalies, and constructs and adapts a dictionary of features that approximately spans the subspace of normal behaviour. The algorithm raises an alarm immediately upon encountering a deviation from the norm. Through comparison with existing block-based offline methods based upon Principal Component Analysis, we demonstrate that our online algorithm is equally effective but has much faster time-to-detection and lower computational complexity. We also explore minimum volume set approaches in identifying the region of normality.
164 citations
TL;DR: Results show that forecasts obtained from recursive adaptive filtering methods are comparable with those from maximum likelihood estimated models, and the adaptive methods deliver this performance at a significantly lower computational cost.
Abstract: Conventionally, most traffic forecasting models have been applied in a static framework in which new observations are not used to update model parameters automatically. The need to perform periodic parameter reestimation at each forecast location is a major disadvantage of such models. From a practical standpoint, the usefulness of any model depends not only on its accuracy but also on its ease of implementation and maintenance. This paper presents an adaptive parameter estimation methodology for univariate traffic condition forecasting through use of three well-known filtering techniques: the Kalman filter, recursive least squares, and least mean squares. Results show that forecasts obtained from recursive adaptive filtering methods are comparable with those from maximum likelihood estimated models. The adaptive methods deliver this performance at a significantly lower computational cost. As recursive, self-tuning predictors, the adaptive filters offer plug-and-play capability ideal for implementation in...
146 citations
01 Dec 2007
TL;DR: New and improved algorithms for the least-squares NNMA problem are presented which are not only theoretically well-founded, but also overcome many of the deficiencies of other methods, and use non-diagonal gradient scaling to obtain rapid convergence.
Abstract: Nonnegative Matrix Approximation is an effective matrix decomposition technique that has proven to be useful for a wide variety of applications ranging from document analysis and image processing to bioinformatics. There exist a few algorithms for nonnegative matrix approximation (NNMA), for example, Lee & Seung’s multiplicative updates, alternating least squares, and certain gradient descent based procedures. All of these procedures suffer from either slow convergence, numerical instabilities, or at worst, theoretical unsoundness. In this paper we present new and improved algorithms for the least-squares NNMA problem, which are not only theoretically well-founded, but also overcome many of the deficiencies of other methods. In particular, we use non-diagonal gradient scaling to obtain rapid convergence. Our methods provide numerical results superior to both Lee & Seung’s method as well to the alternating least squares (ALS) heuristic, which is known to work well in some situations but has no theoretical guarantees (Berry et al. 2006). Our approach extends naturally to include regularization and box-constraints, without sacrificing convergence guarantees. We present experimental results on both synthetic and realworld datasets to demonstrate the superiority of our methods, in terms of better approximations as well as efficiency.
136 citations
TL;DR: Both results from numerical simulations and experiments show that the proposed method is capable of controlling industrial processes with satisfactory performance under setpoint and load changes.
134 citations
TL;DR: This work proposes a generalization of CCA to several data sets, which is shown to be equivalent to the classical maximum variance (MAXVAR) generalization proposed by Kettenring.
122 citations
TL;DR: Experimental results have demonstrated that the proposed filter outperforms many well-accepted median-based filters in terms of both noise suppression and detail preservation and provides excellent robustness at various percentages of impulsive noise.
115 citations
TL;DR: The proposed adaptive linear-receiver structure based on interpolated finite-impulse response filters with adaptive interpolators for direct-sequence code-division multiple-access systems in multipath channels achieves a superior bit-error-rate convergence and steady-state performance to previously reported reduced-rank receivers at lower complexity.
Abstract: In this paper, we propose an adaptive linear-receiver structure based on interpolated finite-impulse response (FIR) filters with adaptive interpolators for direct-sequence code-division multiple-access (DS-CDMA) systems in multipath channels. The interpolated minimum mean-squared error (MMSE) and the interpolated constrained minimum-variance (CMV) solutions are described for a novel scheme, where the interpolator is rendered time-varying in order to mitigate multiple-access interference and multiple-path propagation effects. Based upon the interpolated MMSE and CMV solutions, we present computationally efficient stochastic gradient and exponentially weighted recursive least squares type algorithms for both receiver and interpolator filters in the supervised and blind modes of operation. A convergence analysis of the algorithms and a discussion of the convergence properties of the method are carried out for both modes of operation. Simulation experiments for a downlink scenario show that the proposed structures achieve a superior bit-error-rate convergence and steady-state performance to previously reported reduced-rank receivers at lower complexity.
01 Jan 2007
TL;DR: A single layer functional link ANN is used for the model where the need of hidden layer is eliminated by expanding the input pattern by Chebyshev polynomials, and creation of nonlinear decision boundaries in the multidimensional input space and approximation of complex nonlinear systems becomes easier.
Abstract: This paper proposes a computationally efficient artificial neural network (ANN) model for system identification of unknown dynamic nonlinear discrete time systems. A single layer functional link ANN is used for the model where the need of hidden layer is eliminated by expanding the input pattern by Chebyshev polynomials. Thus, creation of nonlinear decision boundaries in the multidimensional input space and approximation of complex nonlinear systems becomes easier. These models are linear in their parameters and nonlinear in the inputs. The recursive least squares method with forgetting factor is used as on-line learning algorithm for parameter updation. The good behaviour of the identification method is tested on Box and Jenkins Gas furnace benchmark identification problem, single input single output (SISO) and multi input multi output (MIMO) discrete time plants. Stability of the identification scheme is also addressed.
TL;DR: A single layer functional link ANN is used for the model where the need of hidden layer is eliminated by expanding the input pattern by Chebyshev polynomials, and creation of nonlinear decision boundaries in the multidimensional input space and approximation of complex nonlinear systems becomes easier.
Abstract: This paper proposes a computationally efficient artificial neural network (ANN) model for system identification of unknown dynamic nonlinear discrete time systems. A single layer functional link ANN is used for the model where the need of hidden layer is eliminated by expanding the input pattern by Chebyshev polynomials. Thus, creation of nonlinear decision boundaries in the multidimensional input space and approximation of complex nonlinear systems becomes easier. These models are linear in their parameters and nonlinear in the inputs. The recursive least squares method with forgetting factor is used as on-line learning algorithm for parameter updation. The good behaviour of the identification method is tested on Box and Jenkins Gas furnace benchmark identification problem, single input single output (SISO) and multi input multi output (MIMO) discrete time plants. Stability of the identification scheme is also addressed.
TL;DR: A new approach for real-time tracking of resolver parameters specially developed for actuator-control applications with varying speed and long resting periods is proposed and a new recursive and adaptive estimator is proposed to track the parameters of characteristic ellipse.
Abstract: Resolver sensors are utilized as absolute position transducers to control the position and speed of actuators in many industrial applications. The accuracy and convergence of the position and speed measurements provided by resolvers in electromechanical braking system (EMB) designs directly contribute to the braking performance and vehicle safety. In practice, the dc drifts, amplitudes, and phase shift of the resolver signals vary with aging and temperature, and adaptive techniques are required for the calibration of these parameters of resolvers. Existing classical adaptive techniques such as recursive least squares are unable to track the parameters during resting (low-speed actuation or stationary) periods and also a transient period after them. This paper proposes a new approach for real-time tracking of resolver parameters specially developed for actuator-control applications with varying speed and long resting periods. We formulate the algebraic relationship between the resolver parameters and the parameters of resolver characteristic ellipse, which is the ellipse formed by plotting the resolver signals versus each other. Having known the characteristic ellipse parameters, the resolver parameters are calculated using the formulated algebraic relation. Then, a new recursive and adaptive estimator is proposed to track the parameters of characteristic ellipse. The low computational complexity of the proposed method makes it desirable for real-time applications like the EMBs, where limited computational power and memory are available. Experimental results show that the proposed technique is able to track the resolver parameters and the accurate actuator position with a small error in real-time, while other adaptive estimators are unable to track the resolver parameters during and after resting periods
TL;DR: A battery control algorithm that can accommodate an arbitrary number of model parameters, with each model parameter having its own time-weighting factor, is derived and implemented and a method to determine optimal values for the time- Weighting factors is proposed.
Abstract: We derive and implement a battery control algorithm that can accommodate an arbitrary number of model parameters, with each model parameter having its own time-weighting factor, and we propose a method to determine optimal values for the time-weighting factors. Time-weighting factors are employed to give greater impact to recent data for the determination of a system’s state. We employ the (controls) methodology of weighted recursive least squares, and the time weighting corresponds to the exponential-forgetting formalism. The output from the adaptive algorithm is the battery state of charge (remaining energy), state of health (relative to the battery’s nominal performance), and predicted power capability. Results are presented for a high-power lithium ion battery.
TL;DR: Wavelet-based approaches for single-trial evoked potential estimation based on intracortical recordings outperform several existing methods including the Wiener filter, least mean square (LMS), and recursive least squares (RLS), and that the TI wavelet- based estimates have higher SNR and lower RMSE than the conventional wavelets.
22 Oct 2007
TL;DR: It is revealed that processing motion-corrupted PPG signals by least mean squares (LMS) and recursive least squares (RLS) algorithms can be effective to reduce SpO2 and HR errors during jogging, but the degree of improvement depends on filter order.
Abstract: Wearable physiological monitoring using a pulse oximeter would enable field medics to monitor multiple injuries simultaneously, thereby prioritizing medical intervention when resources are limited. However, a primary factor limiting the accuracy of pulse oximetry is poor signal-to-noise ratio since photoplethysmographic (PPG) signals, from which arterial oxygen saturation (SpO2) and heart rate (HR) measurements are derived, are compromised by movement artifacts. This study was undertaken to quantify SpO2 and HR errors induced by certain motion artifacts utilizing accelerometry-based adaptive noise cancellation (ANC). Since the fingers are generally more vulnerable to motion artifacts, measurements were performed using a custom forehead-mounted wearable pulse oximeter developed for real-time remote physiological monitoring and triage applications. This study revealed that processing motion-corrupted PPG signals by least mean squares (LMS) and recursive least squares (RLS) algorithms can be effective to reduce SpO2 and HR errors during jogging, but the degree of improvement depends on filter order. Although both algorithms produced similar improvements, implementing the adaptive LMS algorithm is advantageous since it requires significantly less operations.
TL;DR: This work proposes three ways of initializing, one that uses randomly generated data, a second that is ad-hoc and a third that uses an appropriate distribution, which provide a computing toolbox for analysing the quantitative properties of dynamic stochastic macroeconomic models under adaptive learning.
01 Jan 2007
TL;DR: In this article, the authors present six cases of non-convergence of the PLS path modeling algorithm, and these cases were estimated using Mode A combined with the factorial scheme or the path weighting scheme.
Abstract: This paper adds to an important aspect of Partial Least Squares (PLS) path modeling, namely the convergence of the iterative PLS path modeling algorithm. Whilst conventional wisdom says that PLS always converges in practice, there is no formal proof for path models with more than two blocks of manifest variables. This paper presents six cases of non-convergence of the PLS path modeling algorithm. These cases were estimated using Mode A combined with the factorial scheme or the path weighting scheme, which are two popular options of the algorithm. As a conclusion, efforts to come to a proof of convergence under these schemes can be abandoned, and users of PLS should triangulate their estimation results.
15 Apr 2007
TL;DR: This work proposes a convex combination of one LMS and one RLS filter that should profit of the best tracking behavior of each component filter.
Abstract: Recently, an adaptive convex combination of two LMS (least mean-square) filters was proposed and its tracking performance analyzed. Motivated by the performance of such scheme and by the differences between the tracking capabilities of the RLS (recursive least-squares) and LMS algorithms, we propose a convex combination of one LMS and one RLS filter. The resulting combination should profit of the best tracking behavior of each component filter. A steady-state analysis via energy conservation relation is also presented for stationary and non-stationary environments.
27 May 2007
TL;DR: The presented filter is designed to meet the constraints of channel equalization for fixed wireless communications that typically requires a large number of taps, but a serial updating of the filter coefficients, based on the least mean squares (LMS) algorithm, is allowed.
Abstract: In this paper a low-power implementation of an adaptive FIR filter is presented. The filter is designed to meet the constraints of channel equalization for fixed wireless communications that typically requires a large number of taps, but a serial updating of the filter coefficients, based on the least mean squares (LMS) algorithm, is allowed. Previous work showed that the use of the residue number system (RNS) for the variable FIR filter grants advantages both in area and power consumption. On the other hand, the use of a binary serial implementation of the adaptation algorithm eliminates the need for complex scaling circuits in RNS. The advantages in terms of area and speed of the presented filter, with respect to its two's complement counterpart, are evaluated for implementations in standard cells.
17 Jun 2007
TL;DR: A diffusion recursive least-squares algorithm where nodes need to communicate only with their closest neighbors, which has no topology constraints, and requires no transmission or inversion of matrices, therefore saving in communications and complexity.
Abstract: We consider the problem of distributed estimation in adaptive networks where a collection of nodes are required to estimate in a collaborative manner some parameter of interest from their measurements. The centralized solution to the problem uses a fusion center, thus requiring a large amount of energy for communication. We propose a diffusion recursive least-squares algorithm where nodes need to communicate only with their closest neighbors. The algorithm has no topology constraints, and requires no transmission or inversion of matrices, therefore saving in communications and complexity. We show that the algorithm is stable and analyze its performance comparing it to the centralized global solution.
TL;DR: In this article, a new adaptive recursive least squares (RLS) controller for HVAC systems is proposed, which can be described as a first order plus dead time model.
TL;DR: This paper develops a finite-data-window least squares algorithm with a forgetting factor for dynamical system modeling, derive its recursive version, and also gives its simplified form.
TL;DR: The paper gives the recursive algorithm of model parameters when adding a new data pair and deleting an existent one, respectively, and thus the inversion of a large matrix is avoided and the memory can be controlled by the algorithm entirely.
Patent•
16 May 2007TL;DR: In this paper, an apparatus having a detector for an iterative LDPC-coded MIMO-OFDM system, where the detector is configured to use a structured irregular LDPC code in conjunction with a belief propagation algorithm is presented.
Abstract: Disclosed is an apparatus having a detector for an iterative LDPC-coded MIMO-OFDM system, where the detector is configured to use a structured irregular LDPC code in conjunction with a belief propagation algorithm. Also disclosed is an apparatus having a detector for a structured irregular LDPC-coded MIMO-OFDM system, where the detector is configured to use an iterative Recursive Least Squares-based data detection and channel estimation technique. Corresponding methods and computer program products are also disclosed.
TL;DR: This paper discusses in detail a recently proposed kernel-based version of the recursive least-squares (RLS) algorithm for fast adaptive nonlinear filtering that combines a sliding-window approach with conventional ridge regression to improve generalization.
Abstract: In this paper we discuss in detail a recently proposed kernel-based version of the recursive least-squares (RLS) algorithm for fast adaptive nonlinear filtering. Unlike other previous approaches, the studied method combines a sliding-window approach (to fix the dimensions of the kernel matrix) with conventional ridge regression (to improve generalization). The resulting kernel RLS algorithm is applied to several nonlinear system identification problems. Experiments show that the proposed algorithm is able to operate in a time-varying environment and to adjust to abrupt changes in either the linear filter or the nonlinearity.
TL;DR: In this article, a linear, stochastic, univariate, forward looking model with one lag under adaptive heterogeneous learning is considered, where two different types of agents learn through recursive least squares techniques the parameter values in their forecasting models.
TL;DR: An analysis of the MMax algorithms for time-varying system identification is formulated by modeling the unknown system using a modified Markov process and results are derived for the tracking performance of MMax selective tap algorithms for normalized least mean square, recursive least squares, and affine projection algorithms.
Abstract: Selective-tap algorithms employing the MMax tap selection criterion were originally proposed for low-complexity adaptive filtering. The concept has recently been extended to multichannel adaptive filtering and applied to stereophonic acoustic echo cancellation. This paper first briefly reviews least mean square versions of MMax selective-tap adaptive filtering and then introduces new recursive least squares and affine projection MMax algorithms. We subsequently formulate an analysis of the MMax algorithms for time-varying system identification by modeling the unknown system using a modified Markov process. Analytical results are derived for the tracking performance of MMax selective tap algorithms for normalized least mean square, recursive least squares, and affine projection algorithms. Simulation results are shown to verify the analysis.
TL;DR: This paper gives a description of the Matlab package MILES, which can be used to solve an ordinary integer least squares problem alone and provides a guide for using it.
Abstract: In GNSS, for fixing integer ambiguities and estimating positions, a mixed integer least squares problem has to be solved. The Matlab package MILES provides fast and numerically reliable routines to solve this problem. In the process of solving a mixed integer least squares problem, an ordinary integer least squares problem is solved. Thus this package can also be used to solve an ordinary integer least squares problem alone. An option to compute multiple solutions is provided. This paper gives a description of this package and provides a guide for using it.