Showing papers on "Recursive least squares filter published in 2015"
TL;DR: Two recursive least squares parameter estimation algorithms are proposed by using the data filtering technique and the Auxiliary model identification idea to identify the parameters of the system models and the noise models interactively and can generate more accurate parameter estimates than the auxiliary model based recursion least squares algorithms.
137 citations
TL;DR: In this paper, the authors proposed a novel technique which employs a simplified model and multiple adaptive forgetting factors recursive least-squares (MAFF-RLS) estimation to provide capability to accurately capture the real-time variations and the different dynamics of the parameters whilst the simplicity in computation is still retained.
134 citations
TL;DR: An algorithm to learn optimal actions in convex distributed online problems is developed and shows that decisions made with this saddle point algorithm lead to regret whose order is not larger than $O(\sqrt{T})$ , where $T$ is the total operating time.
Abstract: An algorithm to learn optimal actions in convex distributed online problems is developed Learning is online because cost functions are revealed sequentially and distributed because they are revealed to agents of a network that can exchange information with neighboring nodes only Learning is measured in terms of the global network regret, which is defined here as the accumulated loss of causal prediction with respect to a centralized clairvoyant agent to which the information of all times and agents is revealed at the initial time A variant of the Arrow–Hurwicz saddle point algorithm is proposed to control the growth of global network regret This algorithm uses Lagrange multipliers to penalize the discrepancies between agents and leads to an implementation that relies on local operations and exchange of variables between neighbors We show that decisions made with this saddle point algorithm lead to regret whose order is not larger than $O(\sqrt{T})$ , where $T$ is the total operating time Numerical behavior is illustrated for the particular case of distributed recursive least squares An application to computer network security in which service providers cooperate to detect the signature of malicious users is developed to illustrate the practical value of the proposed algorithm
120 citations
05 Jun 2015
TL;DR: In this article, a simple method of coupling coefficient estimation with RLS (Recursive least squares) filter is proposed to improve the transmission efficiency in dynamic wireless power transfer system for electric vehicles.
Abstract: Maximum efficiency control using a DC/DC converter on the secondary side can improve transmitting efficiency in dynamic wireless power transfer system for electric vehicles. However, the information of coupling coefficient has to be estimated to implement the control although coupling coefficient changes dynamically. In this paper, a simple method of coupling coefficient estimation with RLS (Recursive least squares) filter is proposed. Dynamics of WPT system is analyzed with transfer functions. Moreover, the modeling and the control method of DC/DC converter are introduced. The experimental results of real-time coupling coefficient estimation and maximum efficiency control are provided and they indicate the effectiveness of the proposed control in a real dynamic wireless power transfer system for EVs.
100 citations
TL;DR: A new variable step-size diffusion least mean square algorithm for distributed estimation that adaptively adjusts the step- size in every iteration that achieves both fast convergence speed and low misadjustment by remarkable improvement in an adaptation stage is proposed.
Abstract: We propose a new variable step-size diffusion least mean square algorithm for distributed estimation that adaptively adjusts the step-size in every iteration. For a network application, the proposed method determines a suboptimal step-size at each node to minimize the mean square deviation for the intermediate estimate. The algorithm thus adapts the different node environments and profiles across the networks, and requires relatively less user interaction than existing algorithms. In experiments, the algorithm achieves both fast convergence speed and low misadjustment by remarkable improvement in an adaptation stage. We analyze the mean square performance of the proposed algorithm. Also, the proposed algorithm works well even in non-stationary environments.
98 citations
TL;DR: In this paper, a model-based condition monitoring strategy is developed for lithium-ion batteries on the basis of an electrical circuit model incorporating hysteresis effect, which systematically integrates 1) a fast upper-triangular and diagonal recursive least squares algorithm for parameter identification of the battery model, 2) a smooth variable structure filter for the SOC estimation, and 3) a recursive total least square algorithm for estimating the maximum capacity, which indicates the SOH.
76 citations
TL;DR: A new adaptive algorithm called block-sparse LMS (BS-LMS) is proposed, to insert a penalty of block-sparsity, which is a mixed l2, 0 norm of adaptive tap-weights with equal group partition sizes, into the cost function of traditional LMS algorithm.
Abstract: In order to improve the performance of least mean square (LMS)-based adaptive filtering for identifying block-sparse systems, a new adaptive algorithm called block-sparse LMS (BS-LMS) is proposed in this paper. The basis of the proposed algorithm is to insert a penalty of block-sparsity, which is a mixed l2, 0 norm of adaptive tap-weights with equal group partition sizes, into the cost function of traditional LMS algorithm. To describe a block-sparse system response, we first propose a Markov-Gaussian model, which can generate a kind of system responses of arbitrary average sparsity and arbitrary average block length using given parameters. Then we present theoretical expressions of the steady-state misadjustment and transient convergence behavior of BS-LMS with an appropriate group partition size for white Gaussian input data. Based on the above results, we theoretically demonstrate that BS-LMS has much better convergence behavior than l0-LMS with the same small level of misadjustment. Finally, numerical experiments verify that all of the theoretical analysis agrees well with simulation results in a large range of parameters.
72 citations
Posted Content•
TL;DR: In this article, the authors consider a linear time-variant system that is corrupted with process and measurement noise, and study how the selection of its sensors affects the estimation error of the corresponding Kalman filter over a finite observation interval.
Abstract: In this paper, we focus on sensor placement in linear dynamic estimation, where the objective is to place a small number of sensors in a system of interdependent states so to design an estimator with a desired estimation performance. In particular, we consider a linear time-variant system that is corrupted with process and measurement noise, and study how the selection of its sensors affects the estimation error of the corresponding Kalman filter over a finite observation interval. Our contributions are threefold: First, we prove that the minimum mean square error of the Kalman filter decreases only linearly as the number of sensors increases. That is, adding extra sensors so to reduce this estimation error is ineffective, a fundamental design limit. Similarly, we prove that the number of sensors grows linearly with the system's size for fixed minimum mean square error and number of output measurements over an observation interval; this is another fundamental limit, especially for systems where the system's size is large. Second, we prove that the logdet of the error covariance of the Kalman filter, which captures the volume of the corresponding confidence ellipsoid, with respect to the system's initial condition and process noise is a supermodular and non-increasing set function in the choice of the sensor set. Therefore, it exhibits the diminishing returns property. Third, we provide efficient approximation algorithms that select a small number sensors so to optimize the Kalman filter with respect to this estimation error ---the worst-case performance guarantees of these algorithms are provided as well. Finally, we illustrate the efficiency of our algorithms using the problem of surface-based monitoring of CO2 sequestration sites studied in Weimer et al. (2008).
71 citations
TL;DR: A distributed adaptive algorithm to solve a node-specific parameter estimation problem where the nodes are interested in estimating parameters that can be of local interest, common interest to a subset of nodes and global interest to the whole network is proposed.
Abstract: A distributed adaptive algorithm is proposed to solve a node-specific parameter estimation problem where the nodes are interested in estimating parameters that can be of local interest, common interest to a subset of nodes and global interest to the whole network. To address the different node-specific parameter estimation problems, this novel algorithm relies on a diffusion-based implementation of different, yet coupled Least Mean Squares (LMS) algorithms, each associated with the estimation of a specific set of local, common or global parameters. The study of convergence in the mean sense reveals that the proposed algorithm is asymptotically unbiased. Moreover, a spatial-temporal energy conservation relation is provided to evaluate the steady-state performance at each node in the mean-square sense. Finally, the theoretical results and the effectiveness of the proposed technique are validated through computer simulations in the context of cooperative spectrum sensing in Cognitive Radio networks.
71 citations
TL;DR: The proposed shrinkage linear complex-valued least mean squares and SWL-CLMS algorithms are devised for adaptive beamforming and are more computationally efficient than the RLS solutions though they may have a slightly slower convergence rate.
Abstract: In this paper, shrinkage linear complex-valued least mean squares (SL-CLMS) and shrinkage widely linear complex-valued least mean squares (SWL-CLMS) algorithms are devised for adaptive beamforming. By exploiting the relationship between the noise-free a posteriori and a priori error signals, the SL-CLMS method is able to provide a variable step size to update the weight vector for the adaptive beamformer, significantly enhancing the convergence speed and decreasing the steady-state misadjustment. On the other hand, besides adopting a variable step size determined by minimizing the square of the augmented noise-free a posteriori errors, the SWL-CLMS approach exploits the noncircular properties of the signal of interest, which considerably improves the steady-state performance. Simulation results are presented to illustrate their superiority over the CLMS, complex-valued normalized LMS, variable step size, recursive least squares (RLS) algorithms and their corresponding widely linear-based schemes. Additionally, our proposed algorithms are more computationally efficient than the RLS solutions though they may have a slightly slower convergence rate.
69 citations
TL;DR: The key features of the proposed adaptive link selection algorithms for distributed estimation are more accurate estimates and faster convergence speed and the network is equipped with the ability of link selection that can circumvent link failures and improve the estimation performance.
Abstract: This paper presents adaptive link selection algorithms for distributed estimation and considers their application to wireless sensor networks and smart grids. In particular, exhaustive search-based least mean squares (LMS) / recursive least squares (RLS) link selection algorithms and sparsity-inspired LMS / RLS link selection algorithms that can exploit the topology of networks with poor-quality links are considered. The proposed link selection algorithms are then analyzed in terms of their stability, steady-state, and tracking performance and computational complexity. In comparison with the existing centralized or distributed estimation strategies, the key features of the proposed algorithms are as follows: (1) more accurate estimates and faster convergence speed can be obtained and (2) the network is equipped with the ability of link selection that can circumvent link failures and improve the estimation performance. The performance of the proposed algorithms for distributed estimation is illustrated via simulations in applications of wireless sensor networks and smart grids.
TL;DR: In this article, a comparison of the Kalman filter and the recursive least squares method for the estimation of the grid impedance is presented, and the results are validated by a hardware in the loop and an experimental setup.
Abstract: The design of microgrids at the level of distribution systems requires a stable behavior for multiple operation states. The tools used to study the stability of such systems require the estimation of the grid impedance. By the use of the grid impedance estimation around an operation point, it is possible to define a space variable-parameter to obtain a qualitative or quantitative measure from the operation to the unstable boundary. This study presents a comparison of the Kalman filter and the recursive least squares method for the estimation of the grid impedance. The grid impedance is estimated by the technique based on two neighbor operation points. The results were validated by a hardware in the loop and an experimental setup. Finally, the estimated values of the grid impedance of a microgrid are used with a large signal stability study of a dc constant power load.
TL;DR: A novel algorithm for dimensionality reduction (spatial filter) that is ideally suited for single-trial detection of event-related potentials (ERPs) and can be adapted online to a new subject to minimize or avoid calibration time.
Abstract: Goal: Current brain–computer interfaces (BCIs) are usually based on various, often supervised, signal processing methods. The disadvantage of supervised methods is the requirement to calibrate them with recently acquired subject-specific training data. Here, we present a novel algorithm for dimensionality reduction (spatial filter), that is ideally suited for single-trial detection of event-related potentials (ERPs) and can be adapted online to a new subject to minimize or avoid calibration time. Methods: The algorithm is based on the well-known xDAWN filter, but uses generalized eigendecomposition to allow an incremental training by recursive least squares (RLS) updates of the filter coefficients. We analyze the effectiveness of the spatial filter in different transfer scenarios and combinations with adaptive classifiers. Results: The results show that it can compensate changes due to switching between different users, and therefore allows to reuse training data that has been previously recorded from other subjects. Conclusions: The presented approach allows to reduce or completely avoid a calibration phase and to instantly use the BCI system with only a minor decrease of performance. Significance: The novel filter can adapt a precomputed spatial filter to a new subject and make a BCI system user independent.
TL;DR: In the presence of frequency selective crosstalk, an extension of CTC-DPD which can pre-cancel such cros stalk, performs considerably better than CO-D PD, which cannot model the frequency selectivity of the crosStalk.
Abstract: This paper presents a comparative study of adaptive algorithms for digital predistortion (DPD) in multiple antenna transmitters. Crossover predistorter (CO-DPD) and crosstalk canceling predistorter (CTC-DPD) were proposed to overcome the deleterious effect of RF crosstalk before the power amplifiers (PA) on digital predistortion (DPD) in multiple antenna transmitters. This paper discusses the linearization performance and computational complexity of least mean square (LMS) and recursive least squares (RLS) adaptive algorithms for CO-DPD and CTC-DPD. The adaptive predistortion algorithms for a single antenna transmitter can be extended for CO-DPD, by incorporating the DPD coefficients of more than one branch in to one filter coefficient vector of the adaptive algorithm. The adaptive CTC-DPD involves predistorters for each transmitter running in parallel with the adaptive algorithms that track the coupling between the transmitters. The computational complexity of CTC-DPD is considerably lower compared to CO-DPD, as it has lesser predistorter branches. It is estimated that the number of computations needed per sample duration, for the real-time operation of the adaptive CTC-DPD is 47% lesser compared to CO-DPD for a two-antenna transmitter, when a 9th order memory polynomial with 3 memory taps was used. The linearization performances of these adaptive predistortion techniques for two and four antenna transmitters are evaluated through simulations using QPSK, 16-QAM, UMTS, and LTE signals. It is observed that CTC-DPD performs approximately identical to CO-DPD in all the examined cases. In the presence of frequency selective crosstalk, an extension of CTC-DPD which can pre-cancel such crosstalk, performs considerably better than CO-DPD, which cannot model the frequency selectivity of the crosstalk.
TL;DR: The proposed normalized least mean square algorithm is characterized by robustness against noisy input signals, and has a fast convergence rate when applied to sparse systems, owing to its L0-norm cost in the proposed update equation.
Abstract: This brief proposes a novel normalized least mean square algorithm that is characterized by robustness against noisy input signals. To compensate for the bias caused by the input noise that is added at the filter input, a derivation method based on reasonable assumptions finds a bias-compensating vector. Moreover, the proposed algorithm has a fast convergence rate when applied to sparse systems, owing to its ${\cal L}_{0} $ -norm cost in the proposed update equation. The simulation results verify that the proposed algorithm improves the performance of the filter, in terms of system identification in sparse systems, in the presence of noisy input signals.
TL;DR: An unscented Kalman filter-based constant modulus adaptation algorithm (UKF-CMA) is proposed for blind uniform linear beamforming that does not require a priori information about the process noise and measurement noise covariance matrices and hence it can be applied readily.
Abstract: An unscented Kalman filter-based constant modulus adaptation algorithm (UKF-CMA) is proposed for blind uniform linear beamforming. The proposed algorithm is obtained by first developing a model of the constant modulus (CM) criterion and then fitting that model into the Kalman filter-style state space model by using an auxiliary parameter. The proposed algorithm does not require a priori information about the process noise and measurement noise covariance matrices and hence it can be applied readily. Simulation results demonstrate that the proposed algorithm offers improved performance compared to the recursive least square-based CM (RLS-CMA) and least-mean square-based CM (LMS-CMA) algorithms for adaptive blind beamforming.
TL;DR: In this article, a recursive least square structure is proposed to minimize the weighted summation of the logarithmic transformation of posterior errors and taking the commutation error into consideration, which not only reduces broadband, narrowband and impulse noise successfully, but also mixtures of them.
TL;DR: A new diffusion least mean squares algorithm that utilizes adaptive gains in the adaptation stage for the sparse distributed estimation problem is proposed and the mean stability analysis is provided to establish sufficient condition for the algorithm to converge in the mean sense.
Abstract: We propose a new diffusion least mean squares algorithm that utilizes adaptive gains in the adaptation stage for the sparse distributed estimation problem. We derive the optimal gains that attain a minimum mean-square deviation and propose an adaptive gain control method. We provide the mean stability analysis to establish sufficient condition for the algorithm to converge in the mean sense. The algorithm achieves higher convergence speed than the sparsity-constrained algorithms, regardless of the sparsity of the vector of interest.
TL;DR: In this article, an extended two-step estimation approach was proposed to estimate the mean value of the battery state of charge in the form of a probability density function (PDF) and the intrinsic variations in cell SOC and resistance were identified simultaneously in an extended framework using recursive least squares (RLS) algorithm.
TL;DR: In this article, an overview of disturbance-observer-based adaptive vibration rejection schemes is provided, as well as several new results in algorithm design and new applications in semiconductor manufacturing.
Abstract: Summary
Vibrations with unknown and/or time-varying frequencies significantly affect the achievable performance of control systems, particularly in precision engineering and manufacturing applications. This paper provides an overview of disturbance-observer-based adaptive vibration rejection schemes; studies several new results in algorithm design; and discusses new applications in semiconductor manufacturing. We show the construction of inverse-model-based controller parameterization and discuss its benefits in decoupled design, algorithm tuning, and parameter adaptation. Also studied are the formulation of recursive least squares and output-error-based adaptation algorithms, as well as their corresponding scopes of applications. Experiments on a wafer scanner testbed in semiconductor manufacturing prove the effectiveness of the algorithm in high-precision motion control. Copyright © 2015 John Wiley & Sons, Ltd.
TL;DR: A new convergence analysis is presented for a well-known sparse adaptive filter family, namely, the proportionate-type normalized least mean square (PtNLMS) algorithms, where, unlike all the existing approaches, no assumption of whiteness is made on the input.
Abstract: In this paper, a new convergence analysis is presented for a well-known sparse adaptive filter family, namely, the proportionate-type normalized least mean square (PtNLMS) algorithms, where, unlike all the existing approaches, no assumption of whiteness is made on the input. The analysis relies on a “transform” domain based model of the PtNLMS algorithms and brings out certain new convergence features not reported earlier. In particular, it establishes the universality of the steady-state excess mean square error formula derived earlier under white input assumption. In addition, it brings out a new relation between the mean square deviation of each tap weight and the corresponding gain factor used in the PtNLMS algorithm.
TL;DR: The identification procedure uses the dew-point temperature as the instrumental variable for the exogenous variable (dry-bulb temperature), to better characterize the relationship between exogenous and endogenous variables, and TSRLS helps to reduce the space and time complexity.
Abstract: Operation strategy of combined cooling, heating, and power (CCHP) systems is designed to collect users’ load information to determine the energy input to the system and power flow inside the system. Most of the current operation strategies are designed by assuming that accurate loads during the next time interval are already known. To solve the problem of unknown loads in practical applications, using an autoregressive moving average with exogenous inputs model, whose parameters are identified by an ordinary least squares–two-stage recursive least squares (TSRLS) algorithm, cooling, heating, and electrical loads in the future time intervals are forecasted. The identification procedure uses the dew-point temperature as the instrumental variable for the exogenous variable (dry-bulb temperature), to better characterize the relationship between exogenous and endogenous variables. TSRLS helps to reduce the space and time complexity. A poststrategy is also proposed to compensate for the inaccurate forecasting. A case study is conducted to verify the feasibility and effectiveness of the proposed methods.
TL;DR: The resulting novel adaptive LSFEM with separate marking converges with optimal rates relative to the notion of a nonlinear approximation class.
Abstract: The first-order div least squares finite element methods (LSFEMs) allow for an immediate a posteriori error control by the computable residual of the least squares functional. This paper establishes an adaptive refinement strategy based on some equivalent refinement indicators. Since the first-order div LSFEM measures the flux errors in $H$(div), the data resolution error measures the $L^2$ norm of the right-hand side $f$ minus the piecewise polynomial approximation $\Pi f$ without a mesh-size factor. Hence the data resolution term is neither an oscillation nor of higher order and consequently requires a particular treatment, e.g., by the thresholding second algorithm due to Binev and DeVore. The resulting novel adaptive LSFEM with separate marking converges with optimal rates relative to the notion of a nonlinear approximation class.
TL;DR: This paper considers the parametric identification problems of a Hammerstein nonlinear system which consists of a static nonlinear block followed by a linear dynamic subsystem and a hierarchical least squares algorithm is developed by using the hierarchical identification principle.
Abstract: This paper considers the parametric identification problems of a Hammerstein nonlinear system which consists of a static nonlinear block followed by a linear dynamic subsystem. A hierarchical least squares algorithm is developed by using the hierarchical identification principle, which decomposes a nonlinear system into several subsystems with smaller dimensions and fewer variables and estimates the parameters of each subsystem, respectively. The performance analysis indicates that the parameter estimates given by the proposed algorithm converge to their true values and the proposed algorithm requires higher computational efficiencies compared with the recursive least squares algorithm.
TL;DR: In this paper, an incremental analysis-based auto regressive exogenous (I-ARX) modeling method is proposed to eliminate the modeling error caused by the OCV effect and improve the accuracy of parameter estimation.
TL;DR: The monotonic property of the fuzzy NARX model is first proved, and then, their antecedent parameters can be determined by this property, and the proposed type-2 fuzzy control scheme can realize the control objectives and achieve a good control performance.
Abstract: This paper is concerned with the problem of type-2 fuzzy adaptive inverse control for a cable-driven parallel system. Based on the heuristics and prior knowledge of the system, the system is divided into six subsystems. The proposed control scheme for each subsystem contains a forward model and a fuzzy adaptive inverse controller (FAIC), which are expressed by an interval type-2 fuzzy nonlinear autoregressive exogenous (NARX) model, respectively. To construct the antecedents of the interval type-2 fuzzy NARX forward models and FAICs, the monotonic property of the fuzzy NARX model is first proved, and then, their antecedent parameters can be determined by this property. Furthermore, the consequent parameters of the forward models are computed offline via a constrained least squares algorithm, and the consequent parameters of the FAIC are adjusted online via a recursive least squares algorithm. Experiment results are provided to show that the proposed type-2 fuzzy control scheme can realize the control objectives and achieve a good control performance.
15 Jun 2015
TL;DR: The adaptive filter algorithm, RLS has been used in cancellation of various noises in ECG signals and simulation results depict that RLS algorithm renders a much better performance in removing noises from the ECG signal than LMS algorithm.
Abstract: Electrocardiogram (ECG) is a diagnostic procedure that measures and records the electrical activity of heart in detail. By reviewing an ECG report, one's condition of heart can be evaluated. But ECG signals are often affected and altered by the presence of various noises that degrade the accuracy of an ECG signal and thus misrepresents the recorded data. To filter out these noises conventional digital filters have been used for decades. Yet noise cancellation with finite and determined coefficients has often been unsuccessful due to the non-stationary nature of ECG signal. Adaptive filters adapt their filter coefficients with the continuous change of signal using adaptive algorithms, providing the optimum noise removal features for non-stationary signals like ECG. In this study, the adaptive filter algorithm, RLS has been used in cancellation of various noises in ECG signals. We have also performed noise removal using LMS adaptive filter algorithm to compare the performance of RLS algorithm. We have used MATLAB® to simulate different noise signals and process the noises. The ECG signals used here have been taken from the PhysioNet ECG-ID database. The simulation results depict that RLS algorithm renders a much better performance in removing noises from the ECG signals than LMS algorithm.
TL;DR: This paper addresses issues in online parameter estimation of a linear tooth belt drive with a limited stroke and demonstrates the feasibility of the estimation method to estimate the parameters of a closed-loop controlled servomechanism with alimited stroke and time-varying parameters.
Abstract: Many control schemes rely on an analytical model of the servomechanism to be controlled, and hence, accurate knowledge about the position- and time-dependent parameter variability becomes crucial in many contexts, such as robust control methods. Although a properly designed robust controller can cope with a large parameter variation, real-time identification of the system parameter behavior could lead to several advantages by means of monitoring the varying dynamics, e.g., the predetermined uncertainty region around the nominal value. This paper addresses issues in online parameter estimation of a linear tooth belt drive with a limited stroke. Particular attention is paid to detecting the position-dependent changes in the system dynamics by using recursive least squares algorithm and exciting the system in different cart positions in order to identify the varying dynamics. The algorithm used is based on an indirect output-error identification scheme. The experimentally estimated parameters are compared with the corresponding two-mass model parameters. The results show an acceptable agreement and demonstrate the feasibility of the estimation method to estimate the parameters of a closed-loop controlled servomechanism with a limited stroke and time-varying parameters.
TL;DR: Simulations in the context of time-series prediction and nonlinear regression show that SF-KLMS outperforms not only the kernel adaptive filter with multiple feedback but also the Kernel adaptive filter without feedback in terms of convergence rate and mean square error.
Abstract: In this letter, a novel kernel adaptive filtering algorithm, namely the kernel least mean square with single feedback (SF-KLMS) algorithm, is proposed. In SF-KLMS, only a single delayed output is used to update the weights in a recurrent fashion. The use of past information accelerates the convergence rate significantly. Compared with the kernel adaptive filter using multiple feedback, SF-KLMS has a more compact and efficient structure. Simulations in the context of time-series prediction and nonlinear regression show that SF-KLMS outperforms not only the kernel adaptive filter with multiple feedback but also the kernel adaptive filter without feedback in terms of convergence rate and mean square error.
TL;DR: A system identification framework is considered within which a joint perspective on Kalman filtering and LMS-type algorithms is developed, achieved through analyzing the degrees of freedom necessary for optimal stochastic gradient descent adaptation.
Abstract: The Kalman filter and the least mean square (LMS) adaptive filter are two of the most popular adaptive estimation algorithms that are often used interchangeably in a number of statistical signal processing applications. They are typically treated as separate entities, with the former as a realization of the optimal Bayesian estimator and the latter as a recursive solution to the optimal Wiener filtering problem. In this lecture note, we consider a system identification framework within which we develop a joint perspective on Kalman filtering and LMS-type algorithms, achieved through analyzing the degrees of freedom necessary for optimal stochastic gradient descent adaptation. This approach permits the introduction of Kalman filters without any notion of Bayesian statistics, which may be beneficial for many communities that do not rely on Bayesian methods [1], [2].