Showing papers on "Recursive least squares filter published in 1980"

Stochastic parameter estimation procedures for hydrologie rainfall‐runoff models: Correlated and heteroscedastic error cases

Abstract: A maximum likelihood estimation procedure is presented through which two aspects of the streamflow measurement errors of the calibration phase are accounted for. First, the correlated error case is considered where a first-order autoregressive scheme is presupposed for the additive errors. This proposed procedure first determines the anticipated correlation coefficient of the errors and then uses it in the objective function to estimate the best values of the model parameters. Second, the heteroscedastic error case (changing variance) is considered for which a weighting approach, using the concept of power transformation, is developed. The performances of the new procedures are tested with synthetic data for various error conditions on a two-parameter model. In comparison with the simple least squares criterion and the weighted least squares scheme of the HEC-1 of the U.S. Army Corps of Engineers for the heteroschedastic case, the new procedures constantly produced better estimates. The procedures were found to be easy to implement with no convergence problem. In the absence of correlated errors, as theoretically expected, the correlated error procedure produces the exact same estimates as the simple least squares criterion. Likewise, the self-correcting ability of the heteroschedastic error procedure was effective in reducing the objective function to that of the simple least squares as data gradually became homoscedastic. Finally, the effective residual tests for detection of the above-mentioned error situations are discussed.

Fast implementations of LMS adaptive filters

Abstract: A frequency domain implementation of the LMS adaptive transversal filter is proposed. This fast LMS (FLMS) adaptive filter requires less computation than the conventional LMS adaptive filter when the filter length equals or exceeds 64 sample points.

Least squares with a quadratic constraint

Abstract: We present the theory of the linear least squares problem with a quadratic constraint. New theorems characterizing properties of the solutions are given. A numerical application is discussed.

Analysis of stochastic approximation schemes with discontinuous and dependent forcing terms with applications to data communication algorithms

Abstract: A general convergence result is given for stochastic approximation schemes with (or without) equality constraints. The following features are taken into account. The forcing term is a strongly dependent sequence and may be discontinuous. Many examples are given to illustrate the applicability of the convergence theorem, both classical (recursive least squares scheme) and nonclassical ones (arising in the theory of self-adaptive eqnalizers).

On a realization and related algorithm for adaptive prediction

Abstract: The purpose of this paper is to introduce a novel Gram-Schmidt orthogonalization predictor realization, and also to present an adaptive algorithm to update its coefficients (weights). Experimental results pertaining to this algorithm are included, along with corresponding results obtained via some existing adaptive predictor algorithms.

Learning characteristics of adaptive lattice filtering algorithms

Abstract: This paper describes the performance and learning characteristics of the continuously adaptive lattice form for prediction-error filtering. Quantitative relationships are developed for convergence behavior, and procedures are described for selection of the adaptive weighting constant and filter order. Burg's algorithm is used to calculate the reflection coefficients of the filter. Based on this algorithm, two recursive relationships are developed to calculate the coefficients iteratively, one form assuming a stationary input signal, and a more complex form not making this assumption. A quantitative exposition of the convergence behavior in terms of an adaptive weighting constant is set down for these relationships for the first-order filter. Careful attention is given to the decoupling of higher filter orders, leading to the creation of a decoupling constant for the stationary signal case. Higher order convergence and the factors affecting it are examined, resulting in a procedure for choosing the adaptive weighting constant based on the input signal characteristics. Properties of the filter in the spectral domain are also examined. This leads to selection criteria for choosing the filter order, based on the signal characteristics. Application of the filter to the problem of radar clutter discrimination is presented and discussed.

An adaptive d-step ahead predictor based on least squares

Abstract: This paper examines the asymptotic properties of a least squares algorithm for adaptively calculating a d -step ahead prediction of a time series. It is shown that, with probability one, the sample mean-square difference between time recursive prediction and the optimal linear prediction converges to zero. Relatively weak assumptions are required regarding the underlying model of the time series.

The recursive adaptive LMS filter--A line enhancer application and analytical model for the mean weight behavior

Abstract: A mathematical model is presented for the recursive adaptive filter (RAF) configured as an adaptive line enhancer (ALE) in the frequency domain. The inputs for the model are Markovian and the number of recursive taps is selected to equal the order of the Markov process. Thus, the RAF structure is sufficient to realize the Wiener filter. Assuming that the expectations of all filter-data interactions factor, a system of four deterministic equations for the mean weights is derived. In steady state, the mean weights converge to the Wiener filter, and hence minimize the mean-square error. Excellent agreement between this analysis and stochastic simulations support the expectation-splitting assumptions.

Identification and control of a laboratory model turbogenerator

Abstract: This paper describes the application of a recursive least squares algorithm to identify a laboratory turbogenerator system. The algorithm was initially tested in a computer simulation, which indicated that identified low-order models can represent the small-signal dynamics of a non-linear system more accurately than the corresponding linearized analytical models. This was confirmed by laboratory tests, which clearly showed the difficulty of representing the system by analytical models, and the substantial improvement obtained by identification The availability of accurate low-order linear models is particularly important in the design of controllers, to facilitate the design procedures and reduce the complexity of the system. Controllers based on identified models of the plant were designed and tested over a wide range of operating conditions, and found to give very good performance.

Automatic equalizer for synthesizing recursive filters

Abstract: A recursive automatic equalizer is described for implementing the telephone equalization function at a line circuit which may be multiplexed between a plurality of telephone subscriber sets. A recursive digital filter structure having programmable coefficients minimizes the error between the equalizer input and a reference signal. The recursive filter transfer function is variable via feedback coefficient update, with respect to its input and the reference signal. The recursive filter coefficients are adaptively changed to rapidly converge to a final value based upon a mean square error algorithm. The desired filter transfer function can be achieved with a low number of coefficients, for example, five, rather than the heretofore high number of coefficients required in non-recursive filter structures.

The Kalman Filter as an On-Line Drift Compensator in Multicomponent Analysis Determinations

Abstract: A method is proposed for the on-line compensation of linear drift in analytical measurements. The robustness of the compensator, which is based on the Kalman filter, has been investigated by means of a parametric study. Special emphasis is given to the choice of the initial error covariance matrix, which is an important design quantity for the Kalman filter. The presented method can be applied when there is uncertainty about the presence of drift in the measurements. Some results are compared with results obtained with the non-recursive least squares method. The excellent performance of the on-line compensator suggests a possible solution for the linear drift problem. Of some theoretical importance is the fact that the proposed solution has a strong intuitive appeal: the linear drift can be considered as “two extra components” in a spectrum or equivalently as the time varying mean of the measurement noise.

Computing Expected Mean Squares

Abstract: SUMMARY The evaluation of the expected mean squares arising from the analysis of unbalanced mixed ANOVA models has long been an analytically intractable problem. Irl this paper we present a theoretically and computationally simple technique for computation of the coefficients in the expected mean squares.

A fundamental approach to the convergence analysis of least squares algorithms

Abstract: The literature dealing with the question of convergence of the least squares (L.S.) identification algorithm [1-10] is usually utilizing the properties of the sequential estimator, e.g. the fact that the sequence of estimates is a matringale process, if the noise is an independent sequence, has been used to establish convergence in [10]. In this paper emphasis is put on the fact that the least squares estimates are obtained by minimizing a (quadratic) cost functional. Convergence results for sequence of random variables obtained by minimizing a parameterized random sequence with respect to its parameter are presented. These results in turn are utilized to establish strong convergence (w.p.l and m.s.) of the L.S. procedure under milder conditions than those in previous proofs. Landau's recursive algorithms [12-14] are shown to be variations of the L.S. and thus their convergence is also established. The self tuning regulator [19-22] is also discussed and the importance of the use of the L.S. procedure in it is demonstrated. The importance of this paper, beyond extending previous convergence results is in its approach - utilizing the foundation on which L.S. procedures are based.

Event location using recursive least squares signal processing

Abstract: The problem of locating the position of individual pulses within a group of overlapping pulses can be simplified by preprocessing the data to reduce the overlap. This paper proposes the use of Recursive Least Squares (RLS) prediction for this purpose. The pulse compression performance of two signals derived from the RLS algorithm is compared using all-pole data with different noise levels, and the effects of zeroes in the data and prefiltering are discussed.

Recursive lattice filters - A brief overview

Abstract: Recently lattice filters structures have been employed in numerous adaptive filtering applications such as noise cancelling; speech processing; and data equalization. In this paper we will be concerned with the algorithms that have been proposed to update the lattice filter coefficients. These algorithms typically fall into one of two classes, those based on stochastic (gradient) formulations and those founded on a least squares criterion. The latter are more complex; however, they provide for a faster response to sudden changes in the input data (e.g., a rapid initial convergence). It is the purpose of this paper to provide a brief exposition of the algorithms and to point out their various parallels. In particular, it is hoped that the simpler structure of the stochastic algorithms will shed some light on the more complex least squares procedures.

A Least Squares Adaptive Lattice Equalizer Algorithm.

Abstract: : In many applications of adaptive data equalization, rapid initial convergence of the adaptive equalizer is of paramount importance. Apparently the fastest known equalizer adaptation algorithm is based on a least squares estimation algorithm. The least squares algorithm, which is a special case of the Kalman estimation algorithm, was first applied to channel equalization by Godard in a seminal paper. One disadvantage with the Godard algorithm is that the complexity, i.e., number of additions and multiplications, of the algorithm grows quadradically with the number of filter coefficients. Recently, however, Morf, Ljung, Lee and others have shown how the complexity of the conventionally implemented least squares algorithms (e.g., Godard's algorithm) can be made to grow only linearly with the number of filter coefficients. Furthermore, these computationally simpler least squares algorithms may be implemented either in tapped delay line or lattice form. The application of the tapped delay line form, i.e., the fast Kalman algorithm, to channel equalization has been considered recently by Falconer and Ljung. In this paper, it is shown how the least squares lattice algorithms originally introduced by Morf and Lee can be adapted to the equalizer adjustment algorithm. The extremely rapid start up convergence properties of the least squares lattice equalizer are confirmed by computer simulation. (Author)

The estimation of parameters by least squares from unbalanced experiments.

Abstract: Summary Data from experimental and observational studies, in agriculture as in many other contexts, are often unbalanced in respect of important classifications. Treatments may be unequally replicated, some combinations of factors may be omitted, animals may die for reasons unconnected with an experiment. Unless means are adjusted in some manner that eliminates disturbance from unequal representation of different categories, comparisons between treatments may be thoroughly misleading. Optimal procedures for the simpler situations have been familiar to statisticians for a long time. They have been little used by other scientists analysing their own data, in part because of the computational labour and in part because their nature has not been properly understood. Modern computing power removes all excuse for the retention of methods that may be actively misleading because they bias summaries of data, or that are at best inefficient in their failure to estimate comparisons as precisely as is possible. The only remaining barrier is the mistaken belief that good methods are either so complicated that they can be comprehended only by professional statisticians or so devious that truth is destroyed rather than exhibited. This paper attempts to show good methods as inherently rational, to explain their main properties, and to illustrate them on examples small enough for the processes to be clear. The paper neither expounds statistical theory nor instructs in computing practice.

An Implementation of the Multidimensional Modified LMS Algorithm

Abstract: An implementation of the multidimensional modified LMS algorithm is provided from its relationship to a recently developed class of lhyperstable adaptive filters.

Study of a particular adaptive parameter estimator applied to problems in prediction filtering and control

Abstract: This paper compares and contrasts the characteristics of the least squares parameter estimation algorithm applied to problems in filtering, prediction and control. We shall discuss the underlying assumptions, the specific form that the algorithm takes and comment on the similarities and differences in the analysis of convergence.

Using the least squares estimator in Chebyshev estimation

Abstract: This note investigates the advantage of using the least squares estimator as a starting point in Barrodale and Phillip's L -algorithm

On the use of Ljung's results for studying the convergence properties of Hampton's adaptive filter

Abstract: Recursive stochastic algorithms such as stochastic approximation algorithms introduce problems of convergence in probability. Until now, methods of convergence analysis remained very specific. Recently, Ljung proposed a general method which consists of associating with the stochastic scheme an ordinary differential equation that contains all the information about the asymptotic behavior of the algorithm. These results are used here to study the convergence properties of an adaptive Kalman filter due to Hampton. Some remarks about results obtained by file Ljung approach are introduced and a comparison between these results and the Hampton ones is made.

On the convergence properties of the simple hyperstable adaptive recursive filter (SHARF)

Abstract: In previous work the authors have developed a class of provably convergent adaptive algorithms for digital IIR filters based on the concept of hyperstability. While this class of adaptive filters offers much promise in practical applications little has been done toward providing the characterization that would allow a designer to select filter order, adaptation constants, and other design parameters. This paper reports on the initial effort toward providing this information through an investigation of the local convergence behavior of SHARF, the simple hyperstable adaptive recursive filter.

Miniature CCD-based analog adaptive filters

Abstract: The paper describes the design and operation of a 256-point adaptive filter, based on a monolithic 256-point charge-coupled device programmable transversal filter. Results are presented showing the characteristic performance obtained with the prototype system.

An algorithm for solving the restricted least squares problem

Abstract: This paper presents an algorithm to solve the least squares problem when the parameters are restricted to be non-negative. The algorithm is based on the branch and bound method which has been suggested for this problem, and shares with it the property that an unrestricted least squares subproblem is solved at each step. However, improvements have been made to the branching rules by making use of the convexity of the problem, and the Kuhn–Tucker conditions are used to test for optimality. The resulting algorithm becomes essentially iterative in nature, and linearity of the number of subproblems solved can be shown under assumptions which have always been observed in practice.

Adaptive algorithms for non-stastical parameter estimation in linear models

Abstract: A non-statistical approach to parameter estimation is presented. Assuming bounded noise, two algorithms are developed to obtain membership sets in the parameter space which are consistent with the set of measurements. The set theoretic approach enables the evaluation of information pertaining to the estimation problem as a new measurement is obtained. The proposed algorithms are shown to converge at least as fast as the least squares procedure. This performance is obtained while only about 10% of the data is actually used in the identification. The proposed algorithms are thus very attractive in the context of speach analysis where the assumption of bounded noise is easy to justify.

Covariance sequence approximation for recursive digital filter design

Abstract: In this paper we explore an alternative solution to the general nonlinear least squares approximation problem for digital filter design. We choose to approximate covariance sequences because they arise more naturally than unit pulse sequences when approximating spectra and when identifying ARMA sequences from random data. The design procedure resulting from this approach requires solution of a nonlinear algebraic equation in the form of a polynomial rootfinding problem, in addition to solving a system of linear equations and an Inverse Discrete Fourier Transform (IDFT).

Recursive Least Squares Estimation of Cell Kinetic Parameters Using Sequential Measurements of Cell DNA Contents

Abstract: The proliferation dynamics of a growing cell population can be represented by a discrete-time state model. An application of recursive least squares algorithms to the estimation of important cell cycle kinetic parameters is considered. Cell kinetic parameters are estimated recursively by the following steps: 1) decomposition of state and output spaces, 2) separation of identification of the unperturbed cell system from that of the perturbed cell system, and 3) application of a recursive least squares algorithm for the identification of each decomposed system. This method is feasible due to the availability of a new technology called flow microfluorometry (FMF) which is capable of providing large amounts of quantitative data within a short time period. Emphasis is placed on the construction of a computationally efficient and stable algorithm. The FMF deoxyribonucleic acid (DNA) data of a Chinese hamster ovary (CHO) cell population is used to demonstrate the potential value of the method developed.