Showing papers on "Non-linear least squares published in 2011"

Brief paper: Lleast squares-based recursive and iterative estimation for output error moving average systems using data filtering

Abstract: For parameter estimation of output error moving average (OEMA) systems, this study combines the auxiliary model identification idea with the filtering theory, transforms an OEMA system into two identification models and presents a filtering and auxiliary model-based recursive least squares (F-AM-RLS) identification algorithm. Compared with the auxiliary model-based recursive extended least squares algorithm, the proposed F-AM-RLS algorithm has a high computational efficiency. Moreover, a filtering and auxiliary model-based least squares iterative (F-AM-LSI) identification algorithm is derived for OEMA systems with finite measurement input-output data. Compared with the F-AM-RLS approach, the proposed F-AM-LSI algorithm updates the parameter estimation using all the available data at each iteration, and thus can generate highly accurate parameter estimates.

Monotonic properties of the least squares mean

Abstract: We settle an open problem of several years standing by showing that the least squares mean for positive definite matrices is monotone for the usual (Loewner) order. Indeed we show this is a special case of its appropriate generalization to partially ordered complete metric spaces of nonpositive curvature. Our techniques extend to establish other basic properties of the least squares mean such as continuity and joint concavity. Moreover, we introduce a weighted least squares mean and derive our results in this more general setting.

The Degrees of Freedom of Partial Least Squares Regression

Abstract: The derivation of statistical properties for partial least squares regression can be a challenging task. The reason is that the construction of latent components from the predictor variables also depends on the response variable. While this typically leads to good performance and interpretable models in practice, it makes the statistical analysis more involved. In this work, we study the intrinsic complexity of partial least squares regression. Our contribution is an unbiased estimate of its degrees of freedom. It is defined as the trace of the first derivative of the fitted values, seen as a function of the response. We establish two equivalent representations that rely on the close connection of partial least squares to matrix decompositions and Krylov subspace techniques. We show that the degrees of freedom depend on the collinearity of the predictor variables: The lower the collinearity, the higher the complexity. In particular, they are typically higher than the naive approach that defines the degree...

Frequency and Damping Estimation Methods - An Overview

Abstract: This overview paper presents and compares different methods traditionally used for estimating damped sinusoid parameters. Firstly, direct nonlinear least squares fitting the signal model in the time and frequency domains are described. Next, possible applications of the Hilbert transform for signal demodulation are presented. Then, a wide range of autoregressive modelling methods, valid for da mped sinusoids, are discussed, in which frequency and damping are estimated from calculated signal linear self -prediction coefficients. These methods aim at solving, directly or using least squares, a matrix linear eq uation in which signal or its autocorrelation function samples are used. The Prony, Steiglitz-McBride, Kumaresan-Tu fts, Total Least Squares, Matrix Pencil, YuleWalker and Pisarenko methods are taken into account. Finally, the interpolated discrete Fourier transform is presented with examples of Bertocco, Yoshida, and Agrež algo rithms. The Matlab codes of all the discussed methods are given. The second part of the paper presents sim ulation results, compared with the Cramer-Rao lower bound and commented. All tested methods are compared w ith respect to their accuracy (systematic errors), noise robustness, required signal length, and computational complexity.

Geometric weakly admissible meshes, discrete least squares approximations and approximate Fekete points

Abstract: Using the concept of Geometric Weakly Admissible Meshes (see §2 below) together with an algorithm based on the classical QR factorization of matrices, we compute efficient points for discrete multivariate least squares approximation and Lagrange interpolation.

Recent advances in numerical methods for nonlinear equations andnonlinear least squares

Abstract: Nonlinear equations and nonlinear least squares problems have many applications in physics, chemistry, engineering, biology, economics, finance and many other fields. In this paper, we will review some recent results on numerical methods for these two special problems, particularly on Levenberg-Marquardt type methods, quasi-Newton type methods, and trust region algorithms. Discussions on variable projection methods and subspace methods are also given. Some theoretical results about local convergence results of the Levenberg-Marquardt type methods without non-singularity assumption are presented. A few model algorithms based on line searches and trust regions are also given.

DFT-based Estimation of Damped Oscillation Parameters in Low-Frequency Mechanical Spectroscopy

Abstract: In this paper, we analyze and compare the properties of different well-known and also new nonparametric discrete Fourier transform (DFT)-based methods for resonant frequency and logarithmic decrement estimation in application to mechanical spectroscopy. We derive a new DFT interpolation algorithm for a signal analyzed with Rife-Vincent class-I windows and also propose new formulas that extend Bertocco and Yoshida methods. We study errors of the resonant frequency and logarithmic decrement estimation in realistic conditions that include measurement noise and a zero-point drift. We also investigate the systematic errors of the estimation methods of interest. A nonlinear least squares time-domain parametric signal fitting is used to determine the boundaries of statistical efficiency in all tests.

First-Order System Least Squares for Coupled Stokes-Darcy Flow

Abstract: The coupled problem with Stokes flow in one subdomain and a Darcy flow model in a second subdomain is studied in this paper. Both flow problems are treated as first-order systems, involving pseudostress and velocity in the Stokes case and using a flux-pressure formulation in the Darcy subdomain as process variables, respectively. The Beavers-Joseph-Saffman interface conditions are treated by an appropriate interface functional which is added to the least squares functional associated with the subdomain problems. A combination of $H(\mathrm{div})$-conforming Raviart-Thomas and standard $H^1$-conforming elements is used for the Stokes as well as for the Darcy subsystem. The homogeneous least squares functional is shown to be equivalent to an appropriate norm allowing the use of standard finite element approximation estimates. It also establishes the fact that the local evaluation of the least squares functional itself constitutes an a posteriori error estimator to be used for adaptive refinement strategies.

A contribution to the conditioning of the total least squares problem

Abstract: We derive closed formulas for the condition number of a linear function of the total least-squares solution. Given an overdetermined linear system Ax≈b, we show that this condition number can be computed using the singular values and the right singular vectors of [A,b] and A. We also provide an upper bound that requires the computation of the largest and the smallest singular value of [A,b] and the smallest singular value of A. In numerical examples, we compare these values and the resulting forward error bounds with the error estimates given by Van Huffel and Vandewalle [The Total Least Squares Problem: Computational Aspects and Analysis, Frontiers Appl. Math. 9, SIAM, Philadelphia, 1991], and we show the limitation of the first order approach.

An Improved Weighted Total Least Squares Method with Applications in Linear Fitting and Coordinate Transformation

Abstract: This paper presents an improved weighted total least squares (IWTLS) method for the errors-in-variables (EIV) model with applications in linear fitting and coordinate transformation. In addition, an improved constrained weighted TLS (ICWTLS) method is further obtained based on the IWTLS algorithm. Following the weighted TLS solution (WTLSS) method in which the precisions of any two columns of the design matrix differ only by a scalar factor in linear orthogonal regression problems, the IWTLS method is derived for a more generic case in which there is no proportionality assumption for the cofactor matrix of the design matrix in the EIV model. Compared with existing research on the constrained TLS method under the assumption that both the constraining matrix and the right-hand-side (RHS) vector are error-free, or that only the RHS vector contains errors, the ICWTLS method is proposed for resolving the EIV model with constraints by integrating the observation equations and constraint equations under the assu...

Experimental identification of IPMC actuator parameters through incorporation of linear and nonlinear least squares methods

Abstract: A variety of linear and nonlinear models have been developed to explain the actuation properties of IPMC. The models’ parameters are mostly obtained by linear identification methods. Thus lack of strictly accurate estimation seems to be a big issue. In this paper, an identification approach is presented which incorporates linear with nonlinear least squares method. The IPMC model comprises two parts, the nonlinear relationship between the absorbed current and the applied voltage and the linear relationship between the absorbed current and the blocking force. The parameters of both parts are identified experimentally using linear and nonlinear least squares methods. Experimental results indicate that the methodology works properly in terms of various inputs and convergence of the identified parameters is guaranteed.

Nonlinear Neural-Based Modeling of Soil Cohesion Intercept

Abstract: A new model was derived to estimate undrained cohesion intercept (c) of soil using Multilayer Perceptron (MLP) of artificial neural networks. The proposed model relates c to the basic soil physical properties including coarse and fine-grained contents, grains size characteristics, liquid limit, moisture content, and soil dry density. The experimental database used for developing the model was established upon a series of unconsolidated-undrained triaxial tests conducted in this study. A Nonlinear Least Squares Regression (NLSR) analysis was performed to benchmark the proposed model. The contributions of the parameters affecting c were evaluated through a sensitivity analysis. The results indicate that the developed model is effectively capable of estimating the c values for a number of soil samples. The MLP model provides a significantly better prediction performance than the regression model.

Rank-Deficient Nonlinear Least Squares Problems and Subset Selection

Abstract: We examine the local convergence of the Levenberg-Marquardt method for the solution of nonlinear least squares problems that are rank-deficient and have nonzero residual. We show that replacing the Jacobian by a truncated singular value decomposition can be numerically unstable. We recommend instead the use of subset selection. We corroborate our recommendations by perturbation analyses and numerical experiments.

Hessian Matrix vs. Gauss-Newton Hessian Matrix

Abstract: In this paper, we investigate how the Gauss-Newton Hessian matrix affects the basin of convergence in Newton-type methods. Although the Newton algorithm is theoretically superior to the Gauss-Newton algorithm and the Levenberg-Marquardt (LM) method as far as their asymptotic convergence rate is concerned, the LM method is often preferred in nonlinear least squares problems in practice. This paper presents a theoretical analysis of the advantage of the Gauss-Newton Hessian matrix. It is proved that the Gauss-Newton approximation function is the only nonnegative convex quadratic approximation that retains a critical property of the original objective function: taking the minimal value of zero on an $(n-1)$-dimensional manifold (or affine subspace). Due to this property, the Gauss-Newton approximation does not change the zero-on-$(n-1)$-D “structure” of the original problem, explaining the reason why the Gauss-Newton Hessian matrix is preferred for nonlinear least squares problems, especially when the initial point is far from the solution.

Improvement of grey models by least squares

Abstract: In this paper, we present an approach to the least squares solution to grey Verhulst model, and verify its feasibility by numerical examples. We also present the least squares solutions of grey models GM (1, 1) and GM (2, 1). For the convenience of applications in expert systems, the parameters computing formulas of grey models are also presented here. We carry out some numerical examples to examine the modeling precision of grey models in conventional way and in least squares. The numerical results reveal that the modeling precision of grey models in least squares is always better than that in conventional way.

On-line batch process monitoring using hierarchical kernel partial least squares

Abstract: In this paper, new monitoring approach, hierarchical kernel partial least squares (HKPLS), is proposed for the batch processes. The advantages of HKPLS are: (1) HKPLS gives more nonlinear information compared to hierarchical partial least squares (HPLS) and multi-way PLS (MPLS) and (2) a new batch process monitoring using HKPLS does not need to estimate or fill in the unknown part of the process variable trajectory deviation from the current time until the end. The proposed method is applied to the penicillin process and continuous annealing process and is compared with MPLS and HPLS monitoring results. Applications of the proposed approach indicate that HKPLS effectively capture the nonlinearities in the process variables and show superior fault detectability.

The Total Least Squares Problem in AX≈B: A New Classification with the Relationship to the Classical Works

Abstract: This paper revisits the analysis of the total least squares (TLS) problem AX≈B with multiple right-hand sides given by Van Huffel and Vandewalle in the monograph, The Total Least Squares Problem: Computational Aspects and Analysis, SIAM, Philadelphia, 1991. The newly proposed classification is based on properties of the singular value decomposition of the extended matrix [B|A]. It aims at identifying the cases when a TLS solution does or does not exist and when the output computed by the classical TLS algorithm, given by Van Huffel and Vandewalle, is actually a TLS solution. The presented results on existence and uniqueness of the TLS solution reveal subtleties that were not captured in the known literature.

Least squares solution of the quaternion matrix equation with the least norm

Abstract: For any , is called the j-conjugate matrix of A. If , A is called a j-self-conjugate matrix. If , A is called an anti j-self-conjugate matrix. By using the complex representation of quaternion matrices, the Moore–Penrose generalized inverse and the Kronecker product of matrices, we derive the expressions of the least squares solution with the least norm, the least squares j-self-conjugate solution with the least norm, and the least squares anti j-self-conjugate solution with the least norm of the matrix equation over the skew field of quaternions, respectively.

A near-optimal least squares solution to received signal strength difference based geolocation

Abstract: A simple geometric interpretation for received signal strength (RSS) difference based geolocation can be illustrated by considering a plane containing a single pair of receivers and a transmitter. If the path loss follows a simple inverse power law, the RSS difference (in decibels) between the two receivers can be shown to define a circle on which the transmitter must lie. With additional receivers, the position of the transmitter can be solved by finding the common intersection of the circles corresponding to the different pairs of receivers. In practice, the solution of this problem is complicated by the errors contributed by environmental noise, measurement errors and the deviation of the actual path losses from the model. The optimal nonlinear least squares solution can be obtained by performing a search on a planar grid. However, the computational cost becomes an issue when the number of receivers is large. This paper presents an efficient least squares solution whose performance approaches that of the optimal nonlinear least squares solution.

Nonlinear inverse model for velocity estimation from an image sequence

Abstract: [1] Velocity estimation from an image sequence is one of the most challenging inverse problems in computer vision, geosciences, and remote sensing applications. In this paper a nonlinear model has been created for estimating motion field under the constraint of conservation of intensity. A linear differential form of heat or optical flow equation is replaced by a nonlinear temporal integral form of the intensity conservation constraint equation. Iterative equations with Gauss-Newton and Levenberg-Marguardt algorithms are formulated based on the nonlinear equations, velocity field modeling, and a nonlinear least squares model. An algorithm with progressive relaxation of the overconstraint to improve the performance of the velocity estimation is also proposed. The new estimator is benchmarked using a numerical simulation model. Both angular and magnitude error measurements based on the synthetic surface heat flow from the numerical model demonstrate that the performance of the new approach with the nonlinear model is much better than the results of using a linear model of heat or optical flow equation. Four sequences of NOAA Advanced Very High Resolution Radiometer (AVHRR) images taken in the New York Bight fields is also used to demonstrate the performance of the nonlinear inverse model, and the estimated velocity fields are compared with those measured with the Coastal Ocean Dynamics Radar array. The experimental results indicate that the nonlinear inverse model provides significant improvement over the linear inverse model for real AVHRR data sets.

Consistency of the least weighted squares under heteroscedasticity

Abstract: A robust version of the Ordinary Least Squares accommodating the idea of weighting the order statistics of the squared residuals (rather than directly the squares of residuals) is recalled and its properties are studied. The existence of solution of the corresponding extremal problem and the consistency under heteroscedasticity is proved.