Showing papers on "Non-linear least squares published in 1991"
TL;DR: In this article, logically consistent rules for selecting a vector from any feasible set defined by linear constraints, when either all $n$-vectors or those with positive components or the probability vectors are permissible, are determined.
Abstract: An attempt is made to determine the logically consistent rules for selecting a vector from any feasible set defined by linear constraints, when either all $n$-vectors or those with positive components or the probability vectors are permissible. Some basic postulates are satisfied if and only if the selection rule is to minimize a certain function which, if a "prior guess" is available, is a measure of distance from the prior guess. Two further natural postulates restrict the permissible distances to the author's $f$-divergences and Bregman's divergences, respectively. As corollaries, axiomatic characterizations of the methods of least squares and minimum discrimination information are arrived at. Alternatively, the latter are also characterized by a postulate of composition consistency. As a special case, a derivation of the method of maximum entropy from a small set of natural axioms is obtained.
850 citations
TL;DR: In this article, it was shown that, depending on the specific fitting model and parameter values, there are some limiting situations where bulk resistance, reaction resistance, or diffusion exchange rate cannot be estimated with adequate precision by complex nonlinear least squares fitting.
138 citations
18 Nov 1991
126 citations
TL;DR: In this article, approximate constants of motion of acetylene (C2H2) were derived from analysis of a nonlinear least squares fit of the highly excited vibrational absorption spectrum.
Abstract: Approximate constants of motion of acetylene (C2H2), analyzed previously below 10 000 cm−1, are determined from analysis of a nonlinear least squares fit of the highly excited vibrational absorption spectrum. Although there are at least ten distinct Fermi resonance couplings in the measured spectrum up to 24 000 cm−1, there is one, and quite possibly two, good constants of motion. These constants are pointed out to be equivalent to a preferred energy transfer pathway discussed by Smith and Winn. It is suggested that these constants may also apply to ‘‘unassignable’’ stimulated emission pumping spectra, which sample a different region of phase space.
111 citations
TL;DR: The authors show that fast QR methods and lattice methods in least squares adaptive filtering are duals and follow from identical geometric principles, and develop a fast least squares algorithm of minimal complexity that is a hybrid between a QR and a lattice algorithm.
Abstract: The authors show that fast QR methods and lattice methods in least squares adaptive filtering are duals and follow from identical geometric principles. Whereas the lattice methods compute the residuals of a projection operation via the forward and backward prediction errors, the QR methods compute instead the weights used in the projections. Within this framework, the parameter identification problem is solved using fast QR methods by showing that the reflection coefficients and tap parameters of a least squares lattice filter operating in the joint process mode are immediately available as internal variables in the fast QR algorithms. This parameter set can be readily exploited in system identification, signal analysis, and linear predictive coding, for example. The relations derived also lead to a fast least squares algorithm of minimal complexity that is a hybrid between a QR and a lattice algorithm. The algorithm combines the order recursive properties of the lattice approach with the robust numerical behavior of the QR approach. >
104 citations
TL;DR: It is shown that the Givens-lattice algorithms are computationally more efficient than the fast QR algorithm of Cioffi (1987) and their systolic array implementations are discussed.
Abstract: The author presents a general and systematic approach for deriving new LS (least squares) estimation algorithms that are based solely on Givens rotations. In particular, this approach is used to derive efficient Givens-rotation-based LS lattice algorithms-the Givens-lattice algorithms. By exploiting the relationship between the Givens algorithms and the recursive modified Gram-Schmidt algorithm, it is shown that the time and order update of any order-recursive LS estimation algorithm can be realized by employing only Givens rotations. Applying this general conclusion to LS estimation of time-series signals results in the Givens-lattice algorithms. Two Givens-lattice algorithms, one with square roots and the other without, are presented. It is shown that the Givens-lattice algorithms are computationally more efficient than the fast QR algorithm of Cioffi (1987). The derivation of other Givens rotation-based LS estimation algorithms and their systolic array implementations are discussed. >
96 citations
TL;DR: This paper reformulates, generalizes, and investigates the stability of the modified Prony algorithm introduced by Osborne, and shows that the relative difference between B and the Hessian is shown to converge to zero almost surely in the case of rational fitting.
Abstract: This paper reformulates, generalizes, and investigates the stability of the modified Prony algorithm introduced by Osborne [SIAM J. Numer. Anal. 12 (1975), pp. 571–592], with special reference to rational and exponential fitting. The algorithm, originally for exponential functions, is generalized to the least squares fitting of any function which satisfies a linear homogeneous difference equation. Using the difference equation formulation, the problem is expressed as a separable regression, and hence as a nonlinear eigenproblem in terms of the coefficients of the difference equation. The eigenproblem involves finding the null space of a matrix of data differences B, and is solved using a variant of inverse iteration. Stability of the algorithm is shown to depend on the fact that B closely approximates the Hessian of the sum of squares. The expectations of B and the Hessian are evaluated. In the case of rational fitting, the relative difference between B and the Hessian is shown to converge to zero almost surely. Some details of the implementation of the algorithm are given. A simulation study compares the modified Prony algorithm with the Levenberg algorithm on a rational fitting problem, and supports the theoretical results.
85 citations
TL;DR: In this paper, the authors obtained expressions for the restricted least squares estimator and its covariance matrix in the classical regression model when the matrix of regressors is not necessarily of full rank.
Abstract: The authors obtain expressions for the restricted least squares estimator and its covariance matrix in the classical regression model when the matrix of regressors is not necessarily of full rank. The standard expressions for the restricted least squares estimator are not usable in the short rank case because they rely on the unrestricted estimator. But, in the presence of restrictions, the restricted least squares estimator may be computable even if the unrestricted estimator is not. The authors' derivation produces some additional, useful algebraic results for least squares computation. Copyright 1991 by MIT Press.
80 citations
01 Jan 1991
TL;DR: In this paper, a time-varying, kernel-based analog of nonlinear least squares is proposed to estimate time series models that are possibly nonlinear in parameters, which change smoothly but nonparametrically over time.
Abstract: This chapter discusses the estimation of time series models that are possibly nonlinear in parameters, which change smoothly but nonparametrically over time. We describe a time-varying, kernel-based analog of nonlinear least squares and establish consistency and asymptotic normality for the estimates, with allowance for serial dependence of a general kind in the disturbances. These results draw on general theorems for extremum estimates, which can also be applied to more general time-varying models.
69 citations
TL;DR: The whole experiment design and evaluation procedure is supported: excitation signal optimization, data preprocessing and variance analysis, parameter estimation via nonlinear least squares fitting in the frequency domain, model validation, transfer function and pole/zero plots with uncertainties, simulations.
67 citations
TL;DR: Three new iterative methods for the solution of the linear least squares problem with bound constraints are presented and their performance analyzed, with particular emphasis on the dependence on the starting point and the use of preconditioning for ill-conditioned problems.
01 Aug 1991
TL;DR: The least squares lattice algorithm for adaptive filtering based on the technique of QR decomposition (QRD) is derived from first principles and only requires O(p) operations for the solution of a pth order problem.
Abstract: The least squares lattice algorithm for adaptive filtering based on the technique of QR decomposition (QRD) is derived from first principles. In common with other lattice algorithms for adaptive filtering, this algorithm only requires O(p) operations for the solution of a pth order problem. The algorithm has as its root the QRDbased recursive least squares minimisation algorithm and hence is expected to have superior numerical properties when compared with other fast algorithms. This algorithm contains within it the QRD-based lattice algorithm for solving the least squares linear prediction problem. The algorithm is presented in two forms: one that involves taking square-roots and one that does not. The relationship between the QRD-based lattice algorithm and other least squares lattice algorithms is briefly discussed. The results of some computer simulations of a channel equaliser, using finiteprecision floating-point arithmetic, are presented.
TL;DR: It is discussed how use of both kinds of data is advantageous if the data taken in the visible are significantly less noisy, and how measurements should concentrate on high precision for a few measuring wavelengths, rather than lower precision for many wavelengths.
TL;DR: In this article, the influence of the initial estimator on subsequent iterates or the limit of such estimators has been calculated, and the influence function of the weighted least squares estimator is shown to depend on both position in the design space and the residual relative to an initial regression estimator.
Abstract: The iteratively reweighted least squares algorithm is routinely employed to evaluate robust regression estimates. The importance of beginning the algorithm with a robust estimator of the unknown parameters is often stressed. The precise influence of the initial estimator on subsequent iterates or the limit of such estimators, however, does not seem to have been calculated previously. Because robust regression involves the downweighting of specific points having large rescaled residuals and/or large leverages in the design space, it is natural to think in terms of the weights themselves, rather than some less intuitive function such as the η function of generalized M estimators. Therefore, we consider multiple linear regression by the method of weighted least squares, where the weights are estimated quantities depending on both position in the design space and the residual relative to an initial regression estimator. The influence function of the weighted least squares estimator is shown to depend...
TL;DR: In this paper, an adaptive control problem for some linear stochastic evolution systems in Hilbert spaces is formulated and solved by showing the strong consistency of a family of least squares estimates of the unknown parameters and the convergence of the average quadratic costs with a control based on these estimates to the optimal average cost.
Abstract: An adaptive control problem for some linear stochastic evolution systems in Hilbert spaces is formulated and solved in this paper. The solution includes showing the strong consistency of a family of least squares estimates of the unknown parameters and the convergence of the average quadratic costs with a control based on these estimates to the optimal average cost. The unknown parameters in the model appear affinely in the infinitesimal generator of the C 0 semigroup that defines the evolution system. A recursive equation is given for a family of least squares estimates and the bounded linear operator solution of the stationary Riccati equation is shown to be a continuous function of the unknown parameters in the uniform operator topology
TL;DR: In this article, it was shown that the minimum norm solution is equivalent to the total least square solution, and that two versions of the TLS solution exist, one based on the signal subspace and another based on noise subspace.
Abstract: It is shown that the minimum norm solution is equivalent to the total least squares solution. It is noted that two versions of the total least squares solution exist, one based on the signal subspace and another based on the noise subspace. >
TL;DR: In this paper, nonlinear least squares has proven to be a very useful tool for the estimation of parameters in groundwater modeling, where the only fundamental requirement is to posses one numerical simulator able to calculate the output of the model (i.e., the computed pressure history) once a value x (for example the hydraulic conductivity map) has been assessed to the unknown parameter (hence cp denotes the parameter --* output or modelling operator).
TL;DR: Perturbation bounds for the linear least squares problem minx ‖Ax −b‖2 corresponding to component-wise perturbations in the data are derived using a method of Hager and are often much better than the bounds derived from the standard perturbation analysis.
Abstract: Perturbation bounds for the linear least squares problem min
x
‖Ax −b‖2 corresponding tocomponent-wise perturbations in the data are derived. These bounds can be computed using a method of Hager and are often much better than the bounds derived from the standard perturbation analysis. In particular this is true for problems where the rows ofA are of widely different magnitudes. Generalizing a result by Oettli and Prager, we can use the bounds to compute a posteriori error bounds for computed least squares solutions.
TL;DR: In this article, a weighted least square method is given for the numerical solution of parabolic partial differential equations where the diffusion coefficient changes sign, and the second-order equation is transformed into a first-order system of symmetric-positive differential equations in the sense of Friedrichs and the system is solved using least squares techniques.
Abstract: A weighted least squares method is given for the numerical solution of parabolic partial differential equations where the diffusion coefficient changes sign. The second-order equation is transformed into a first-order system of symmetric-positive differential equations in the sense of Friedrichs and the system is solved using least squares techniques. Error estimates and some numerical examples are presented.
TL;DR: In this article, a data inversion procedure is presented, for the one-dimensional case, to retrieve the subsurface defect depth and thermal resistance from the time evolution of the surface temperature distribution as recorded by a thermographic camera.
Abstract: A data inversion procedure is presented, for the one-dimensional case, to retrieve the subsurface defect depth and thermal resistance from the time evolution of the surface temperature distribution as recorded by a thermographic camera (thermal “defectometry”). The inversion algorithm is based on the global minimization of a specific functional (nonlinear least squares problem) through an iterational routine taking into account simultaneously all of the defect variables and their effects on the values of the observable parameters. Surface losses are included as an unknown Biot number in the inversion routine. This paper deals with one-dimensional thermal flow. Part II, in this same issue, covers the two-dimensional theory and the experimental results.
01 Jan 1991
TL;DR: It is shown that the minimum norm solution is equivalent to the total least squares solution, and it is noted that two versions of the total worst squares solution exist.
Abstract: In this correspondence, we show that the minimum norm solution is equivalent to the total least squares solution. It is also noted that two versions of the total least squares solution exist, one based on the signal subspace and another based on the noise subspace.
Journal Article•
TL;DR: In this article, a convergence method based on the use of nonlinear least squares is described, which is adapted to a layered elastic program (CHEVRON N-layer).
Abstract: In recent years the analysis of pavement structures has relied increasingly on characterizing material properties (such as resilient modulus) by use of nondestructive deflection testing and backcalculation procedures. An important element common to all backcalculation procedures--the technique used to achieve a "convergence" of the measured and calculated deflection basins--will be described. A convergence method based on the use of nonlinear least squares is described. The method was adapted to a layered elastic program (CHEVRON N-layer). This convergence approach improves moduli estimates over prior procedures; however, the most important element is the ability to efficiently backcalculate not only layer moduli but also layer thicknesses. This ability is illustrated by using hypothetical two- and three-layer pavement sections and by using real data for a three-layer section.
TL;DR: The new method is based on structured quasi-Newton methods which yield a good approximation to the second derivative matrix of the objective function and proposes BFGS-like and DFP-like updates in a factorized form which give descent search directions for the objectivefunction.
Abstract: This paper provides a modification to the Gauss—Newton method for nonlinear least squares problems. The new method is based on structured quasi-Newton methods which yield a good approximation to the second derivative matrix of the objective function. In particular, we propose BFGS-like and DFP-like updates in a factorized form which give descent search directions for the objective function. We prove local and q-superlinear convergence of our methods, and give results of computational experiments for the BFGS-like and DFP-like updates.
01 Jan 1991
TL;DR: In this paper, the authors compute the principle curvatures and discuss these results in view of a lemma by A. Pazman (1984b) that ensures the solution of the normal equations to become the sought "least-squares estimate" under certain conditions.
03 Jun 1991
TL;DR: The authors present the details of the second stage in which they use the weighted bicubic spline as a surface representation in a regularization framework, with a Tikhonov stabilizer, as the smoothness norm.
Abstract: The surface reconstruction problem is formulated as a two-stage reconstruction procedure. The first stage is a robust local fit to the data in a multiresolution scheme and the second is a regularized least squares fit, with the addition of an adaptive mechanism in the smoothness functional in order to make the solution well behaved. The authors present the details of the second stage in which they use the weighted bicubic spline as a surface representation in a regularization framework, with a Tikhonov stabilizer, as the smoothness norm. It is shown how the adaptive weights, in the stabilizer help the surface bend across discontinuities by varying the energy of the surface. >
11 Dec 1991
TL;DR: The authors report preliminary results on the development of practical, multivariate, nonlinear, model predictive control for fossil fuel power plants, and the results appear to be significantly better than those of most existing control systems.
Abstract: The authors report preliminary results on the development of practical, multivariate, nonlinear, model predictive control for fossil fuel power plants. The approach used involves the development of a first-principles, nonlinear reduced-order model which captures the dominant static and dynamic characteristics of a power plant. This model is used to predict the plant response to control inputs. Since the model will not exactly match the true plant structure, the parameters of the model must be estimated using prediction error methods or nonlinear least squares. This model is then used in a Kalman filter to estimate process states in real time. These estimated states are used for prediction, enabling the computation of the optimal control sequence. The results of full-scale boiler control simulation were encouraging, and conclusively demonstrate the feasibility of the approach. The results appear to be significantly better than those of most existing control systems. Although some additional problems remain to be solved, no serious problems with the technique have been identified. >
01 Oct 1991
TL;DR: An algorithm is presented to obtain the total least squares estimates of the motion parameters of an object from range/stereo data or perspective views in a closed form and the Cramer-Rao lower bound of the error covariance matrix was derived for the TLS model under the assumption of uncorrelated additive Gaussian noise.
Abstract: An algorithm is presented to obtain the total least squares (TLS) estimates of the motion parameters of an object from range/stereo data or perspective views in a closed form. TLS estimates are suitable when data in both time frames are corrupted by noise, which is an appropriate model for motion analysis in practice. The robustness of different linear least squares methods is analyzed for the estimation of motion parameters against the sensor noise and possible mismatches in establishing object feature point correspondence. As the errors in point correspondence increase, the performance of an ordinary least squares (LS) estimator was found to deteriorate much faster than that of the TLS estimator. The Cramer-Rao lower bound (CRLB) of the error covariance matrix was derived for the TLS model under the assumption of uncorrelated additive Gaussian noise. The CRLB for the TLS model is shown to be always higher than that for the LS model. >