Showing papers on "Non-linear least squares published in 1978"

Algorithms for the Solution of the Nonlinear Least-Squares Problem

Abstract: This paper describes a modification to the Gauss–Newton method for the solution of nonlinear least-squares problems. The new method seeks to avoid the deficiencies in the Gauss–Newton method by improving, when necessary, the Hessian approximation by specifically including or approximating some of the neglected terms. The method seeks to compute the search direction without the need to form explicitly either the Hessian approximation or a factorization of this matrix. The benefits of this are similar to that of avoiding the formation of the normal equations in the Gauss-Newton method. Three algorithms based on this method are described; one which assumes that second derivative information is available and two which only assume first derivatives can be computed.

Dud, A Derivative-Free Algorithm for Nonlinear Least Squares

Abstract: Derivative-free nonlinear least squares algorithms which make efficient use of function evaluations are important for fitting models defined by systems of nonlinear differential equations. A new Gauss-Newton-like algorithm with these properties is developed. The performance of the new algorithm (called Dud for “doesn't use derivatives”) is evaluated on a number of standard test problems from the literature. On these problems Dud competes favorably with even the best derivative-based algorithms.

Updating the singular value decomposition

Abstract: LetA be anmA—n matrix with known singular value decomposition. The computation of the singular value decomposition of a matrixAƒ is considered, whereAƒ is obtained by appending a row or a column toA whenmÂ?n or by deleting a row or a column fromA whenm>n. An algorithm is also presented for solving the updated least squares problemAƒ yÂ?bÂ?, obtained from the least squares problemAxÂ?b by appending an equation, deleting an equation, appending an unknown, or deleting an unknown.

Approximation to Data by Splines with Free Knots

Abstract: Approximations to data by splines improve greatly if the knots are free variables. Using the B-spline representation for splines, and separating the linear and nonlinear aspects, the approximation problem reduces to nonlinear least squares in the variable knots.We describe the problems encountered in this formulation caused by the “lethargy” theorem, and how a logarithmic transformation of the knots can lead to an effective method for computing free knot spline approximations.

The principal components of mixed measurement level multivariate data: An alternating least squares method with optimal scaling features

Abstract: A method is discussed which extends principal components analysis to the situation where the variables may be measured at a variety of scale levels (nominal, ordinal or interval), and where they may be either continuous or discrete. There are no restrictions on the mix of measurement characteristics and there may be any pattern of missing observations. The method scales the observations on each variable within the restrictions imposed by the variable's measurement characteristics, so that the deviation from the principal components model for a specified number of components is minimized in the least squares sense. An alternating least squares algorithm is discussed. An illustrative example is given.

Methods of Analysis of Linear Models with Unbalanced Data

Abstract: The objective of this article is to review existing methods for analyzing experimental design models with unbalanced data and to relate them to existing computer programs The methods are distinguished by the hypotheses associated with the sums of squares which are generated The choice of a method should be based on the appropriateness of the hypothesis rather than on computational convenience or the orthogonality of the quadratic forms The sums of squares are described using the R ( ) notation as applied to the over-parameterized linear model, but the hypotheses are stated in terms of the full-rank cell means model The zero-cell frequency situation is treated briefly

Strong consistency of least squares estimates in multiple regression

Abstract: The strong consistency of least squares estimates in multiple regression models with independent errors is obtained under minimal assumptions on the design and weak moment conditions on the errors.

Techniques for nonlinear least squares and robust regression

Abstract: Recently, the authors and others have made considerable progress in developing algorithms for solving certain large-residual nonlinear least-squares problems where Gauss-Newton (GN) methods can be expected to perform poorly. These methods take account of the term in the Hessian ignored by the GN methods and use quasi-Newton procedures to update this term explicitly. This paper reviews these new approaches and discusses how they can be modified to give good performance on nonlinear models with robust loss functions where lack of scale invariance causes several new problems to arise.

Internal dynamics of van der Waals complexes. II. Determination of a potential energy surface for ArHCl

Abstract: The Born–Oppenheimer angular‐radial separation method for calculating ground state properties of atom‐diatomic complexes has been used to determine a potential energy surface for ArHCl. Using a nonlinear least squares procedure, the calculated properties from trial surfaces were fit to molecular beam electric resonance data including both radial and angular expectation values. The inclusion of coriolis coupling terms in the Hamiltonian were found to have a small but discernable effect on the calculated properties. Both the number and type of parameters used to describe the surface affected their correlations dramatically. Fitting the angular properties of the complex required the potential to have an anisotropic to isotropic strength ratio of about 1:2. The isotropic portion of the potential could not be uniquely determined from the bound‐state data alone, but was fixed by predicted differential elastic scattering cross‐sections. In terms of R, the length of the vector connecting the Ar and the center of ...

A Comparison of Ridge Estimators

Abstract: Least squares estimates of the parameters in the usual linear regression model are likely to be too large in absolute value and possibly of the wrong sign when the vectors of explanatory variables are multicollinear. Hoer1 and Kennard have demonstrated that these undesirable effects of multicollinearity can be reduced by using “ridge” estimates in place of the least squares estimates. Unfortunately, ordinary ridge estimates depend on a value, k, which, in practice, is determined by the data. Several mechanical rules and a graphical procedure, known as the ridge trace, have been proposed for selecting k. In this paper we evaluate the relative performances of several mechanical selection rules, the ridge trace and the least squares procedure using computer simulation experiments.

Estimation for a Linear Regression Model with Unknown Diagonal Covariance Matrix

Abstract: A method of estimating the parameters of a linear regression model when the covariance matrix is an unknown diagonal matrix is investigated. It is assumed that the observations fall into $k$ groups with constant error variance for a group. The estimation is carried out in two steps, the first step being an ordinary least squares regression. The least squares residuals are used to estimate the covariance matrix and the second step is the calculation of the generalized least squares estimator using the estimated covariance matrix. The large sample properties of the estimator are derived for increasing $k$, assuming the numbers in the groups form a fixed sequence.

Constrained multidimensional scaling inN spaces

Abstract: A gradient method is used to obtain least squares estimates of parameters of them-dimensional euclidean model simultaneously inN spaces, given the observation of all pairwise distances ofn stimuli for each space. The procedure can estimate an additive constant as well as stimulus projections and the metric of the reference axes of the configuration in each space. Each parameter in the model can be fixed to equal some a priori value, constrained to be equal to any other parameter, or free to take on any value in the parameter space. Two applications of the procedure are described.

On least squares collocation

Abstract: It is shown that the least squares collocation approach to estimating geodetic parameters is identical to conventional minimum variance estimation. Hence, the least squares collocation estimator can be derived either by minimizing the usual least squares quadratic loss function or by computing a conditional expectation by means of the regression equation. When a deterministic functional relationship between the data and the parameters to be estimated is available, one can implement a least squares solution using the functional relation to obtain an equation of condition. It is proved the solution so obtained is identical to what is obtained through least squares collocation. The implications of this equivalance for the estimation of mean gravity anomalies are discussed.

On least squares approximations to indefinite problems of the mixed type

Abstract: A least squares method is presented for computing approximate solutions of indefinite partial differential equations of the mixed type such as those that arise in connection with transonic flutter analysis. The method retains the advantages of finite difference schemes namely simplicity and sparsity of the resulting matrix system. However, it offers some great advantages over finite difference schemes. First, the method is insensitive to the value of the forcing frequency, i.e., the resulting matrix system is always symmetric and positive definite. As a result, iterative methods may be successfully employed to solve the matrix system, thus taking full advantage of the sparsity. Furthermore, the method is insensitive to the type of the partial differential equation, i.e., the computational algorithm is the same in elliptic and hyperbolic regions. In this work the method is formulated and numerical results for model problems are presented. Some theoretical aspects of least squares approximations are also discussed.

Numerically stable computations for general univariate linear models

Abstract: We consider the general univariate linear model E(y) = Xb, V(y) = o2 W, W symmetric nonnegative definite. A numerically stable method based on orthogonal rotations is given for computing the least squares estimate [bcirc] of b , as well as a representation of [/(b). It is shown how to extend the computations to update these results quickly and accurately when columns or rows of (y,X) are added or taken away. One of these techniques will handle the usual F-test for the general linear hypothesis, and the updating techniques can easily handle less than full rank X and W , while checking for consistency of the model. The first section describes some disadvantages of the original formulation of the problem and gives a general formulation which avoids these. The second section describes a numerically stable method for solution, while the third considers the statistical meaning of the computed quantities. Section 4 introduces the updating techniques as continuations of the original decomposition and Section 5 tr...

A Method for Separable Nonlinear Least Squares Problems with Separable Nonlinear Equality Constraints

Abstract: Recently several algorithms have been proposed for solving separable nonlinear least squares problems which use the explicit coupling between the linear and nonlinear variables to define a new nonlinear least squares problem in the nonlinear variables only whose solution is the solution to the original problem. In this paper we extend these techniques to the separable nonlinear least squares problem subject to separable nonlinear equality constraints.

The computation of fiml estimates as iterative generalized least squares estimates in linear and nonlinear simultaneous equations models

Abstract: THE PURPOSE OF THIS PAPER is threefold: (i) to provide a unified iterative generalized least squares approach for computing FIML estimates for both linear and nonlinear simultaneous equations models of medium size (containing about 50 parameters in the nonlinear case and 75 in the linear case); (ii) to suggest a useful procedure to determine the length of the step to be made at each iteration of the algorithm; (iii) to report on numerical results which illustrate the robustness and the relative efficiency of the proposed method.

Comment on the Iterative Refinement of Least-Squares Solutions

Abstract: Three different iterative algorithms for refining a computed least-squares estimate are compared. The first two algorithms are based on the orthogonal decomposition of the matrix X, whereas the third, recently advocated by Fletcher (1975), uses the Cholesky decomposition of XT X. The third algorithm theoretically can give good results even when the condition number κ (X) is very large, but in general, the first two algorithms are superior and far more reliable. A variant of one of these may be advantageous when X is large and sparse. Numerical results confirm the theoretical analysis.

An Optimal Property of Principal Components in the Context of Restricted Least Squares

Abstract: A new optimal property for principal components regression is presented. In particular, it is shown that the trace of the covariance matrix for estimators obtained by deleting principal components associated with the smallest eigenvalues is at least as small as that for any other least-squares estimator with an equal or smaller number of linear restrictions. This property is useful in suggesting data transformations and determining the maximum variance reduction obtainable from the introduction of linear restrictions on the parameter space.

A Bound for the Euclidean Norm of the Difference Between the Least Squares and the Best Linear Unbiased Estimators

Abstract: Haberman's bound for a norm of the difference between the least squares and the best linear unbiased estimators in a linear model with nonsingular covariance structure is examined in the particular case when a vector norm involved is taken as the Euclidean one. In this frequently occurring case, a new substantially improved bound is developed which, furthermore, is applicable regardless of any additional condition.

Discrete least squares approximations for ordinary differential equations

Abstract: The application of the least squares method, using $C^q $ piecewise polynomials of order $k + m,k \geqq m,q \geqq m$, for obtaining approximations to an isolated solution of a nonlinear mth order ordinary differential equation, involves integrals which in practice need to be discretized. Using for this latter purpose the k-point Gaussian quadrature rule in each subinterval, the discrete least squares schemes obtained are close to collocation, on the same points, by piecewise polynomials from $C^{m - 1} $.We prove here that under smoothness assumptions similar to those made by de Boor and Swartz for the collocation procedure, i.e. that the solution be in $C^{m + 2k} $, an optimal global rate of convergence $O(|\Delta |^{k + m} )$ is obtained in the uniform norm for the discrete least squares schemes, provided that the partitions $\Delta $ are quasiuniform. In addition, a superconvergence rate of $O(|\Delta |^{2k} )$ is obtained at the knots for those derivatives l which satisfy $0 \leqq l \leqq 2(m - 1) - q$.