Showing papers on "Non-linear least squares published in 1992"
TL;DR: A novel iterative algorithm for deriving the least squares frequency response weighting function which will produce a quasi-equiripple design is presented and typically produces a design which is only about 1 dB away from the minimax optimum solution in two iterations and converges to within 0.1 dB in six iterations.
Abstract: It has been demonstrated by several authors that if a suitable frequency response weighting function is used in the design of a finite impulse response (FIR) filter, the weighted least squares solution is equiripple. The crux of the problem lies in the determination of the necessary least squares frequency response weighting function. A novel iterative algorithm for deriving the least squares frequency response weighting function which will produce a quasi-equiripple design is presented. The algorithm converges very rapidly. It typically produces a design which is only about 1 dB away from the minimax optimum solution in two iterations and converges to within 0.1 dB in six iterations. Convergence speed is independent of the order of the filter. It can be used to design filters with arbitrarily prescribed phase and amplitude response. >
266 citations
TL;DR: A unified description of the multilinear models in an array notation is presented, showing how to interpret one initialization of the nonlinear least- squares fits of these models.
Abstract: Multilinear models are models in which the expectation of a multiway array is the sum of products of parameters, where each parame- ter is associated with only one of the ways. In spectroscopy, multilinear models permit mathematical decompositions of data sets when chemical decomposition of specimens is difficult or impossible. This paper presents a unified description of the models in an array notation. The spectroscopic context shows how to interpret one initialization of the nonlinear least- squares fits of these models. Several examples show that these models can be applied successfully.
139 citations
TL;DR: An alternative approach to numerical convolution with exponential decays is described, which constructs a continuous instrumental response function by quadratic fitting of the discrete data and evaluates the convolution integral directly, without approximations.
120 citations
TL;DR: In this article, a nonlinear regression procedure is used to determine parameters for the specification of the spectral functional form of complex permittivity and permeability, which can be applied to many low and high-permittivity dielectric and magnetic materials in either the low or high loss range.
Abstract: A technique for the solution of one-port and two-port scattering equations for complex permittivity and permeability determination is presented. Using a nonlinear regression procedure, the model determines parameters for the specification of the spectral functional form of complex permittivity and permeability. The method is based on a nonlinear regression technique and uses the fact that a causal, analytic function can be represented by poles and zeros. The technique allows the accurate determination of many low- and high-permittivity dielectric and magnetic materials in either the low- or high-loss range. The model allows for small adjustments, consistent with the physics of the problem, to independent variable data such as angular frequency, sample length, sample position, and cut-off wavelength. The model can determine permittivity and permeability for samples where sample length, sample position, and sample holder length are not known precisely. The problem of local minima is discussed. >
108 citations
TL;DR: In this article, the authors proposed a control-relevant identification strategy for a class of long-range predictive controllers and showed that under certain conditions the best process model for predictive control is that which is estimated using an identification objective function that is a dual of the control objective function.
Abstract: The question of a suitable control-relevant identification strategy for a class of long-range predictive controllers is addressed. It is shown that under certain conditions the best process model for predictive control is that which is estimated using an identification objective function that is a dual of the control objective function. The resulting nonlinear least squares calculation is asymptotically equal to a standard recursive least squares with an appropriate (model and controller-dependent) FIR data prefilter. Experimental results demonstrate the validity and practicality of the proposed estimation law. >
105 citations
TL;DR: This work presents a unified derivation of four rotation-based recursive least squares algorithms that solve the adaptive least squares problems of the linear combiner, thelinear combiner without a desired signal, the single channel, and the multichannel linear prediction and transversal filtering.
Abstract: This work presents a unified derivation of four rotation-based recursive least squares (RLS) algorithms. They solve the adaptive least squares problems of the linear combiner, the linear combiner without a desired signal, the single channel, and the multichannel linear prediction and transversal filtering. Compared to other approaches, the authors' derivation is simpler and unified, and may be useful to readers for better understanding the algorithms and their relationships. Moreover, it enables improvements of some algorithms in the literature in both the computational and the numerical issues. All algorithms derived in this work are based on Givens rotations. They offer superior numerical properties as shown by computer simulations. They are computationally efficient and highly concurrent. Aspects of parallel implementation and parameter identification are discussed. >
93 citations
TL;DR: In this article, two different methods for complex nonlinear least squares fitting of the small-signal ac response of dielectric and partially conducting systems are described and compared. And the results of fitting with several different weighting schemes available as choices in the vector-minimization approach are compared and discussed.
68 citations
TL;DR: In this paper, a set of 5 functions for robust regression using the technique of iteratively reweighted least squares (IRLS) is developed, which is simple to understand and provides great flexibility for IRLS methods.
Abstract: We develop a set of 5 functions for robust regression using the technique of iteratively reweighted least squares (IRLS). Together with a set of weight functions, function rreg is simple to understand and provides great flexibility for IRLS methods. This article focuses on the programming strategies adopted to achieve the twin goals of power and simplicity.
63 citations
TL;DR: In this article, the estimation procedure is based on iterative generalised least squares and a number of special cases are described and the procedure is implemented within a general purpose statistical package.
59 citations
TL;DR: The problem addressed here is the determination of the total least squares solution, subject to a unitary constraint, of an overdetermined, inconsistent, linear system of equations.
Abstract: The problem addressed here is the determination of the total least squares solution, subject to a unitary constraint, of an overdetermined, inconsistent, linear system of equations. The problem arises in many signal processing applications, three of which are briefly presented and studied here. The solution to the constrained total least squares problem is seen to be the same as the solution to the orthogonal Procrustes problem.
52 citations
TL;DR: A new way of looking at a class of methods for the weighted linear least squares problem by introducing a modified QR-decomposition with QM-invariant, i.e., QMQ^T = M, where linear constraints are easily incorporated by letting the corresponding diagonal elements in M become zero.
Abstract: A new way of looking at a class of methods for the weighted linear least squares problem $\min _x \| M^{ - ( 1/2 )} ( b - Ax ) \|_2 $ where $M = {\operatorname{diag}}(\mu _i )$ is presented by intr...
TL;DR: In this article, a method of identifying stationary and invertible vector autoregressive moving-average time series is proposed, where models are assumed to be represented in (reversed) echelon canonical form.
Abstract: In this paper a method of identifying stationary and invertible vector autoregressive moving-average time series is proposed. The models are presumed to be represented in (reversed) echelon canonical form. Consideration is given to both parameter estimation and the determination of structural indices, the evaluations being based on the use of closed form least squares calculations. Consistency of the technique is shown and the operational characteristics of the procedure when employed as a means of approximating more general processes is discussed.
TL;DR: In this paper, the least squares estimates of the multiplicative terms are obtained from singular value decomposition of the matrix where but the least square estimate of the shift parameter depends on estimates of parameters contained in the multiplier terms.
Abstract: This paper presents analysis of a two-way table of data using a "shifted multiplicative model (SHMM) of the form The least squares estimates of the multiplicative terms are obtained from singular value decomposition of the matrix where but the least squares estimate of the shift parameter depends on estimates of parameters contained in the multiplicative terms. The sum of squares can be minimized as a function only of for which iterative Newton-Raphson and generalized En algorithms are developed. Expectations of sums of squares owing to sequentially increasing the number, t, of multiplicative terms (presented in an AMOVA format) were obtained by Monte Carlo simulation for the case where errors are i.i.d.N(0,σ2) and all The analysis is illustrated with several examples from the literature.
Book•
01 Jun 1992
TL;DR: In this paper, the authors present a model selection approach based on Fourier Transform Techniques and Potentiometric Titrations for non-Linear Least Squares (NLSS).
Abstract: Observational Errors. Linear Least Squares. Non-Linear Least Squares. Formulation and Selection of Models. Criteria for Model Selection. Polynomials. Fitting Functions. Fourier Transform Techniques. Potentiometric Titrations. Appendices. References and Additional Reading. Index.
TL;DR: In this paper, the authors explore model misspecification diagnostics based on least squares and least median of squares fits, and show that in some circumstances, least median-of-squares methods (or any other estimator with the exact fit property) fail to reveal an incorrectly specified mean function, but least squares methods succeed.
Abstract: This article explores model misspecification diagnostics based on least squares and least median of squares fits. It shows that in some circumstances, least median of squares methods (or any other estimator with the exact fit property) fail to reveal an incorrectly specified mean function, but least squares methods succeed.
TL;DR: In this paper, a generalized least square based estimator is proposed based on nonparametric nearest neighbor estimates of the conditional variance matrices for the multiple equations nonlinear regression model with heteroskedasticity of unknown form.
Abstract: Asymptotically efficient estimates for the multiple equations nonlinear regression model are obtained in the presence of heteroskedasticity of unknown form. The proposed estimator is a generalized least squares based on nonparametric nearest neighbor estimates of the conditional variance matrices. Some Monte Carlo experiments are reported.
TL;DR: In this article, the TLS problem is extended to the more general case and the sets of the solutions and the squared residuals for the TLS and LS problems are compared, and the connection between the approximate subspaces and the perturbation theories are studied.
Abstract: This paper completes our previous discussion on the total least squares (TLS) and the least squares (LS) problems for the linear systemAX=B which may contain more than one solution [12, 13], generalizes the work of Golub and Van Loan [1,2], Van Huffel [8], Van Huffel and Vandewalle [11]. The TLS problem is extended to the more general case. The sets of the solutions and the squared residuals for the TLS and LS problems are compared. The concept of the weighted squares residuals is extended and the difference between the TLS and the LS approaches is derived. The connection between the approximate subspaces and the perturbation theories are studied.
It is proved that under moderate conditions, all the corresponding quantities for the solution sets of the TLS and the modified LS problems are close to each other, while the quantities for the solution set of the LS problem are close to the corresponding ones of a subset of that of the TLS problem.
TL;DR: In this paper, the authors considered the problem of reconstructing the phase phi of a complex-valued function given knowledge of the magnitude mod f mod and the magnitude of the Fourier transform mod (fei phi )V-product mod.
Abstract: Consider the problem of reconstructing the phase phi of a complex-valued function fei phi , given knowledge of the magnitude mod f mod and the magnitude of the Fourier transform mod (fei phi )V-product mod . The author considers the formulation as a least-squares minimization problem. It is shown that the linearized problem is ill posed. Also, surprisingly, the gradient of the least-squares objective functional is not Frechet differentiable. A regularization is introduced which restores differentiability and also counteracts instability. It is shown how a certain implementation of Newton's method can be used to solve the regularized least-squares problem efficiently, and that the method converges locally, almost quadratically. Numerical examples are given with an application to diffractive optics.
TL;DR: This work derives a formula for estimating the Lagrange multiplier which depends on the amount the unconstrained solution violates the constraint and an estimate of the smallest generalized singular value of A andB, which can be used as a good initial guess for solving the secular equation.
Abstract: We present algorithms for solving quadratically constrained linear least squares problems that do not necessarily require expensive dense matrix factorizations. Instead, only “black box” solvers for certain related unconstrained least squares problems, as well as the solution of two related linear systems involving the coefficient matrixA and the constraint matrixB, are required. Special structures in the problem can thus be exploited in these solvers, and iterative as well as direct solvers can be used. Our approach is to solve for the Lagrange multiplier as the root of an implicitly-defined secular equation. We use both a linear and a rational (Hebden) local model and a Newton and secant method. We also derive a formula for estimating the Lagrange multiplier which depends on the amount the unconstrained solution violates the constraint and an estimate of the smallest generalized singular value ofA andB. The Lagrange multiplier estimate can be used as a good initial guess for solving the secular equation. We also show conditions under which this estimate is guaranteed to be an acceptable solution without further refinement. Numerical results comparing the different algorithms are presented.
TL;DR: The algorithm generalizes the algorithm of Rutishauser-Gragg-Harrod-Reichel for discrete least squares polynomial approximation to the rational case and is fast in the sense that it requires ordermα computation time.
Abstract: A new method for discrete least squares linearized rational approximation is presented. It generalizes the algorithm of Rutishauser-Gragg-Harrod-Reichel for discrete least squares polynomial approximation to the rational case. The algorithm is fast in the sense that it requires orderm? computation time wherem is the number of data points and ? is the degree of the approximant. We describe how this algorithm can be implemented in parallel.
16 Dec 1992
TL;DR: In this paper, a weighted least square (WLS) algorithm was proposed for adaptive tracking problems with a complex multivariable ARMAX (autoregressive moving-average with exogeneous inputs).
Abstract: For a complex multivariable ARMAX (autoregressive moving-average with exogeneous inputs) model, the author studies the weighted least squares algorithm which improves the usual least squares algorithm by the choice of suitable ponderations. Concerning adaptive tracking problems, both strong consistency of the estimator and control optimality are ensured. >
TL;DR: In this article, a least square finite element (LSFEF) method was proposed for the solution of fluid flow problems, which is based on the minimization of the L 2 norm of the equation residuals.
Abstract: We investigate the application of a least squares finite element method for the solution of fluid flow problems. The least squares finite element method is based on the minimization of the L2 norm of the equation residuals. Upon discretization, the formulation results in a symmetric, positive definite matrix system which enables efficient iterative solvers to be used. The other motivations behind the development of least squares finite element methods are the applicability of higher order elements and the possibility of using the norm associated to the least squares functional for error estimation. For steady incompressible flows, we develop a method employing linear and quadratic triangular elements and compare their respective accuracy. For steady compressible flows, an implicit conservative least squares scheme which can capture shocks without the addition of artificial viscosity is proposed. A refinement strategy based upon the use of the least squares residuals is developed and several numerical examples are used to illustrate the capabilities of the method when implemented on unstructured triangular meshes.
TL;DR: In this paper, the performances of the combination forecasts of a macroeconomic time series obtained using Nonnegativity Restricted Least Squares (NRLS) and other combination methods are exhaustively compared.
TL;DR: In this paper, the exact, small-signal impedance and admittance response of a fully dissociated, equi-valent material with arbitrary reaction rates and mobilities of the positive and negative charge carriers is discussed and illustrated for many different parameter values of physical significance.
TL;DR: In this paper, the full-disk line profiles of the solar O I resonant triplet at 1304 A were estimated using data from the SMM and the Orbiting Solar Observatory 8 for the time period 1975-1985.
Abstract: Estimates were made of the full-disk line profiles of the solar O I resonant triplet at 1304 A, using data from the SMM and the Orbiting Solar Observatory 8 for the time period 1975-1985. The observed line profiles are parameterized using a nonlinear least squares fit of the data by a simple empirical representation of the general line shape. The derived full-disk line shape parameters may be used, in combination with an appropriate value for the line-integrated full-disk solar flux, in the analysis of planetary or cometary O I 1304-A observations under any solar conditions.
TL;DR: An algorithm for the selection of knot point locations for approximation of functions from large sets of scattered data by least squares thin plate splines is given, based on the idea that each data point is equally important in defining the surface.
Abstract: An algorithm for the selection of knot point locations for approximation of functions from large sets of scattered data by least squares thin plate splines is given. The algorithm is based on the idea that each data point is equally important in defining the surface, which allows the knot selection process to be decoupled from the least squares. Properties of the algorithm are investigated, and examples demonstrating it are given. Results of some least squares approximations are given and compared with other approximation methods.
TL;DR: In this paper, a semiparametric method for the estimation of truncated regression models where the disturbances are independent of the regressors before truncation is presented, which provides useful information on model identification and estimation.
Abstract: This article provides a semiparametric method for the estimation of truncated regression models where the disturbances are independent of the regressors before truncation. This independence property provides useful information on model identification and estimation. Our estimate is shown to be -consistent and asymptotically normal. A consistent estimate of the asymptotic covariance matrix of the estimator is provided. Monte Carlo experiments are performed to investigate some finite sample properties of the estimator.
09 Aug 1992
TL;DR: In this article, the authors derived a general expression of the optimal v for which the breakdown point of the LMS attains the highest possible fraction of outliers that any regression equivariant estimator can handle.
Abstract: The least median of squares (LMS) estimator minimizes the vth ordered squared residual. The authors derived a general expression of the optimal v for which the breakdown point of the LMS attains the highest possible fraction of outliers that any regression equivariant estimator can handle. This fraction is equal to half of the minimum surplus divided by the number of measurements in the network. The surplus of a fundamental set is defined as the smallest number of measurements whose removal from that fundamental set turns at least one measurement in the network into a critical one. Based on the surplus concept, a system decomposition scheme that significantly increases the number of outliers that can be identified by the LMS is developed. In addition, it dramatically reduces the computing time of the LMS, opening the door to real-time applications of that estimator to large-scale systems. Finally, outlier diagnostics based on robust Mahalanobis distances are proposed. >
TL;DR: In this article, it was shown that the solution to a total linear least squares problem satisfies a quadratic matrix equation, which turns into an algebraic Riccati equation when the matrix of unknowns is square.