Showing papers on "Rate of convergence published in 1986"
TL;DR: In this paper, an unconditionally stable algorithm for plane stress elastoplasticity is developed, based upon the notion of elastic predictor-return mapping (plastic corrector). Enforcement of the consistency condition is shown to reduce to the solution of a simple nonlinear equation.
Abstract: An unconditionally stable algorithm for plane stress elastoplasticity is developed, based upon the notion of elastic predictor-return mapping (plastic corrector). Enforcement of the consistency condition is shown to reduce to the solution of a simple nonlinear equation. Consistent elastoplastic tangent moduli are obtained by exact linearization of the algorithm. Use of these moduli is essential in order to preserve the asymptotic rate of quadratic convergence of Newton methods. The accuracy of the algorithm is assessed by means of iso-error maps. The excellent performance of the algorithm for large time steps is illustrated in numerical experiments.
662 citations
TL;DR: In this paper, it is shown that the annealing algorithm converges with probability arbitrarily close to 1, and that it is no better than a deterministic method. But it is also shown that there are cases where convergence takes exponentially long.
Abstract: The annealing algorithm is a stochastic optimization method which has attracted attention because of its success with certain difficult problems, including NP-hard combinatorial problems such as the travelling salesman, Steiner trees and others. There is an appealing physical analogy for its operation, but a more formal model seems desirable. In this paper we present such a model and prove that the algorithm converges with probability arbitrarily close to 1. We also show that there are cases where convergence takes exponentially long—that is, it is no better than a deterministic method. We study how the convergence rate is affected by the form of the problem. Finally we describe a version of the algorithm that terminates in polynomial time and allows a good deal of ‘practical’ confidence in the solution.
609 citations
TL;DR: In this paper, a variant of the Byzantine Generals problem is considered, in which processes start with arbitrary real values rather than Boolean values or values from some bounded range, and in which approximate, rather than exact, agreement is the desired goal.
Abstract: This paper considers a variant of the Byzantine Generals problem, in which processes start with arbitrary real values rather than Boolean values or values from some bounded range, and in which approximate, rather than exact, agreement is the desired goal. Algorithms are presented to reach approximate agreement in asynchronous, as well as synchronous systems. The asynchronous agreement algorithm is an interesting contrast to a result of Fischer et al, who show that exact agreement with guaranteed termination is not attainable in an asynchronous system with as few as one faulty process. The algorithms work by successive approximation, with a provable convergence rate that depends on the ratio between the number of faulty processes and the total number of processes. Lower bounds on the convergence rate for algorithms of this form are proved, and the algorithms presented are shown to be optimal.
531 citations
TL;DR: The present paper demonstrates how the additive correction method of Settari and Aziz can be used and extended to improve the convergence rate for two- and three-dimensional problems when the coefficients are anisotropic.
Abstract: The solution of large sets of equations is required when discrete methods are used to solve fluid flow and heat transfer problems The cost of the solution often becomes prohibitive when the coefficients of the algebraic equations become strongly anisotropic or when the number of equations in the set becomes large The present paper demonstrates how the additive correction method of Settari and Aziz can be used and extended to improve the convergence rate for two- and three-dimensional problems when the coefficients are anisotropic Such methods are interpreted as simple multigrid methods With this as the basis a new general multigrid method is developed that has attractive properties The efficiency of the new method is compared to that of a conventional multigrid method, and its performance is demonstrated on other problems
429 citations
TL;DR: In this paper, it has been shown that a very modest degree of convergence of an extreme Ritz value already suffices for an increased rate of convergence to occur, which is known as superlinear convergence.
Abstract: It has been observed that the rate of convergence of Conjugate Gradients increases when one or more of the extreme Ritz values have sufficiently converged to the corresponding eigenvalues (the “superlinear convergence” of CG). In this paper this will be proved and made quantitative. It will be shown that a very modest degree of convergence of an extreme Ritz value already suffices for an increased rate of convergence to occur.
351 citations
TL;DR: In this article, a theoretical analysis of simulated annealing based on its precise model, a time-inhomogeneous Markov chain, is presented and a bound on the departure of the probability distribution of the state at finite time from the optimum is obtained.
Abstract: Simulated annealing is a randomized algorithm which has been proposed for finding globally optimum least-cost configurations in large NP-complete problems with cost functions which may have many local minima. A theoretical analysis of simulated annealing based on its precise model, a time-inhomogeneous Markov chain, is presented. An annealing schedule is given for which the Markov chain is strongly ergodic and the algorithm converges to a global optimum. The finite-time behavior of simulated annealing is also analyzed and a bound obtained on the departure of the probability distribution of the state at finite time from the optimum. This bound gives an estimate of the rate of convergence and insights into the conditions on the annealing schedule which gives optimum performance.
341 citations
01 Dec 1986
TL;DR: This paper gives a short survey of Monte Carlo algorithms for stochastic optimization, with emphasis on the analysis of convergence rate.
Abstract: This paper gives a short survey of Monte Carlo algorithms for stochastic optimization. Both discrete and continuous parameter stochastic optimization are discussed, with emphasis on the analysis of convergence rate. Some future research directions for the area are also indicated.
323 citations
TL;DR: The two-dimensional Euler equations have been solved on a triangular grid by a multigrid scheme using the finite volume approach and in its present form, the accuracy and convergence rate of the triangle code are comparable to that of the quadrilateral mesh code of Jameson.
Abstract: The two-dimensional Euler equations have been solved on a triangular grid by a multigrid scheme using the finite volume approach. By careful construction of the dissipative terms, the scheme is designed to be secondorder accurate in space, provided the grid is smooth, except in the vicinity of shocks, where it behaves as firstorder accurate. In its present form, the accuracy and convergence rate of the triangle code are comparable to that of the quadrilateral mesh code of Jameson.
322 citations
TL;DR: In this paper, a method for adaptive stabilization without a minimum-phase assumption and without knowledge of the sign of the high-frequency gain is developed, which leads to a guarantee of Lyapunov stability and an exponential rate of convergence for the state.
Abstract: In this paper, we develop a method for adaptive stabilization without a minimum-phase assumption and without knowledge of the sign of the high-frequency gain. In contrast to recent work by Martensson [8], we include a compactness requirement on the set of possible plants and assume that an upper bound on the order of the plant is known. Under these additional hypotheses, we generate a piecewise linear time-invariant switching control law which leads to a guarantee of Lyapunov stability and an exponential rate of convergence for the state. One of the main objectives in this paper is to eliminate the possibility of "large state deviations" associated with a search Over the space of gain matrices which is required in [8].
318 citations
TL;DR: In this paper, the authors derived new estimates for the rate of convergence of the conjugate gradient method by utilizing isolated eigenvalues of parts of the spectrum and compared the derived estimates of the number of iterations with the number actually found for some elliptic difference equations and for a similar problem with a model empirical distribution function.
Abstract: We derive new estimates for the rate of convergence of the conjugate gradient method by utilizing isolated eigenvalues of parts of the spectrum. We present a new generalized version of an incomplete factorization method and compare the derived estimates of the number of iterations with the number actually found for some elliptic difference equations and for a similar problem with a model empirical distribution function.
285 citations
TL;DR: In this article, the convergence rates of estimates of the parametric and nonparametric components of the model under a particular assumption on the design were analyzed, and it was shown that the estimate of the non-parametric component is generally biased and that this bias can be larger than the standard error.
TL;DR: A fast maximum likelihood algorithm is presented that jointly estimates the frequency and frequency rate of a sinusoid corrupted by additive Gaussian white noise and consists of a coarse search and a fine search.
Abstract: A fast maximum likelihood algorithm is presented that jointly estimates the frequency and frequency rate of a sinusoid corrupted by additive Gaussian white noise It consists of a coarse search and a fine search First the two-dimensional frequency-frequency rate plane is subdivided into parallelograms whose size depends on the region of convergence of Newton's method used in maximizing the log-likelihood function (LLF) The size of the parallelogram is explicitly computed and is optimal for the method used The coarse search consists of maximizing the LLF over the vertices of the parallelograms Then starting at the vertex where the LLF attained its maximum, a two-dimensional Newton's method to find the absolute maximum of the LLF is implemented This last step consists of the fine search The rate of convergence of Newton's method is cubic, and is extremely fast Furthermore Newton's method will converge after two iterations when the starting point used in the method lies within 75 percent of the distances defined by the parallelogram of convergence whose center coincides with the true values of frequency and frequency rate In this case, the root mean square error (RMSEs) for frequency and frequency rate are practically equal to the Cramer-Rao bound at all signal-to-noise ratio (SNR)?15 dB The frequency-frequency rate ambiguity function is shown to be even and its periodicities are extracted
TL;DR: In this article, the authors analyzed the h-p version of the finite element method in two dimensions and showed that it leads to an exponential rate of convergence when solving problems with piecewise analytic data.
Abstract: The paper is the first of the series of two which analyses the h-p version of the finite element method in two dimensions. It proves the basic approximation results which in part 2 will be generalized and applied in a computational setting. The main result is that the h-p version leads to an exponential rate of convergence when solving problems with piecewise analytic data.
07 Apr 1986
TL;DR: Simulation of the algorithm shows that the RMSEs for frequency and frequency-rate are practically equal to the Cramer-Rao bound for allSNR \geq 15dB.
Abstract: A Fast Maximum Likelihood Algorithm for the Joint estimation of the Frequency and Frequency rate of a sinusoid in Gaussian white noise is presented here. The novelty of this algorithm is the use of Newton's method for finding the zero of a function to determine the maximum of the log-likelihood function. The rate of convergence of Newton's method is extremely fast. Simulation of the algorithm shows that the RMSEs for frequency and frequency-rate are practically equal to the Cramer-Rao bound for all SNR \geq 15 dB.
TL;DR: In this paper, a family of path-following methods based on consistent linearization is presented, where the scaling bypasses physical inconsistencies associated with mixed quantities like displacements and rotations in the global vector of the unknowns.
Abstract: This paper is focussed on path following methods which are derived from consistent linearizations. The linearization procedure leads to some well-known constraint equations—like the constant arc length in the load-displacement space—and to different formulations than those given in the literature. A full Newton scheme for the unknown quantities (displacements and load parameter) can be formulated. A comparison of the derived algorithms with other path following methods is included to show advantages and limits of the methods. Using the linearization technique together with scaling a family of path following methods is introduced. Here, the scaling bypasses physical inconsistencies associated with mixed quantities like displacements and rotations in the global vector of the unknowns. Several possible scaling procedures are derived from a unified formulation. A discussion of these methods by means of numerical examples shows that up to now the choice of the scaling procedure is problem-dependent. If the arc-length methods are combined with a modified Newton method, an enhancement of the algorithms is achieved by line search techniques. Here, a simple but efficient line search was implemented and compared with a numerical relaxation technique. Both methods improve the convergence rate considerably.
TL;DR: It is shown that the algorithm converges to the characteristic as the number of observations tend to infinity, for sufficiently smooth characteristics, the rate of convergence is O(n^{-2/5}) in probability.
Abstract: In this note a discrete-time Hammerstein system is identified. The weighting function of the dynamical subsystem is recovered by the correlation method. The main results concern estimation of the nonlinear memoryless subsystem. No conditions concerning functional form of the transform characteristic of the subsystem are made and an algorithm for estimation of the characteristic is presented.The algorithm is a nonparametric kernel estimate of regression functions calculated from dependent data. It is shown that the algorithm converges to the characteristic as the number of observations tend to infinity. For sufficiently smooth characteristics, the rate of convergence is O(n^{-2/5}) in probability.
TL;DR: It is proven that in the case that the solution has x sub alpha-type singularity, the adaptive algorithm give an exponential rate of convergence, very close to the optimal one analyzed in the second part of the paper.
Abstract: : The paper is the third and final part in the series of three devoted to the detailed analysis of the three basic versions of the finite element method in one dimension. The first part analyzed the p-version, and the second part concentrated on the h and h-p version. This paper analyzes a theoretical frame of the adaptive h-p version and based on it the authors provide concrete algorithm for the one dimensional problem. It is proven that in the case that the solution has x sub alpha-type singularity, the adaptive algorithm give an exponential rate of convergence, very close to the optimal one analyzed in the second part of the paper. Additional keyword: Error analysis.
TL;DR: A class of preconditioning methods depending on a relaxation parameter is presented for the solution of large linear systems of equationAx=b, whereA is a symmetric positive definite matrix.
Abstract: A class of preconditioning methods depending on a relaxation parameter is presented for the solution of large linear systems of equationAx=b, whereA is a symmetric positive definite matrix. The methods are based on an incomplete factorization of the matrixA and include both pointwise and blockwise factorization. We study the dependence of the rate of convergence of the preconditioned conjugate gradient method on the distribution of eigenvalues ofC−1A, whereC is the preconditioning matrix. We also show graphic representations of the eigenvalues and present numerical tests of the methods.
TL;DR: In this paper, the convergence rate of least square identification for the multidimensional ARMAX model is established under some decaying excitation conditions which are satisfied if both input and output do not grow too fast and the attenuating excitation technique is applied.
Abstract: The strong consistency and the convergence rate of least-squares identification for the multidimensional ARMAX model are established under some decaying excitation conditions which are satisfied if both input and output do not grow too fast and the attenuating excitation technique is applied. The parameter-identification results are applied to adaptive-control systems with a quadratic loss function. The rate of convergence of the loss function to its minimum is also obtained.
TL;DR: The problem of estimating time-varying harmonic components of a signal measured in noise is considered, and a new class of filters, akin to recursive frequency-sampling filters, is developed for inclusion in a parallel bank to produce sliding harmonic estimates.
Abstract: The problem of estimating time-varying harmonic components of a signal measured in noise is considered. The approach used is via state estimation. Two methods are proposed, one involving pole-placement of a state observer, the other using quadratic optimization techniques. The result is the development of a new class of filters, akin to recursive frequency-sampling filters, for inclusion in a parallel bank to produce sliding harmonic estimates. Kalman filtering theory is applied to effect the good performance in noise, and the class of filters is parameterized by the design tradeoff between noise rejection and convergence rate. These filters can be seen as generalizing the DFT.
TL;DR: In this article, a spectral moments code with constraints (MOMCON) was proposed to compute three-dimensional ideal magnetohydrodynamic (MHD) equilibria in a fixed toroidal domain using a Fourier expansion for the inverse coordinates representing nested magnetic surfaces.
TL;DR: In this article, a multigrid technique was used for steady multidimensional fluid flow analysis in four laminar two-dimensional recirculating flows representative of those in industrial equipment.
Abstract: The performance of a recently developed calculation procedure for steady multidimensional fluid flows is assessed in four laminar two-dimensional recirculating flows representative of those in industrial equipment. The calculation procedure is based on a coupled solution of the momentum and continuity equations by the multigrid technique. The rate of convergence of the algorithm is critically assessed by varying the flow Reynolds number, the number of finite difference nodes, and the numerical underrelaxation factor. The procedure is observed to converge rapidly to an acceptable accuracy in all the flow situations.
TL;DR: The results help to explain why the DFP method is often less suitable than the BFGS algorithm for general unconstrained optimization calculations, and they show that quadratic functions provide much information about efficiency when the current vector of variables is too far from the solution for an asymptotic convergence analysis.
Abstract: We study the use of the BFGS and DFP algorithms with step-lengths of one for minimizing quadratic functions of only two variables. The updating formulae in this case imply nonlinear three term recurrence relations between the eigenvalues of consecutive second derivative approximations, which are analysed in order to explain some gross inefficiencies that can occur. Specifically, the BFGS algorithm may require more than 10 iterations to achieve the first decimal place of accuracy, while the performance of the DFP method is far worse. The results help to explain why the DFP method is often less suitable than the BFGS algorithm for general unconstrained optimization calculations, and they show that quadratic functions provide much information about efficiency when the current vector of variables is too far from the solution for an asymptotic convergence analysis.
TL;DR: A local convergence property is proved which shows that whenever an FCM algorithm is started sufficiently near a minimizer of the corresponding objective function, then the iteration sequence must converge to that particular minimizer.
TL;DR: In this article, a conservative zonal boundary condition that was used with explicit integration schemes is extended to implicit, upwind, relaxation schemes; in particular to the Osher scheme, and the rate of convergence was found to increase considerably with the use of the implicit, relaxation-zonal scheme when compared to the explicit scheme.
TL;DR: In this article, it is shown that the initial-boundary value problem for Burgers' equation converges in time to a unique steady state, but the speed of convergence depends on the boundary conditions and can be exponentially slow.
TL;DR: In this article, the authors focus on the analysis of numerical techniques for solving the nonlinear system and on the theoretical issues that arise when certain extensions of the convex interpolation problem are considered.
Abstract: Numerical and theoretical questions related to constrained interpolation and smoothing are treated. The prototype problem is that of finding the smoothest convex interpolant to given univariate data. Recent results have shown that this convex programming problem with infinite constraints can be recast as a finite parametric nonlinear system whose solution is closely related to the second derivative of the desired interpolating function. This paper focuses on the analysis of numerical techniques for solving the nonlinear system and on the theoretical issues that arise when certain extensions of the problem are considered. In particular, we show that two standard iteration techniques, the Jacobi and Gauss-Seidel methods, are globally convergent when applied to this problem. In addition we use the problem structure to develop an efficient implementation of Newton's method and observe consistent quadratic convergence. We also develop a theory for the existence, uniqueness, and representation of solutions to the convex interpolation problem with nonzero lower bounds on the second derivative (strict convexity). Finally, a smoothing spline analogue to the convex interpolation problem is studied with reference to the computation of convex approximations to noisy data.
TL;DR: Both nonrecursive and recursive nonparametric regression estimates are studied and the rates of weak and strong convergence of kernel estimates, as well as corresponding multiple classification errors, are derived without assuming the existence of the density of the measurements.
Abstract: Both nonrecursive and recursive nonparametric regression estimates are studied. The rates of weak and strong convergence of kernel estimates, as well as corresponding multiple classification errors, are derived without assuming the existence of the density of the measurements. An application of the obtained results to a nonparametric Bayes predication is presented.
TL;DR: A simple Newton-like descent algorithm for linear programming is proposed together with results of preliminary computational experiments on small- and medium-size problems and, experimentally, shows global linear convergence.
Abstract: A simple Newton-like descent algorithm for linear programming is proposed together with results of preliminary computational experiments on small- and medium-size problems. The proposed algorithm gives local superlinear convergence to the optimum and, experimentally, shows global linear convergence. It is similar to Karmarkar's algorithm in that it is an interior feasible direction method and self-correcting, while it is quite different from Karmarkar's in that it gives superlinear convergence and that no artificial extra constraint is introduced nor is protective geometry needed, but only affine geometry suffices.
TL;DR: This work introduces a new multigrid continuation method for computing solutions of nonlinear elliptic eigenvalue problems which contain limit points (also called turning points or folds), which produces considerable storage savings over direct continuation methods, as well as better initial coarse grid approximations, and avoids complicated algorithms for determining the parameter on finer grids.
Abstract: We introduce a new multigrid continuation method for computing solutions of nonlinear elliptic eigenvalue problems which contain limit points (also called turning points or folds). Our method combines the frozen tau technique of Brandt with pseudo-arc length continuation and correction of the parameter on the coarsest grid. This produces considerable storage savings over direct continuation methods,as well as better initial coarse grid approximations, and avoids complicated algorithms for determining the parameter on finer grids. We provide numerical results for second, fourth and sixth order approximations to the two-parameter, two-dimensional stationary reaction-diffusion problem:\[ \Delta u + \lambda \exp (u/(1 + \alpha u)) = 0. \]For the higher order interpolations we use bicubic and biquintic splines. The convergence rate is observed to be independent of the occurrence of limit points.