Showing papers on "Rate of convergence published in 1993"
TL;DR: In this paper, a trust region approach for minimizing nonlinear functions subject to simple bounds is proposed, where the trust region is defined by minimizing a quadratic function subject only to an ellipsoidal constraint and the iterates generated by these methods are always strictly feasible.
Abstract: We propose a new trust region approach for minimizing nonlinear functions subject to simple bounds. By choosing an appropriate quadratic model and scaling matrix at each iteration, we show that it is not necessary to solve a quadratic programming subproblem, with linear inequalities, to obtain an improved step using the trust region idea. Instead, a solution to a trust region subproblem is defined by minimizing a quadratic function subject only to an ellipsoidal constraint. The iterates generated by these methods are always strictly feasible. Our proposed methods reduce to a standard trust region approach for the unconstrained problem when there are no upper or lower bounds on the variables. Global and quadratic convergence of the methods is established; preliminary numerical experiments are reported.
3,026 citations
TL;DR: This article surveys iterative domain decomposition techniques that have been developed in recent years for solving several kinds of partial differential equations, including elliptic, parabolic, and differential systems such as the Stokes problem and mixed formulations of elliptic problems.
Abstract: Domain decomposition (DD) has been widely used to design parallel efficient algorithms for solving elliptic problems. In this thesis, we focus on improving the efficiency of DD methods and applying them to more general problems. Specifically, we propose efficient variants of the vertex space DD method and minimize the complexity of general DD methods. In addition, we apply DD algorithms to coupled elliptic systems, singular Neumann boundary problems and linear algebraic systems.
We successfully improve the vertex space DD method of Smith by replacing the exact edge, vertex dense matrices by approximate sparse matrices. It is extremely expensive to calculate, invert and store the exact vertex and edge Schur complement dense sub-matrices in the vertex space DD algorithm. We propose several approximations for these dense matrices, by using Fourier approximation and an algebraic probing technique. Our numerical and theoretical results show that these variants retain the fast convergence rate and greatly reduce the computational cost.
We develop a simple way to reduce the overall complexity of domain decomposition methods through choosing the coarse grid size. For sub-domain solvers with different complexities, we derive the optimal coarse grid size $H\sb{opt},$ which asymptotically minimizes the total computational cost of DD methods under the sequential and parallel environments. The overall complexity of DD methods is significantly reduced by using this optimal coarse grid size.
We apply the additive and multiplicative Schwarz algorithms to solving coupled elliptic systems. Using the Dryja-Widlund framework, we prove that their convergence rates are independent of both the mesh and the coupling parameters. We also construct several approximate interface sparse matrices by using Sobolev inequalities, Fourier analysis and probe technique.
We further discuss the application of DD to the singular Neumann boundary value problems. We extend the general framework to these problems and show how to deal with the null space in practice. Numerical and theoretical results show that these modified DD methods still have optimal convergence rate.
By using the DD methodology, we propose algebraic additive and multiplicative Schwarz methods to solve general sparse linear algebraic systems. We analyze the eigenvalue distribution of the iterative matrix of each algebraic DD method to study the convergence behavior.
550 citations
TL;DR: An alternative convergence proof of a proximal-like minimization algorithm using Bregman functions, recently proposed by Censor and Zenios, is presented and allows the establishment of a global convergence rate of the algorithm expressed in terms of function values.
Abstract: An alternative convergence proof of a proximal-like minimization algorithm using Bregman functions, recently proposed by Censor and Zenios, is presented. The analysis allows the establishment of a global convergence rate of the algorithm expressed in terms of function values.
481 citations
TL;DR: A general approach to analyzing the convergence and the rate of convergence of feasible descent methods that does not require any nondegeneracy assumption on the problem is surveyed and extended.
Abstract: We survey and extend a general approach to analyzing the convergence and the rate of convergence of feasible descent methods that does not require any nondegeneracy assumption on the problem. This approach is based on a certain error bound for estimating the distance to the solution set and is applicable to a broad class of methods.
477 citations
TL;DR: A new, simple algorithm for the matrix-geometric rate matrix has quadratic convergence and is shown theoretically and through numerical examples that it converges very fast and provides extremely accurate results even for almost unstable models.
Abstract: Quasi-birth-death processes are commonly used Markov chain models in queueing theory, computer performance, teletraffic modeling and other areas. We provide a new, simple algorithm for the matrix-geometric rate matrix. We demonstrate that it has quadratic convergence. We show theoretically and through numerical examples that it converges very fast and provides extremely accurate results even for almost unstable models.
377 citations
TL;DR: In this paper, an estimator of σ based on discrete observation of the diffusion X throughout a given finite time interval is proposed and the asymptotic behavior of this estimator when the step of discretization tends to zero.
Abstract: This paper is concerned with the problem of estimation for the diffusion coefficient of a diffusion process on R, in a non-parametric situation. The drift function can be unknown and considered as a nuisance parameter. We propose an estimator of σ based on discrete observation of the diffusion X throughout a given finite time interval. We describe the asymptotic behaviour of this estimator when the step of discretization tends to zero. We prove consistency and asymptotic normality, the rate of convergence to the normal law being a random variable linked to the local time of the diffusion or to its suitable discrete approximation. This can also be interpreted as a convergence to a mixture of normal law.
283 citations
Book•
01 Jun 1993TL;DR: In this article, the spectrum, resolvent, and power boundedness of the spectrum are considered, and the spectral mapping theorem is applied to the problem of polynomial acceleration.
Abstract: 1. Motivation, problem and notation.- 1.1 Motivation.- 1.2 Problem formulation.- 1.3 Usual tools.- 1.4 Notation for polynomial acceleration.- 1.5 Minimal error and minimal residual.- 1.6 Approximation of the solution operator.- 1.7 Location of zeros.- 1.8 Heuristics.- Comments to Chapter 1.- 2. Spectrum, resolvent and power boundedness.- 2.1 The spectrum.- 2.2 The resolvent.- 2.3 The spectral mapping theorem.- 2.4 Continuity of the spectrum.- 2.5 Equivalent norms.- 2.6 The Yosida approximation.- 2.7 Power bounded operators.- 2.8 Minimal polynomials and algebraic operators.- 2.9 Quasialgebraic operators.- 2.10 Polynomial numerical hull.- Comments to Chapter 2.- 3. Linear convergence.- 3.1 Preliminaries.- 3.2 Generating functions and asymptotic convergence factors.- 3.3 Optimal reduction factor.- 3.4 Green's function for G?.- 3.5 Optimal polynomials for.- 3.6 Simply connected G?(L).- 3.7 Stationary recursions.- 3.8 Simple examples.- Comments to Chapter 3.- 4. Sublinear convergence.- 4.1 Introduction.- 4.2 Convergence of Lk(L?1).- 4.3 Splitting into invariant subspaces.- 4.4 Uniform convergence.- 4.5 Nonisolated singularity and successive approximation.- 4.6 Nonisolated singularity and polynomial acceleration.- 4.7 Fractional powers of operators.- 4.8 Convergence of iterates.- 4.9 Convergence with speed.- Comments to Chapter 4.- 5. Superlinear convergence.- 5.1 What is superlinear.- 5.2 Introductory examples.- 5.3 Order and type.- 5.4 Finite termination.- 5.5 Lower and upper bounds for optimal polynomials.- 5.6 Infinite products.- 5.7 Almost algebraic operators.- 5.8 Estimates using singular values.- 5.9 Multiple clusters.- 5.10 Approximation with algebraic operators.- 5.11 Locally superlinear implies superlinear.- Comments to Chapter 5.- References.- Definitions.
275 citations
TL;DR: A conjugate gradient-like method is proposed which is applicable to symmetric indefinite problems, the effects of stabilisation on the algebraic structure of the discrete Stokes operator are described and estimates of the eigenvalue spectrum of this operator are derived on which the convergence rate of the iteration depends.
Abstract: Mixed finite element approximation of the classical Stokes problem describing slow viscous incompressible flow gives rise to symmetric indefinite systems for the discrete velocity and pressure variables. Iterative solution of such indefinite systems is feasible and is an attractive approach for large problems. The use of stabilisation methods for convenient (but unstable) mixed elements introduces stabilisation parameters. We show how these can be chosen to obtain rapid iterative convergence. We propose a conjugate gradient-like method (the method of preconditioned conjugate residuals) which is applicable to symmetric indefinite problems, describe the effects of stabilisation on the algebraic structure of the discrete Stokes operator and derive estimates of the eigenvalue spectrum of this operator on which the convergence rate of the iteration depends. Here we discuss the simple case of diagonal preconditioning. Our results apply to both locally and globally stabilised mixed elements as well as to elements which are inherently stable. We demonstrate that convergence rates comparable to that achieved using the diagonally scaled conjugate gradient method applied to the discrete Laplacian are approachable for the Stokes problem.
246 citations
TL;DR: A modified technique for calculating a direction in weight-space which decreases the error for each class is presented and the rate of learning for two-class classification problems is accelerated by an order of magnitude.
Abstract: The backpropagation algorithm converges very slowly for two-class problems in which most of the exemplars belong to one dominant class. An analysis shows that this occurs because the computed net error gradient vector is dominated by the bigger class so much that the net error for the exemplars in the smaller class increases significantly in the initial iteration. The subsequent rate of convergence of the net error is very low. A modified technique for calculating a direction in weight-space which decreases the error for each class is presented. Using this algorithm, the rate of learning for two-class classification problems is accelerated by an order of magnitude. >
235 citations
TL;DR: The authors' estimators are different from Joe's, and may be computed without numerical integration, but it can be shown that the same interaction of tail behaviour, smoothness and dimensionality also determines the convergence rate of Joe's estimator.
Abstract: Motivated by recent work of Joe (1989,Ann. Inst. Statist. Math.,41, 683–697), we introduce estimators of entropy and describe their properties. We study the effects of tail behaviour, distribution smoothness and dimensionality on convergence properties. In particular, we argue that root-n consistency of entropy estimation requires appropriate assumptions about each of these three features. Our estimators are different from Joe's, and may be computed without numerical integration, but it can be shown that the same interaction of tail behaviour, smoothness and dimensionality also determines the convergence rate of Joe's estimator. We study both histogram and kernel estimators of entropy, and in each case suggest empirical methods for choosing the smoothing parameter.
222 citations
TL;DR: In this paper, a simple dynamic model of rational learning through market interaction by asymmetrically informed risk-neutral agents, uncertain about a valuation parameter but whose pooled information reveals it, is presented.
Abstract: A simple dynamic model of rational learning through market interaction by asymmetrically informed risk-neutral agents, uncertain about a valuation parameter but whose pooled information reveals it, is presented. The model is a variation of the classical partial equilibrium model of learning in rational expectations in which the market price is informative about the unknown parameter only through the actions of agents. It is found that learning from market prices and convergence to the rational expectations equilibrium is slow, at the rate 1/v/ (where n is the number periods of market interaction), whenever the average precision of private information in the market is finite. Convergence obtains at the standard rate 1/v'i if there is a positive mass of perfectly informed agents. Comparative static results on more refined measures of the speed of convergence with respect to basic technological and informational parameters are also provided. In this paper I present a simple dynamic model of rational learning by asymmetrically informed risk-neutral agents, uncertain about a valuation parameter but whose pooled information reveals it, and examine what factors influence the speed of learning and the rate of convergence to rational expectations equilibria. Rational expectations models have been widely used in many applications. A rational expectations equilibrium entails the solution to a fixed-point problem from beliefs to correct beliefs with the mediation of agents' actions. How do agents come to form rational expectations in the presence of unknown payoff-relevant parameters? The usual answer is that agents learn to form "correct" expectations through repeated observations of market data. Taking for granted that a learning process converges to the rational expectations equilibrium it is of the utmost importance to know how fast is and what factors affect the speed of convergence. It is not much use for practical purposes to show convergence if it is not known whether this will happen quickly or will take a long time, when the underlying conditions of the economy will have changed and the parameters learned may well be irrelevant. In this sense "slow" convergence may mean no convergence. We would like to know also how structural market conditions, like the nature of uncertainty and its relation to market observables (namely, prices), the degree of asymmetric information, and the precision of private signals, affect the speed of convergence and in what direction. The literature on learning and convergence to rational expectations splits naturally
TL;DR: A Jacobi-like algorithm for simultaneous diagonalization of commuting pairs of complex normal matrices by unitary similarity transformations is presented, which preserves the special structure of real matrices, quaternion matrices and real symmetric matrices.
Abstract: A Jacobi-like algorithm for simultaneous diagonalization of commuting pairs of complex normal matrices by unitary similarity transformations is presented. The algorithm uses a sequence of similarity transformations by elementary complex rotations to drive the off-diagonal entries to zero. Its asymptotic convergence rate is shown to be quadratic and numerically stable. It preserves the special structure of real matrices, quaternion matrices, and real symmetric matrices.
TL;DR: In this article, the second largest eigenvalues in terms of the eigen values of a comparison chain with known eigenvalue were derived for the symmetric random walk on a finite group.
Abstract: We develop techniques for bounding the rate of convergence of a symmetric random walk on a finite group to the uniform distribution. The techniques gives bounds on the second largest (and other) eigenvalues in terms of the eigenvalues of a comparison chain with known eigenvalues. The techniques yield sharp rates for a host of previously intractable problems on the symmetric group.
TL;DR: This paper proposes to modify the Newton method for variational inequality problems by using a certain differentiable merit function to determine a suitable step length and shows that the method is globally convergent and the rate of convergence is quadratic.
Abstract: Variational inequality problems have been used to formulate and study equilibrium problems, which arise in many fields including economics, operations research and regional sciences. For solving variational inequality problems, various iterative methods such as projection methods and the nonlinear Jacobi method have been developed. These methods are convergent to a solution under certain conditions, but their rates of convergence are typically linear. In this paper we propose to modify the Newton method for variational inequality problems by using a certain differentiable merit function to determine a suitable step length. The purpose of introducing this merit function is to provide some measure of the discrepancy between the solution and the current iterate. It is then shown that, under the strong monotonicity assumption, the method is globally convergent and, under some additional assumptions, the rate of convergence is quadratic. Limited computational experience indicates the high efficiency of the proposed method.
TL;DR: In this paper, a pressure-dependent J2-flow theory is proposed for use within the framework of the Cosserat continuum, and the second invariant of the deviatoric stresses is generalised to include couple-stresses, and strain-hardening hypothesis of plasticity is extended to take account of micro-curvatures.
Abstract: A pressure-dependent J2-flow theory is proposed for use within the framework of the Cosserat continuum. To this end the definition of the second invariant of the deviatoric stresses is generalised to include couple-stresses, and the strain-hardening hypothesis of plasticity is extended to take account of micro-curvatures. The temporal integration of the resulting set of differential equations is achieved using an implicit Euler backward scheme. This return-mapping algorithm results in an exact satisfaction of the yield condition at the end of the loading step. Moreover, the integration scheme is amenable to exact linearisation, so that a quadratic rate of convergence is obtained when Newton's method is used. An important characteristic of the model is the incorporation of an internal length scale. In finite element simulations of localisation, this property warrants convergence of the load-deflection curve to a physically realistic solution upon mesh refinement and to a finite width of the localisation zone. This is demonstrated for an infinitely long shear layer and for a biaxial specimen composed of a strain-softening Drucker-Prager material.
TL;DR: It is proved that as soon as eigen values of the original operator are sufficiently well approximated by Ritz values, GMRES from then on converges at least as fast as for a related system in which these eigenvalues (and their eigenvector components) are missing.
TL;DR: In this article, the convergence rates of expected spectral distributions of large dimensional Wigner and sample covariance matrices are established using certain inequalities to bound the difference between distributions in terms of their Stieltjes transforms.
Abstract: In this paper, we shall develop certain inequalities to bound the difference between distributions in terms of their Stieltjes transforms. Using these inequalities, convergence rates of expected spectral distributions of large dimensional Wigner and sample covariance matrices are established. The paper is organized into two parts. This is the first part, which is devoted to establishing the basic inequalities and a convergence rate for Wigner matrices.
TL;DR: This paper analyzes the exponential method of multipliers for convex constrained minimization problems, which operates like the usual Augmented Lagrangian method, except that it uses an exponential penalty function in place of the usual quadratic.
Abstract: In this paper, we analyze the exponential method of multipliers for convex constrained minimization problems, which operates like the usual Augmented Lagrangian method, except that it uses an exponential penalty function in place of the usual quadratic We also analyze a dual counterpart, the entropy minimization algorithm, which operates like the proximal minimization algorithm, except that it uses a logarithmic/entropy “proximal” term in place of a quadratic We strengthen substantially the available convergence results for these methods, and we derive the convergence rate of these methods when applied to linear programs
TL;DR: In this article, the authors combine several novel techniques for spectrum simulation in the Eddington computer program which solves the comoving frame equation of transfer coupled with the statistical and radiative equilibrium equations.
Abstract: The study combines several novel techniques for spectrum simulation in the Eddington computer program which solves the comoving frame equation of transfer coupled with the statistical and radiative equilibrium equations. One of these is a generalization of the accelerated lambda iteration (ALI) scheme to include an approximate frequency-derivative operator. This greatly enhances the convergence rate of ALI in optically thick, high-velocity shear flows. Another is a partial linearization technique which is capable of efficiently solving a very large number of rate equations on a moderately sized computer. An expansion opacity and emissivity approximation is derived which makes it possible to determine the effect on the transfer and statistical equilibrium of a very large number of lines not explicitly represented in the frequency grid and additionally to treat line-blanketing from species not explicitly included in the rate equations. The utility of these techniques is illustrated with models of two supernovae.
TL;DR: In this article, an a-posteriors strategy for choosing the regularization parameter in Tikhonov regularization for solving nonlinear ill-posed problems is proposed and the convergence rate obtained with this strategy is optimal.
Abstract: The authors propose an a-posteriors strategy for choosing the regularization parameter in Tikhonov regularization for solving nonlinear ill-posed problems and show that under certain conditions, the convergence rate obtained with this strategy is optimal. As a by-product, a new stability estimate for the regularized solutions is given which applies to a class of parameter identification problems. The authors compare the parameter choice strategy with Morozov’s Discrepancy Principle. Finally, numerical results are presented.
TL;DR: A new and detailed analysis of the basic Uzawa algorithm for decoupling of the pressure and the velocity in the steady and unsteady Stokes operator is presented, focusing on explicit construction of the Uzawa pressure-operator spectrum for a semiperiodic model problem.
Abstract: A new and detailed analysis of the basic Uzawa algorithm for decoupling of the pressure and the velocity in the steady and unsteady Stokes operator is presented. The paper focuses on the following new aspects: explicit construction of the Uzawa pressure-operator spectrum for a semiperiodic model problem; general relationship of the convergence rate of the Uzawa procedure to classical inf-sup discretization analysis; and application of the method to high-order variational discretization.
TL;DR: To increase reconstruction speed, spatially invariant preconditioning filters that can be designed using the tomographic system response and implemented using 2-D frequency-domain filtering techniques have been applied.
Abstract: Because of the characteristics of the tomographic inversion problem, iterative reconstruction techniques often suffer from poor convergence rates-especially at high spatial frequencies. By using preconditioning methods, the convergence properties of most iterative methods can be greatly enhanced without changing their ultimate solution. To increase reconstruction speed, spatially invariant preconditioning filters that can be designed using the tomographic system response and implemented using 2-D frequency-domain filtering techniques have been applied. In a sample application, reconstructions from noiseless, simulated projection data, were performed using preconditioned and conventional steepest-descent algorithms. The preconditioned methods demonstrated residuals that were up to a factor of 30 lower than the assisted algorithms at the same iteration. Applications of these methods to regularized reconstructions from projection data containing Poisson noise showed similar, although not as dramatic, behavior. >
TL;DR: Two CMOS implementations of the variable-step-size, power-of-two quantizer algorithm are presented to demonstrate that the performance gains are attainable with only a modest increase in circuit complexity.
Abstract: Stochastic gradient adaptive filtering algorithms using variable step sizes are investigated. The variable-step-size algorithm improves the convergence rate while sacrificing little in steady-state error. Expressions describing the convergence of the mean and mean-squared values of the coefficients are developed and used to calculate the mean-square-error evolution. The initial convergence rate and the steady-state error are also investigated. The performance of the algorithm is studied when a power-of-two quantizer algorithm is used, and finite-word-length effects are considered. The analytical results are verified with simulations encompassing variable applications. Two CMOS implementations of the variable-step-size, power-of-two quantizer algorithm are presented to demonstrate that the performance gains are attainable with only a modest increase in circuit complexity. >
TL;DR: It is proved that the simple additive multilevel algorithm discussed recently together with J. Xu and the standard V-cycle algorithm with one smoothing step per grid have a uniform reduction per iteration independent of the mesh sizes and number of levels, even on nonconvex domains which do not provide full elliptic regularity.
Abstract: The purpose of this paper is to provide new estimates for certain multilevel algorithms. In particular, we are concerned with the simple additive multilevel algorithm discussed recently together with J. Xu and the standard V-cycle algorithm with one smoothing step per grid. We shall prove that these algorithms have a uniform reduction per iteration independent of the mesh sizes and number of levels, even on nonconvex domains which do not provide full elliptic regularity. For example, the theory applies to the standard multigrid Vcycle on the L-shaped domain, or a domain with a crack, and yields a uniform convergence rate. We also prove uniform convergence rates for the multigrid V-cycle for problems with nonuniformly refined meshes. Finally, we give a new multigrid approach for problems on domains with curved boundaries and prove a uniform rate of convergence for the corresponding multigrid V-cycle algorithms.
TL;DR: In this paper, the authors developed certain inequalities to bound the difference between distributions in terms of their Stieltjes transforms and established a convergence rate of expected spectral distributions of large Wigner matrices.
Abstract: In the first part of the paper, we develop certain inequalities to bound the difference between distributions in terms of their Stieltjes transforms and established a convergence rate of expected spectral distributions of large Wigner matrices. The second part is devoted to establishing convergence rates for the sample covariance matrices, for the cases where the ratio of the dimension to the degrees of freedom is bounded away from 1 or close to 1, respectively.
TL;DR: This surprising result is the first instance of a demonstration of polynomiality and superlinear (or quadratic) convergence for an interior-point algorithm which does not assume the convergence of the iteration sequence or nondegeneracy.
Abstract: Recently, Ye, Tapia and Zhang (1991) demonstrated that Mizuno--Todd--Ye's predictor--corrector interior-point algorithm for linear programming maintains the O( $$\sqrt n $$ L)-iteration complexity while exhibiting superlinear convergence of the duality gap to zero under the assumption that the iteration sequence converges, and quadratic convergence of the duality gap to zero under the assumption of nondegeneracy. In this paper we establish the quadratic convergence result without any assumption concerning the convergence of the iteration sequence or nondegeneracy. This surprising result, to our knowledge, is the first instance of a demonstration of polynomiality and superlinear (or quadratic) convergence for an interior-point algorithm which does not assume the convergence of the iteration sequence or nondegeneracy.
TL;DR: This paper proves by example that the existence of a strictly complementarity solution appears to be necessary to achieve superlinear convergence for the predictor—corrector algorithm for linear programming (LP), assuming only that a strictly complementary solution exists.
Abstract: Recently several new results have been developed for the asymptotic (local) convergence of polynomial-time interior-point algorithms. It has been shown that the predictor—corrector algorithm for linear programming (LP) exhibits asymptotic quadratic convergence of the primal—dual gap to zero, without any assumptions concerning nondegeneracy, or the convergence of the iteration sequence. In this paper we prove a similar result for the monotone linear complementarity problem (LCP), assuming only that a strictly complementary solution exists. We also show by example that the existence of a strictly complementarity solution appears to be necessary to achieve superlinear convergence for the algorithm.
TL;DR: In this paper, an application of the return mapping algorithm for the inviscid two invariant cap model, originally proposed by DiMaggio and Sandler, is presented.
TL;DR: In this article, the authors derive algorithms for the iterative reconstruction of discrete band-limited signals from their irregular samples at a geometric rate, and all constants are computable explicitly.
TL;DR: In this article, the authors consider the problem of stabilizing an uncertain system when the norm of the control input is bounded by a prespecified constant and present controllers which guarantee that, for all allowable uncertainties and nonlinearities, there is a region of attraction from which all solutions converge to the given ball with the pre-specified convergence rate.
Abstract: We consider the problem of stabilizing an uncertain system when the norm of the control input is bounded by a prespecified constant. We treat continuous-time dynamical systems whose nominal part is linear and whose uncertain part is norm-bounded by a known affine function of the norm of the system state and the norm of the control input. Given a prespecified rate of convergence and a ball containing the origin of the state space, we present controllers which guarantee that, for all allowable uncertainties and nonlinearities, there is a region of attraction from which all solutions converge to the given ball with the prespecified convergence rate.