Showing papers on "Rate of convergence published in 2001"
TL;DR: In this paper, a methodology to model arbitrary holes and material interfaces (inclusions) without meshing the internal boundaries is proposed, which couples the level set method with the extended finite element method (X-FEM).
1,112 citations
TL;DR: A number of variants of incremental subgradient methods for minimizing a convex function that consists of the sum of a large number of component functions are established, including some that are stochastic.
Abstract: We consider a class of subgradient methods for minimizing a convex function that consists of the sum of a large number of component functions. This type of minimization arises in a dual context from Lagrangian relaxation of the coupling constraints of large scale separable problems. The idea is to perform the subgradient iteration incrementally, by sequentially taking steps along the subgradients of the component functions, with intermediate adjustment of the variables after processing each component function. This incremental approach has been very successful in solving large differentiable least squares problems, such as those arising in the training of neural networks, and it has resulted in a much better practical rate of convergence than the steepest descent method.
In this paper, we establish the convergence properties of a number of variants of incremental subgradient methods, including some that are stochastic. Based on the analysis and computational experiments, the methods appear very promising and effective for important classes of large problems. A particularly interesting discovery is that by randomizing the order of selection of component functions for iteration, the convergence rate is substantially improved.
611 citations
Book•
09 Oct 2001
TL;DR: Investigating stochastic models for simulation, and basic morphological concepts, and Gaussian variations: some basic notions.
Abstract: 1. Introduction.- 2. Investigating stochastic models.- 3. Variographic tools.- 4. The integral range.- 5. Basic morphological concepts.- 6. Stereology: some basic notions.- 7. Basics about simulations.- 8. Iterative algorithms for simulation.- 9. Rate of convergence of iterative algorithms.- 10. Exact simulations.- 11. Point processes.- 12. Tessellations.- 13. Boolean model.- 14. Object based models.- 15. Gaussian random function.- 16. Gaussian variations.- 17. Substitution random functions.
509 citations
TL;DR: The main result of the paper is the construction of an adaptive scheme which produces an approximation to u with error O(N -s ) in the energy norm, whenever such a rate is possible by N-term approximation.
Abstract: This paper is concerned with the construction and analysis of wavelet-based adaptive algorithms for the numerical solution of elliptic equations. These algorithms approximate the solution u of the equation by a linear combination of N wavelets, Therefore, a benchmark for their performance is provided by the rate of best approximation to u by an arbitrary linear combination of N wavelets (so called N-term approximation), which would be obtained by keeping the N largest wavelet coefficients of the real solution (which of course is unknown). The main result of the paper is the construction of an adaptive scheme which produces an approximation to u with error O(N -s ) in the energy norm, whenever such a rate is possible by N-term approximation. The range of s > 0 for which this holds is only limited by the approximation properties of the wavelets together with their ability to compress the elliptic operator. Moreover, it is shown that the number of arithmetic operations needed to compute the approximate solution stays proportional to N. The adaptive algorithm applies to a wide class of elliptic problems and wavelet bases. The analysis in this paper puts forward new techniques for treating elliptic problems as well as the linear systems of equations that arise from the wavelet discretization.
488 citations
TL;DR: The hp-version of the discontinuous Galerkin finite element method for second-order partial differential equations with nonnegative characteristic form is considered, and an hp-optimal error bound is derived in the hyperbolic case and in the self-adjoint elliptic case.
Abstract: We consider the hp-version of the discontinuous Galerkin finite element method (DGFEM) for second-order partial differential equations with nonnegative characteristic form. This class of equations includes second-order elliptic and parabolic equations, advection-reaction equations, as well as problems of mixed hyperbolic-elliptic-parabolic type. Our main concern is the error analysis of the method in the absence of streamline-diffusion stabilization. In the hyperbolic case, an hp-optimal error bound is derived; here, we consider only advection-reaction problems which satisfy a certain (standard) positivity condition. In the self-adjoint elliptic case, an error bound that is h-optimal and p-suboptimal by $\frac{1}{2}$ a power of p is obtained. These estimates are then combined to deduce an error bound in the general case. For elementwise analytic solutions the method exhibits exponential rates of convergence under p-refinement. The theoretical results are illustrated by numerical experiments.
440 citations
TL;DR: A streamlined and fully three-dimensional version of the quasicontinuum (QC) theory of Tadmor et al is presented and the effect of the summation rules on accuracy, rate of convergence and refinement tolerance are analyzed.
Abstract: The aim of this paper is to present a streamlined and fully three-dimensional version of the quasicontinuum (QC) theory of Tadmor et al. (Philos. Mag. A 73 (1996) 1529; Langmuir 12 (1996) 4529) and to analyze its accuracy and convergence characteristics. Specifically, we assess the effect of the summation rules on accuracy; we determine the rate of convergence of the method in the presence of strong singularities, such as point loads; and we assess the effect of the refinement tolerance, which controls the rate at which new nodes are inserted in the model, on the development of dislocation microstructures.
404 citations
TL;DR: In this paper, two modified fast Fourier transform methods were proposed to handle composites with high contrast (typically above 104) or infinite contrast (those containing voids or rigid inclusions or highly non-linear materials).
Abstract: A numerical method making use of fast Fourier transforms has been proposed in Moulinec and Suquet (1994, 1998) to investigate the effective properties of linear and non-linear composites. This method is based on an iterative scheme the rate of convergence of which is proportional to the contrast between the phases. Composites with high contrast (typically above 104) or infinite contrast (those containing voids or rigid inclusions or highly non-linear materials) cannot be handled by the method. This paper presents two modified schemes. The first one is an accelerated scheme for composites with high contrast which extends to elasticity a scheme initially proposed in Eyre and Milton (1999). Its rate of convergence varies as the square root of the contrast. The second scheme, adequate for composites with infinite contrast, is based on an augmented Lagrangian method. The resulting saddle-point problem involves three steps. The first step consists of solving a linear elastic problem, using the fast Fourier transform method. In the second step, a non-linear problem is solved at each individual point in the volume element. The third step consists of updating the Lagrange multiplier. Applications of this scheme to rigidly reinforced and to voided composites are shown. Copyright © 2001 John Wiley & Sons, Ltd.
398 citations
01 Jan 2001
TL;DR: In this article, the authors consider a system of nonlinear equations F(x) = 0, where F is a mapping from Rn into Rm, and show that LMM has a quadratic rate of convergence when m = n, the Jacobian matrix of F is nonsingular at a solution x and an initial point is chosen sufficiently close to x.
Abstract: We consider a rate of convergence of the Levenberg-Marquardt method (LMM) for solving a system of nonlinear equations F(x) = 0, where F is a mapping from Rn into Rm. It is well-known that LMM has a quadratic rate of convergence when m = n, the Jacobian matrix of F is nonsingular at a solution x and an initial point is chosen sufficiently close to x. In this paper, we show that if
376 citations
TL;DR: The convergence of a penalty method for solving the discrete regularized American option valuation problem is studied and it is observed that an implicit treatment of the American constraint does not converge quadratically if constant timesteps are used.
Abstract: The convergence of a penalty method for solving the discrete regularized American option valuation problem is studied. Sufficient conditions are derived which both guarantee convergence of the nonlinear penalty iteration and ensure that the iterates converge monotonically to the solution. These conditions also ensure that the solution of the penalty problem is an approximate solution to the discrete linear complementarity problem. The efficiency and quality of solutions obtained using the implicit penalty method are compared with those produced with the commonly used technique of handling the American constraint explicitly. Convergence rates are studied as the timestep and mesh size tend to zero. It is observed that an implicit treatment of the American constraint does not converge quadratically (as the timestep is reduced) if constant timesteps are used. A timestep selector is suggested which restores quadratic convergence.
319 citations
TL;DR: Results with twist averaged variational Monte Carlo on free particles, the Stoner model and the electron gas are shown using Hartree-Fock, Slater-Jastrow, and three-body and backflow wave function.
Abstract: We develop and test Quantum Monte Carlo algorithms that use a``twist'' or a phase in the wave function for fermions in periodic boundary conditions. For metallic systems, averaging over the twist results in faster convergence to the thermodynamic limit than periodic boundary conditions for properties involving the kinetic energy and has the same computational complexity. We determine exponents for the rate of convergence to the thermodynamic limit for the components of the energy of coulomb systems. We show results with twist averaged variational Monte Carlo on free particles, the Stoner model and the electron gas using Hartree-Fock, Slater-Jastrow, and three-body and backflow wave function. We also discuss the use of twist averaging in the grand canonical ensemble, and numerical methods to accomplish the twist averaging.
317 citations
TL;DR: The run method converges slowly but can withstand blocks as well as a high proportion of isolated outliers and the rate of convergence of the taut-string multiresolution method is almost optimal.
Abstract: The paper considers the problem of nonparametric regression with emphasis on controlling the number of local extremes. Two methods, the run method and the taut-string multiresolution method, are introduced and analyzed on standard test beds. It is shown that the number and locations of local extreme values are consistently estimated. Rates of convergence are proved for both methods. The run method converges slowly but can withstand blocks as well as a high proportion of isolated outliers. The rate of convergence of the taut-string multiresolution method is almost optimal. The method is extremely sensitive and can detect very low power peaks. Section 1 contains an introduction with special reference to the number of local extreme values. The run method is described in Section 2 and the taut-string-multiresolution method in Section 3. Low power peaks are considered in Section 4. Section contains a comparison with other methods and Section 6 a short conclusion. The proofs are given in Section 7 and the taut-string algorithm is described in the Appendix.
01 Jan 2001
TL;DR: An incremental approach to minimizing a convex function that consists of the sum of a large number of component functions is considered, which has been very successful in solving large differentiable least squares problems, such as those arising in the training of neural networks.
Abstract: We consider a class of subgradient methods for minimizing a convex function that consists of the sum of a large number of component functions. This type of minimization arises in a dual context from Lagrangian relaxation of the coupling constraints of large scale separable problems. The idea is to perform the subgradient iteration incrementally, by sequentially taking steps along the subgradients of the component functions, with intermediate adjustment of the variables after processing each component function. This incremental approach has been very successful in solving large differentiable least squares problems, such as those arising in the training of neural networks, and it has resulted in a much better practical rate of convergence than the steepest descent method.
TL;DR: The error estimates and counterexamples in this paper will confirm that the FVE method cannot have the standard O(h2) convergence rate in the L2 norm when the source term has the minimum regularity, only being in L2, even if the exact solution is in H2.
Abstract: We present a general error estimation framework for a finite volume element (FVE) method based on linear polynomials for solving second-order elliptic boundary value problems. This framework treats the FVE method as a perturbation of the Galerkin finite element method and reveals that regularities in both the exact solution and the source term can affect the accuracy of FVE methods. In particular, the error estimates and counterexamples in this paper will confirm that the FVE method cannot have the standard O(h2) convergence rate in the L2 norm when the source term has the minimum regularity, only being in L2, even if the exact solution is in H2.
TL;DR: This work modify the standard algorithm by adding GMRES iterations at coarse levels and as an outer iteration to produce an algorithm whose performance depends relatively mildly on wave number and is robust for normalized wave numbers as large as 200.
Abstract: Standard multigrid algorithms have proven ineffective for the solution of discretizations of Helmholtz equations. In this work we modify the standard algorithm by adding GMRES iterations at coarse levels and as an outer iteration. We demonstrate the algorithm's effectiveness through theoretical analysis of a model problem and experimental results. In particular, we show that the combined use of GMRES as a smoother and outer iteration produces an algorithm whose performance depends relatively mildly on wave number and is robust for normalized wave numbers as large as 200. For fixed wave numbers, it displays grid-independent convergence rates and has costs proportional to the number of unknowns.
01 May 2001
TL;DR: In this article, Component averaging (CAV) is introduced as a new iterative parallel technique suitable for large and sparse unstructured systems of linear equations, which simultaneously projects the current iterate onto all the system's hyperplanes, and is thus inherently parallel.
Abstract: Component averaging (CAV) is introduced as a new iterative parallel technique suitable for large and sparse unstructured systems of linear equations. It simultaneously projects the current iterate onto all the system's hyperplanes, and is thus inherently parallel. However, instead of orthogonal projections and scalar weights (as used, for example, in Cimmino's method), it uses oblique projections and diagonal weighting matrices, with weights related to the sparsity of the system matrix. These features provide for a practical convergence rate which approaches that of algebraic reconstruction technique (ART) (Kaczmarz's row-action algorithm) – even on a single processor. Furthermore, the new algorithm also converges in the inconsistent case. A proof of convergence is provided for unit relaxation, and the fast convergence is demonstrated on image reconstruction problems of the Herman head phantom obtained within the SNARK93 image reconstruction software package. Both reconstructed images and convergence plots are presented. The practical consequences of the new technique are far reaching for real-world problems in which iterative algorithms are used for solving large, sparse, unstructured and often inconsistent systems of linear equations.
TL;DR: In this article, the weak Poincare inequality is introduced to describe L 2 -convergence rates slower than exponential, and conditions for the weak poincare inequalities to hold are presented.
15 Jul 2001
TL;DR: The modified Levenberg-Marquardt algorithm for feedforward neural networks gives a better convergence rate compared to the standard LM method and is less computationally intensive and requires less memory.
Abstract: In this work, two modifications on Levenberg-Marquardt (LM) algorithm for feedforward neural networks are studied. One modification is made on performance index, while the other one is on calculating gradient information. The modified algorithm gives a better convergence rate compared to the standard LM method and is less computationally intensive and requires less memory. The performance of the algorithm has been checked on several example problems.
09 Sep 2001
TL;DR: Convergence rate, widely used in the arena of the finite element method for evaluating new element’s performance, was exploited in this vehicle impact example.
Abstract: Response surface methods or metamodels are commonly used to approximate large computationally expensive engineering systems. Five response surface methods are studied: Stepwise Regression, Moving Least Square, Kriging, Multiquadric, and Adaptive and Interactive Modeling System. A real-world frontal impact design problem is used as an example, which is a complex, highly nonlinear, transient, dynamic, large deformation finite element model. To study the accuracy of the metamodel, the optimal Latin Hypercube Sampling method is used to distribute the sampling points uniformly over the entire design space. The Root Mean Square Error (RMSE) is used as the error indicator. Convergence rate, widely used in the arena of the finite element method for evaluating new element’s performance, was exploited in this vehicle impact example.
TL;DR: In this article, the authors studied the time-asymptotic behavior of solutions for the isentropic Euler equations with damping in multi-dimensions and obtained the global existence and pointwise estimates of the solutions.
TL;DR: The extent to which this order of convergence can be improved is investigated, and it is found that better approximations are possible for the case of additive noise, but for multiplicative noise it is shown that no improvements are possible.
Abstract: We consider the numerical solution of the stochastic partial differential equation ∂u/∂t = ∂ 2 u/∂x 2 + σ(u)W(x,t), where W is space-time white noise, using finite differences. For this equation Gyongy has obtained an estimate of the rate of convergence for a simple scheme, based on integrals of W over a rectangular grid. We investigate the extent to which this order of convergence can be improved, and find that better approximations are possible for the case of additive noise (σ(u) = 1) if we wish to estimate space averages of the solution rather than pointwise estimates, or if we are permitted to generate other functionals of the noise. But for multiplicative noise (σ(u) = u) we show that no such improvements are possible.
TL;DR: In this article, a new class of robust and efficient methods for solving subproblems that arise in the linearization and operator splitting of Navier-Stokes equations are presented.
TL;DR: The Newton and quasi-Newton methods as well as various variants of SQP methods are developed for applications to optimal flow control, and their complexity in terms of system solves is discussed.
Abstract: Second order methods for open loop optimal control problems governed by the two-dimensional instationary Navier--Stokes equations are investigated Optimality systems based on a Lagrangian formulation and adjoint equations are derived The Newton and quasi-Newton methods as well as various variants of SQP methods are developed for applications to optimal flow control, and their complexity in terms of system solves is discussed Local convergence and rate of convergence are proved A numerical example illustrates the feasibility of solving optimal control problems for two-dimensional instationary Navier--Stokes equations by second order numerical methods in a standard workstation environment
TL;DR: A systematic approach to constructing high-order tangential vector basis functions for the multilevel finite element solution of electromagnetic wave problems and shows that number of iterations needed for the solution by MPCG is basically constant, regardless of the order of the basis or of the matrix size.
Abstract: This paper presents a systematic approach to constructing high-order tangential vector basis functions for the multilevel finite element solution of electromagnetic wave problems. The new bases allow easy computation of a preconditioner to eliminate or at least weaken the indefiniteness of the system matrix and thus reduce the condition number of the system matrix. When these bases are used in multilevel solutions, where the multilevels correspond to the order of the basis functions, the resulting p-multilevel-ILU preconditioned conjugate gradient method (MPCG) provides an optimal rate of convergence. We first derive an admissible set of vectors of order p, and decompose this set into two subspaces---rotational and irrotational (gradient). We then reduce the number of vectors by making them orthogonal to all previously constructed lower-order bases. The remaining vectors are made mutually orthogonal in both the vector space and in the range space of the curl operator. The resulting vector basis functions provide maximum orthogonality while maintaining tangential continuity of the field. The zeroth-order space is further decomposed using a scalar-vector formulation to eliminate convergence problems at extremely low frequencies. Numerical experiments show that number of iterations needed for the solution by MPCG is basically constant, regardless of the order of the basis or of the matrix size. Computational speed is improved by several orders of magnitude due to the fast matrix solution of MPCG and to the high accuracy of the higher-order bases.
TL;DR: It is shown that under some conditions an iteration method converges to a positive definite solution of a set of equations of the form X+A^{\star}{\cal F}(X)A =Q, where A is arbitrary and Q is apositive definite matrix.
Abstract: This paper treats a set of equations of the form $X+A^{\star}{\cal F}(X)A =Q$, where ${\cal F}$ maps positive definite matrices either into positive definite matrices or into negative definite matrices, and satisfies some monotonicity property. Here A is arbitrary and Q is a positive definite matrix. It is shown that under some conditions an iteration method converges to a positive definite solution. An estimate for the rate of convergence is given under additional conditions, and some numerical results are given. Special cases are considered, which cover also particular cases of the discrete algebraic Riccati equation.
TL;DR: In this paper, the authors developed an asymptotic theory for nonlinear least squares estimators, extending earlier work by Phillips and Hansen (1990) on linear cointegrating regressions.
Abstract: Summary This paper develops an asymptotic theory for a general class of nonlinear nonstationary regressions, extending earlier work by Phillips and Hansen (1990) on linear cointegrating regressions.The model considered accommodates a linear time trend and stationary regressors, as well as multiple I(1) regressors. We establish consistency and derive the limit distribution of the nonlinear least squares estimator. The estimator is consistent under fairly general conditions but the convergence rate and the limiting distribution are critically dependent upon the type of the regression function. For integrable regression functions, the parameter estimates converge at a reduced n 1/4 rate and have mixed normal limit distributions. On the other hand, if the regression functions are homogeneous at infinity, the convergence rates are determined by the degree of the asymptotic homogeneity and the limit distributions are non-Gaussian. It is shown that nonlinear least squares generally yields inefficient estimators and invalid tests, just as in linear nonstationary regressions. The paper proposes a methodology to overcome such difficulties. The approach is simple to implement, produces efficient estimates and leads to tests that are asymptotically chi-square. It is implemented in empirical applications in much the same way as the fully modified estimator of Phillips and Hansen.
21 Oct 2001
TL;DR: A new algorithm for fitting a Catmull-Clark subdivision surface to a given shape within a prescribed tolerance, based on the method of quasi-interpolation is introduced, which is fast, local and scales well since it does not require the solution of linear systems.
Abstract: We introduce a new algorithm for fitting a Catmull-Clark subdivision surface to a given shape within a prescribed tolerance, based on the method of quasi-interpolation. The fitting algorithm is fast, local and scales well since it does not require the solution of linear systems. Its convergence rate is optimal for regular meshes and our experiments show that it behaves very well for irregular meshes. We demonstrate the power and versatility of our method with examples from interactive modeling, surface fitting, and scientific visualization.
TL;DR: In this paper, an analytical proof of the second order convergence of the Multilevel-Newton algorithm is given by authors in the field of non-linear electrical networks, which can be applied in the current context based on the DAE interpretation mentioned above.
Abstract: For the numerical solution of materially non-linear problems like in computational plasticity or viscoplasticity the finite element discretization in space is usually coupled with point-wise defined evolution equations characterizing the material behaviour. The interpretation of such systems as differential–algebraic equations (DAE) allows modern-day integration algorithms from Numerical Mathematics to be efficiently applied. Especially, the application of diagonally implicit Runge–Kutta methods (DIRK) together with a Multilevel-Newton method preserves the algorithmic structure of current finite element implementations which are based on the principle of virtual displacements and on backward Euler schemes for the local time integration. Moreover, the notion of the consistent tangent operator becomes more obvious in this context. The quadratical order of convergence of the Multilevel-Newton algorithm is usually validated by numerical studies. However, an analytical proof of this second order convergence has already been given by authors in the field of non-linear electrical networks. We show that this proof can be applied in the current context based on the DAE interpretation mentioned above. We finally compare the proposed procedure to several well-known stress algorithms and show that the inclusion of a step-size control based on local error estimations merely requires a small extra time-investment. Copyright © 2001 John Wiley & Sons, Ltd.
TL;DR: In this paper, a model explicit fourth-order staggered finite-difference method for the hyperbolic Maxwell's equations is considered and an eigenvalue analysis of the overall scheme provides a necessary, but not sufficient, stability condition and indicates long time stability.
TL;DR: In this paper, the authors derived globally convergent multigrid methods for the discretized Signorini problem in linear elasticity and observed similar convergence rates as for corresponding linear problems.
Abstract: We derive globally convergent multigrid methods for the discretized Signorini problem in linear elasticity. Special care has to be taken in the case of spatially varying normal directions. In numerical experiments for 2 and 3 space dimensions we observed similar convergence rates as for corresponding linear problems.
TL;DR: The new asymptotic bounds for the rate of convergence are illustrated by discussing Toeplitz systems as well as a model problem stemming from the discretization of the Poisson equation.
Abstract: We give a theoretical explanation for superlinear convergence behavior observed while solving large symmetric systems of equations using the conjugate gradient method or other Krylov subspace methods. We present a new bound on the relative error after $n$ iterations. This bound is valid in an asymptotic sense when the size $N$ of the system grows together with the number of iterations. The bound depends on the asymptotic eigenvalue distribution and on the ratio $n/N$. Under appropriate conditions we show that the bound is asymptotically sharp.
Our findings are related to some recent results concerning asymptotics of discrete orthogonal polynomials. An important tool in our investigations is a constrained energy problem in logarithmic potential theory.
The new asymptotic bounds for the rate of convergence are illustrated by discussing Toeplitz systems as well as a model problem stemming from the discretization of the Poisson equation.