Showing papers on "Rate of convergence published in 1987"

An efficient newton‐like method for molecular mechanics energy minimization of large molecules

Abstract: Techniques from numerical analysis and crystallographic refinement have been combined to produce a variant of the Truncated Newton nonlinear optimization procedure. The new algorithm shows particular promise for potential energy minimization of large molecular systems. Usual implementations of Newton's method require storage space proportional to the number of atoms squared (i.e., O(N2)) and computer time of O(N3). Our suggested implementation of the Truncated Newton technique requires storage of less than O(N1.5) and CPU time of less than O(N2) for structures containing several hundred to a few thousand atoms. The algorithm exhibits quadratic convergence near the minimum and is also very tolerant of poor initial structures. A comparison with existing optimization procedures is detailed for cyclohexane, arachidonic acid, and the small protein crambin. In particular, a structure for crambin (662 atoms) has been refined to an RMS gradient of 3.6 × 10−6 kcal/mol/A per atom on the MM2 potential energy surface. Several suggestions are made which may lead to further improvement of the new method.

$U$-Processes: Rates of Convergence

Abstract: This paper introduces a new stochastic process, a collection of $U$-statistics indexed by a family of symmetric kernels. Conditions are found for the uniform almost-sure convergence of a sequence of such processes. Rates of convergence are obtained. An application to cross-validation in density estimation is given. The proofs adapt methods from the theory of empirical processes.

The optimal convergence rate of the p-version of the finite element method

Abstract: Optimal error estimates for the p-version of the finite element method in two dimensions are proven for the case when $u \in H^k (\Omega )$ or u has singularities induced by the corners of the domain. The case of nonhomogenous essential boundary conditions is also analyzed.

Approximate Counting, Uniform Generation and Rapidly Mixing Markov Chains

Abstract: The paper studies effective approximate solutions to combinatorial counting and uniform generation problems. Using a technique based on the simulation of ergodic Markov chains, it is shown that, for self-reducible structures, almost uniform generation is possible in polynomial time provided only that randomised approximate counting to within some arbitrary polynomial factor is possible in polynomial time. It follows that, for self-reducible structures, polynomial time randomised algorithms for counting to within factors of the form (1 +n-@) are available either for all fl E R or for no fi E R. A substantial part of the paper is devoted to investigating the rate of convergence of finite ergodic Markov chains, and a simple but powerful characterisation of rapid convergence for a broad class of chains based on a structural property of the underlying graph is established. Finally, the general techniques of the paper are used to derive an almost uniform generation procedure for labelled graphs with a given degree sequence which is valid over a much wider range of degrees than previous methods: this in turn leads to randomised approximate counting algorithms for these graphs with very good

Extrapolation methods for vector sequences

Abstract: This paper derives, describes, and compares five extrapolation methods for accelerating convergence of vector sequences or transforming divergent vector sequences to convergent ones. These methods are the scalar epsilon algorithm (SEA), vector epsilon algorithm (VEA), topological epsilon algorithm (TEA), minimal polynomial extrapolation (MPE), and reduced rank extrapolation (RRE). MPE and RRE are first derived and proven to give the exact solution for the right 'essential degree' k. Then, Brezinski's (1975) generalization of the Shanks-Schmidt transform is presented; the generalized form leads from systems of equations to TEA. The necessary connections are then made with SEA and VEA. The algorithms are extended to the nonlinear case by cycling, the error analysis for MPE and VEA is sketched, and the theoretical support for quadratic convergence is discussed. Strategies for practical implementation of the methods are considered.

A numerical approach to the infinite horizon problem of deterministic control theory

Abstract: We are concerned with the Hamilton-Jacobi equation related to the infinite horizon problem of deterministic control theory. Approximate solutions are constructed by means of a discretization in time as well as in the state variable and we prove that their rate of convergence to the viscosity solution is of order 1, provided a semiconcavity assumption is satisfied. A computational algorithm, originally due to R. Gonzales and E. Rofman, is adapted and reformulated for the problem at hand in order to obtain an error estimate for the numerical approximate solutions.

Artificial dissipation and central difference schemes for the Euler and Navier-Stokes equations

Abstract: An artificial dissipation model, including boundary treatment, that is employed in many central difference schemes for solving the Euler and Navier-Stokes equations is discussed. Modifications of this model such as the eigenvalue scaling suggested by upwind differencing are examined. Multistage time stepping schemes with and without a multigrid method are used to investigate the effects of changes in the dissipation model on accuracy and convergence. Improved accuracy for inviscid and viscous airfoil flow is obtained with the modified eigenvalue scaling. Slower convergence rates are experienced with the multigrid method using such scaling. The rate of convergence is improved by applying a dissipation scaling function that depends on mesh cell aspect ratio.

Local convergence analysis of a grouped variable version of coordinate descent

Abstract: LetF(x,y) be a function of the vector variablesx∈R n andy∈R m . One possible scheme for minimizingF(x,y) is to successively alternate minimizations in one vector variable while holding the other fixed. Local convergence analysis is done for this vector (grouped variable) version of coordinate descent, and assuming certain regularity conditions, it is shown that such an approach is locally convergent to a minimizer and that the rate of convergence in each vector variable is linear. Examples where the algorithm is useful in clustering and mixture density decomposition are given, and global convergence properties are briefly discussed.

The regularizing properties of the adjoint gradient method in ill-posed problems

Abstract: It is shown that the adjoint gradient method with a stop using the discrepancy principle is optimal with respect to the order of error and the rate of convergence in classes of linear ill-posed problems with source-like represented solutions in a Hubert space.

On the Numerical Approximation of Phase Portraits Near Stationary Points

Abstract: We show that the phase portrait of a dynamical system near a stationary hyperbolic point is reproduced correctly by numerical methods such as one-step methods or multi-step methods satisfying a strong root condition. This means that any continuous trajectory can be approximated by an appropriate discrete trajectory, and vice versa, to the correct order of convergence and uniformly on arbitrarily large time intervals. In particular, the stable and unstable manifolds of the discretization converge to their continuous counterparts.

On ill-posed problems and the method of conjugate gradients

Abstract: This chapter discusses the convergence rate of iterative methods and the convergence rate of the method of conjugate gradients (cg-method) when applied to ill-posed problems of the kind K x = g , where K: H → H 1 is a linear bounded operator between Hilbert spaces and g ∈ H 1 . For the methods of steepest descent and of conjugate gradients the first convergence proofs for ill-posed problems were given by Kammerer–Nashed in 1971 and 1972. For the cases of the Landweber–Fridman method and the method of steepest descent, the exponent v is best-possible for the class P [ v ]. Iterative methods exist for P [ v ] with convergence rates ∥ e n ∥ = 0((1/ n ) 2 v ), and the cg-method has this property for all v > 0. The chapter presents a family of iterative methods depending on a parameter v > 0, where the sequence S n ( v ) = 1 - λ R n ( v ) of associated polynomials has the property that ω( v, S n ( v ) ) = 0((1/ n ) 2 v ).

On finding 'exciting' trajectories for identification experiments involving systems with non-linear dynamics

Abstract: When designing an identification experiment for a system described by non-linear functions, such as those of manipulator dynamics, it is necessary to consider the sufficiency of excitation. It is shown that the convergence rate and noise immunity of a parameter identification experiment depend directly upon the condition number of the persistent excitation matrix. A method is presented to optimize this condition number using the calculus of variations. Analysis of condition numbers of several trajectories has shown that intuitively selected trajectories can be very poorly conditioned. The optimizer applied to the best trajectory in one experiment reported in the literature has reduced the convergence time from 1 hour and 25 minutes to 4 minutes.

Convergence rates in the strong law for bounded mixing sequences

Abstract: Speed of convergence is studied for a Marcinkiewicz-Zygmund strong law for partial sums of bounded dependent random variables under conditions on their mixing rate. Though α-mixing is also considered, the most interesting result concerns absolutely regular sequences. The results are applied to renewal theory to show that some of the estimates obtained by other authors on coupling are best possible. Another application sharpens a result for averaging a function along a random walk.

Strong convergence and convergence rates of approximating solutions for algebraic riccati equations in Hilbert spaces

Abstract: The linear quadratic optimal control problem on infinite time interval for linear time-invariant systems defined on Hilbert spaces is considered. The optimal control is given by a feedback form in terms of solution pi to the associated algebraic Riccati equation (ARE). A Ritz type approximation is used to obtain a sequence pi sup N of finite dimensional approximations of the solution to ARE. A sufficient condition that shows pi sup N converges strongly to pi is obtained. Under this condition, a formula is derived which can be used to obtain a rate of convergence of pi sup N to pi. The results of the Galerkin approximation is demonstrated and applied for parabolic systems and the averaging approximation for hereditary differential systems.

Convergence rate estimates for iterative methods for a mesh symmetrie eigenvalue problem

Abstract: In this paper we consider a symmetric partial algebraic eigenvalue problem. In Section 1 we present several estimates for the rates of convergence of some classical algorithms of vector iterations. Estimates of the accuracy of the Rayleigh-Ritz method and of the subspace iterations are considered in Section 2. The convergence rates of several methods of solving generalized eigenvalue problems by means of a preconditioner are analysed in Section 3. Finally, Section 4 deals with the Temple-Lehmann two-sided estimates for eigenvalues. The paper constitutes a systematic review of recent results mainly due to the author. Consider in a Euclidean space H the problem of computing p maximal eigenvalues A! > · · · > λρ and the corresponding eigenvectors for a generalized eigenvalue problem Mu = XLu, M = M*, L = L*>0 (0.1) (to simplify the notation, all the eigenvalues λί > ··· > λρ are taken to be simple). To calculate the minimal eigenvalues of (0.1), we need only to replace Μ by — Μ throughout. In computational practice the problem is traditionally tackled by implicit reduction of the generalized eigenproblem (0.1) to the ordinary one, i.e. ΑΗ = λΐΛ, A = A* (0.2) where, for example, A = L\"M in the space HL equipped with the scalar product (Λ *)*. = (£·,*). In Section 1 we present several estimates for the convergence rates of some classical methods of vector iterations for problem (0.2) with ρ = 1. In Section 2 we consider some estimates for the accuracy of the Rayleigh-Ritz method and of the subspace iterations for problem (0.2) with ρ > 1. In Section 3 we investigate the convergence rates of several methods of solving eigenvalue problem (0.1) using a preconditioner Β = Β* > 0 such that the system Bu = f can be efficiently solved, and the ratio δ, δ = δ0/δΐ9 0<δ0Β*ζΙιζδ1Β (0.3) is as close to 1 as possible. Finally, Section 4 deals with the Temple-Lehmann approach to obtaining two-sided estimates for the eigenvalues of (0.1). The results of this paper can be useful in evaluating the efficiency of various iterative techniques for solving problems of type (0.1), which can occur from the finite difference or finite element discretization of differential eigenvalue problems. Our choice of Originally published in Russian in Numerical Methods and Mathematical Modelling, Transactions of the Department of Numerical Mathematics of the USSR Academy of Sciences, Moscow, 1986.

Hammerstein system identification by non-parametric regression estimation

Abstract: A discrete-time, multiple-input non-linear Hammerstein system is identified. The dynamical subsystem is recovered using the standard correlation method. The main results concern estimation of the non-linear memoryless subsystem. No conditions concerning the functional form of the transform characteristic of the subsystem are made and an algorithm for estimation of the characteristic is given. The algorithm is simply a non-parametric kernel estimate of the regression function calculated from the dependent data. It is shown that the algorithm converges to the characteristic of the subsystem in the pointwise as well as the global sense. For sufficiently smooth characteristics, the rate of convergence is o(n-1/(2+d in probability, where d is the dimension of the input variable.

Parallel Superconvergent Multigrid

Abstract: We describe a class of multiscale algorithms for the solution of large sparse linear systems that are particularly well adapted to massively parallel supercomputers. While standard multigrid algorithms are unable to effectively use all processors when computing on coarse grids, the new algorithms utilize the same number of processors at all times. The basic idea is to solve many coarse scale problems simultaneously, combining the results in an optimal way to provide an improved fine scale solution. As a result, convergence rates are much faster than for standard multigrid methods we have obtained V-cycle convergence rates as good as .0046 with one smoothing application per cycle, and .0013 with two smoothings. On massively parallel machines, the improved convergence rate is attained at no extra computational cost since processors that would otherwise be sitting idle are utilized to provide the better convergence. On serial machines, the algorithm is slower because of the extra time spent on multiple coarse scales, though in certain cases the improved convergence rate may justify this particularly in cases where other methods do not converge. In constant coefficient situations the algorithm is easily analyzed theoretically using Fourier methods on a single grid. The fact that only one grid is involved substantially simplifies convergence proofs. A feature of the algorithms is the use of a matched pair of operators: an approximate inverse for smoothing and a super-interpolation operator to move the correction from coarse to fine scales, chosen to optimize the rate of convergence.

On the efficiency of Newton's method in approximating all zeros of a system of complex polynomials

Abstract: This paper studies the efficiency of an algorithm based on Newton's method is approximating all zeros of a system of polynomials f = (f1, f2, …, fn): ℂn → ℂn. The criteria for a successful approximation y of a zero w of f include the following: given ϵ > 0, y is within distance ϵ of w; Newton's method applied to f and initiated at y results in quadratic convergence to w; given ϵ > 0, |fi(y)| < ϵ for all i = 1, 2, …, n, where | | is the Euclidean norm on ℂ. It is shown that, probabilistically, each zero of f is successfully approximated within a determined number of steps.

A local relaxation method for solving PDEs on mesh-connected arrays

Abstract: A local relaxation method for solving linear elliptic PDEs with $O(N)$ processors and $O(\sqrt N )$ computation time is proposed. We first examine the implementation of traditional relaxation algorithms for solving elliptic PDEs on mesh-connected processor arrays, which require $O(N)$ processors and $O(N)$ computation time. The disadvantage of these implementations is that the determination of the acceleration factors requires some global communication at each iteration. The high communication cost increases the computation time per iteration significantly. Therefore, a local relaxation scheme is proposed to achieve the acceleration effect with very little global communication in the loading stage. We use a Fourier analysis approach to analyze the local relaxation method and also show its convergence. The convergence rate of the local relaxation method is studied by computer simulation.

On Smoothing Sparse Multinomial Data

Abstract: Summary Asymptotic theory is developed for the problem of smoothing sparse multinomial data, with emphasis on the criterion of mean summed square error of estimators of the probability mass function. If the data are not too sparse, in a well-defined sense, then the optimal rate of convergence is that achieved by the unsmoothed cell proportions. Otherwise, this rate can be improved upon by smoothing. Explicit results, including formulae for the optimal smoothing parameter, are presented for a kernel-type estimator. Also for this case, a cross-validatory choice procedure is shown to be asymptotically optimal.

A projected Newton method for l p norm location problems

Abstract: This paper is concerned with the numerical solution of continuous minisum multifacility location problems involving thel p norm, where 1

Hierarchical Motion Detection

Abstract: This thesis presents algorithms for the efficient computation of image motions using hierarchical multiresolution methods operating on image data pyramids in the processing cone architecture Three topics are addressed: (1) fast construction of image pyramids; (2) hierarchical motion detection algorithms: correlation-based and gradient-based; and (3) multilevel relaxation algorithms Pyramid building is the first step in hierarchical motion detection A family of discrete Gaussian low pass filters for building low pass pyramids is presented that provide good anti-aliasing characteristics, efficient computation, and a good hierarchical Gaussian approximation Frequency space analysis, using Fourier Transform theory, is used to compare alternative filters Hierarchical correlation overcomes two disadvantages of correlation matching: large search areas which require expensive searches, and repeating features which cause incorrect matches Coarse-to-fine control provides speed and efficiency when search spaces are large, and accuracy when repeating details can be confused Hierarchical gradient-based algorithms extend single level gradient-based algorithms to the computation of large disparities They use a coarse-to-fine method in which approximate disparities are refined at each level by computing relatively small updates This allows the gradient-based method's assumption of local linearity to apply in spite of the large total disparities Experiments show the failure of single level methods (for large disparities) and the success of hierarchical methods The two hierarchical algorithms are shown to have comparable accuracy Comparison of the computational costs, both arithmetic and data transfer, show that the gradient-based algorithm are, in general, less costly Multilevel relaxation algorithms for the computation of optic flow are developed and experiments show the expected increased convergence rate over single level relaxation, although some experiments present a problem of divergence at coarse levels A local mode analysis of the relaxation equations shows that convergence is at least as fast as simple smoothing, and that, with strong gradients, convergence is accelerated towards the constraint line The local mode analysis does not account for coarse level divergence Divergence is then shown to be due to spatial variation in the image data Fixed up/down cycling schemes are used to overcome the divergence problem

Convergence of a Dinkelbach-type algorithm in generalized fractional programming

Abstract: The convergence of a Dinkelbach-type algorithm in generalized fractional programming is obtained by considering the sensitivity of a parametrized problem. We show that the rate of convergence is at least equal to (1+√5)/2 when regularity conditions hold in a neighbourhood of the optimal solution. We give also a necessary and sufficient condition for the convergence to be quadratic (which will be verified in particular in the linear case) and an idea of its implementation in the convex case.

A hybrid BEM model

Abstract: The hybrid stress method is very successful for stress concentration problems.1–7 Especially for problems of fracture mechanics, procedures can be found that work efficiently for two- and three-dimensional problems. The rate of convergence with this method, evidently, is higher than that with conventional FE models. The BEM procedure, too, works more efficiently, but shows some essential disadvantages against the FEM, such as that for the direct method no symmetric positive definite matrix can be found and that there occur numerical problems at corners.8,9 This happens also when BEM and FEM are even coupled commonly.10–12. In the following, a hybrid BEM model will be described which combines the advantages of both the FEM and the BEM. It will be shown in this paper that BEM is very successful in formulating finite element functions for the hybrid assumed stress method.