Showing papers on "Rate of convergence published in 1996"

Linear least-squares algorithms for temporal difference learning

Abstract: We introduce two new temporal diffence (TD) algorithms based on the theory of linear least-squares function approximation. We define an algorithm we call Least-Squares TD (LS TD) for which we prove probability-one convergence when it is used with a function approximator linear in the adjustable parameters. We then define a recursive version of this algorithm, Recursive Least-Squares TD (RLS TD). Although these new TD algorithms require more computation per time-step than do Sutton's TD(λ) algorithms, they are more efficient in a statistical sense because they extract more information from training experiences. We describe a simulation experiment showing the substantial improvement in learning rate achieved by RLS TD in an example Markov prediction problem. To quantify this improvement, we introduce theTD error variance of a Markov chain, ωTD, and experimentally conclude that the convergence rate of a TD algorithm depends linearly on ωTD. In addition to converging more rapidly, LS TD and RLS TD do not have control parameters, such as a learning rate parameter, thus eliminating the possibility of achieving poor performance by an unlucky choice of parameters.

Rates of convergence of the Hastings and Metropolis algorithms

Abstract: We apply recent results in Markov chain theory to Hastings and Metropolis algorithms with either independent or symmetric candidate distributions, and provide necessary and sufficient conditions for the algorithms to converge at a geometric rate to a prescribed distribution $\pi$. In the independence case (in $\mathbb{R}^k$) these indicate that geometric convergence essentially occurs if and only if the candidate density is bounded below by a multiple of $\pi$; in the symmetric case (in $\mathbb{R}$ only) we show geometric convergence essentially occurs if and only if $\pi$ has geometric tails. We also evaluate recently developed computable bounds on the rates of convergence in this context: examples show that these theoretical bounds can be inherently extremely conservative, although when the chain is stochastically monotone the bounds may well be effective.

A class of smoothing functions for nonlinear and mixed complementarity problems

Abstract: We propose a class of parametric smooth functions that approximate the fundamental plus function, (x)+=max{0, x}, by twice integrating a probability density function. This leads to classes of smooth parametric nonlinear equation approximations of nonlinear and mixed complementarity problems (NCPs and MCPs). For any solvable NCP or MCP, existence of an arbitrarily accurate solution to the smooth nonlinear equations as well as the NCP or MCP, is established for sufficiently large value of a smoothing parameter α. Newton-based algorithms are proposed for the smooth problem. For strongly monotone NCPs, global convergence and local quadratic convergence are established. For solvable monotone NCPs, each accumulation point of the proposed algorithms solves the smooth problem. Exact solutions of our smooth nonlinear equation for various values of the parameter α, generate an interior path, which is different from the central path for interior point method. Computational results for 52 test problems compare favorably with these for another Newton-based method. The smooth technique is capable of solving efficiently the test problems solved by Dirkse and Ferris [6], Harker and Xiao [11] and Pang & Gabriel [28].

Convergence to the Law of One Price Without Trade Barriers or Currency Fluctuations

Abstract: Using a panel of 51 prices from 48 cities in the United States we provide an upper bound estimate of the rate of convergence to Purchasing Power Parity. We find convergence rates substantially higher than typically found in cross-country data. We investigate some potentially serious biases induced by i.i.d. measurement errors in the data, and find our estimates to be robust to these potential biases. We also present evidence that convergence occurs faster for larger price differences. Finally, we find that rates of convergence are slower for cities farther apart. However, our estimates suggest that distance alone can only account for a small portion of the much slower convergence rates across national borders.

Convergence to the Law of One Price Without Trade Barriers or Currency Fluctuations

Abstract: Using a panel of 51 prices from 48 cities in the United States, we provide an upper bound estimate of the rate of convergence to purchasing power parity. We find convergence rates substantially higher than typically found in cross-country data. We investigate some potentially serious biases induced by i.i.d. measurement errors in the data, and find our estimates to be robust to these potential biases. We also present evidence that convergence occurs faster for larger price differences. Finally, we find that rates of convergence are slower for cities farther apart. However, our estimates suggest that distance alone can only account for a small portion of the much slower convergence rates across national borders.

Iterative learning control for discrete-time systems with exponential rate of convergence

Abstract: An algorithm for iterative learning control is proposed based on an optimisation principle used by other authors to derive gradient-type algorithms. The new algorithm is a descent algorithm and has potential benefits which include realisation in terms of Riccati feedback and feedforward components. This realisation also has the advantage of implicitly ensuring automatic step-size selection and hence guaranteeing convergence without the need for empirical choice of parameters. The algorithm achieves a geometric rate of convergence for invertible plants. One important feature of the proposed algorithm is the dependence of the speed of convergence on weight parameters appearing in the norms of the signals chosen for the optimisation problem.

The law of the euler scheme for stochastic differential equations: i. convergence rate of the distribution function

Abstract: We study the approximation problem ofE f(X T ) byE f(X ), where (X t ) is the solution of a stochastic differential equation, (X ) is defined by the Euler discretization scheme with stepT/n, andf is a given function. For smoothf's, Talay and Tubaro have shown that the errorE f(X T ) −f(X ) can be expanded in powers of 1/n, which permits to construct Romberg extrapolation precedures to accelerate the convergence rate. Here, we prove that the expansion exists also whenf is only supposed measurable and bounded, under an additional nondegeneracy condition of Hormander type for the infinitesimal generator of (X t ): to obtain this result, we use the stochastic variations calculus. In the second part of this work, we will consider the density of the law ofX and compare it to the density of the law ofX T .

Genetic algorithms, selection schemes, and the varying effects of noise

Abstract: This paper analyzes the effect of noise on different selection mechanisms for genetic algorithms (GAs). Models for several selection schemes are developed that successfully predict the convergence characteristics of GAs within noisy environments. The selection schemes modeled in this paper include proportionate selection, tournament selection, (μ, λ) selection, and linear ranking selection. An allele-wise model for convergence in the presence of noise is developed for the OneMax domain, and then extended to more complex domains where the building blocks are uniformly scaled. These models are shown to accurately predict the convergence rate of GAs for a wide range of noise levels.

A semismooth equation approach to the solution of nonlinear complementarity problems

Abstract: In this paper we present a new algorithm for the solution of nonlinear complementarity problems. The algorithm is based on a semismooth equation reformulation of the complementarity problem. We exploit the recent extension of Newton's method to semismooth systems of equations and the fact that the natural merit function associated to the equation reformulation is continuously differentiable to develop an algorithm whose global and quadratic convergence properties can be established under very mild assumptions. Other interesting features of the new algorithm are an extreme simplicity along with a low computational burden per iteration. We include numerical tests which show the viability of the approach.

Asymptotic Analysis of

Abstract: This paper presents an analysis of stochastic gradient-based adaptive algorithms with general cost functions. The analysis holds under mild assumptions on the inputs and the cost function. The method of analysis is based on an asymptotic analysis of fixed stepsize adaptive algorithms and gives almost sure results regarding the behavior of the parameter estimates, whereas previous stochastic analyses typically consider mean and mean square behavior. The parameter estimates are shown to enter a small neighborhood about the optimum value and remain there for a finite length of time. Furthermore, almost sure exponential bounds are given for the rate of convergence of the parameter estimates. The asymptotic distribution of the parameter estimates is shown to be Gaussian with mean equal to the optimum value and covariance matrix that depends on the input statistics. Specific adaptive algorithms that fall under the framework of this paper are signed error least mean squre (LMS), dual sign LMS, quantized state LMS, least mean fourth, dead zone algorithms, momentum algorithms, and leaky LMS.

Modified Projection-Type Methods for Monotone Variational Inequalities

Abstract: We propose new methods for solving the variational inequality problem where the underlying function $F$ is monotone. These methods may be viewed as projection-type methods in which the projection direction is modified by a strongly monotone mapping of the form $I - \alpha F$ or, if $F$ is affine with underlying matrix $M$, of the form $I+ \alpha M^T$, with $\alpha \in (0,\infty)$. We show that these methods are globally convergent, and if in addition a certain error bound based on the natural residual holds locally, the convergence is linear. Computational experience with the new methods is also reported.

Dextrous hand grasping force optimization

Abstract: A key goal in dextrous robotic hand grasping is to balance external forces and at the same time achieve grasp stability and minimum grasping energy by choosing an appropriate set of internal grasping forces. Since it appears that there is no direct algebraic optimization approach, a recursive optimization, which is adaptive for application in a dynamic environment, is required. One key observation in this paper is that friction force limit constraints and force balancing constraints are equivalent to the positive definiteness of a certain matrix subject to linear constraints. Based on this observation, we formulate the task of grasping force optimization as an optimization problem on the smooth manifold of linearly constrained positive definite matrices for which there are known globally exponentially convergent solutions via gradient flows. There are a number of versions depending on the Riemannian metric chosen, each with its advantages, Schemes involving second derivative information for quadratic convergence are also studied. Several forms of constrained gradient flows are developed for point contact and soft-finger contact friction models. The physical meaning of the cost index used for the gradient flows is discussed in the context of grasping force optimization. A discretized version for real-time applicability is presented. Numerical examples demonstrate the simplicity, the good numerical properties, and optimality of the approach.

Fast nonsymmetric iterations and preconditioning for Navier-Stokes equations

Abstract: Discretization and linearization of the steady–state Navier-Stokes equations gives rise to a nonsymmetric indefinite linear system of equations. In this paper, we introduce preconditioning techniques for such systems with the property that the eigenvalues of the preconditioned matrices are bounded independently of the mesh size used in the discretization. We confirm and supplement these analytic results with a series of numerical experiments indicating that Krylov subspace iterative methods for nonsymmetric systems display rates of convergence that are independent of the mesh parameter. In addition, we show that preconditioning costs can be kept small by using iterative methods for some intermediate steps performed by the preconditioner.

Jacobi-davidson type methods for generalized eigenproblems and polynomial eigenproblems ∗

Abstract: In this paper we will show how the Jacobi-Davidson iterative method can be used to solve generalized eigenproblems. Similar ideas as for the standard eigenproblem are used, but the projections, that are required to reduce the given problem to a small manageable size, need more attention. We show that by proper choices for the projection operators quadratic convergence can be achieved. The advantage of our approach is that none of the involved operators needs to be inverted. It turns out that similar projections can be used for the iterative approximation of selected eigenvalues and eigenvectors of polynomial eigenvalue equations. This approach has already been used with great success for the solution of quadratic eigenproblems associated with acoustic problems.

Conservative modeling of 3-D electromagnetic fields, Part II: Biconjugate gradient solution and an accelerator

Abstract: The preceding paper derives a staggered-grid, finite-difference approximation applicable to electromagnetic induction in the Earth. The staggered-grid, finite-difference approximation results in a linear system of equations Ax = b, where A is symmetric but not Hermitian. This is solved using the biconjugate gradient method, preconditioned with a modified, partial Cholesky decomposition of A. This method takes advantage of the sparsity of A, and converges much more quickly than methods used previously to solve the 3-D induction problem. When simulating a conductivity model at a number of frequencies, the rate of convergence slows as frequency approaches 0. The convergence rate at low frequencies can be improved by an order of magnitude, by alternating the incomplete Cholesky preconditioned biconjugate gradient method with a procedure designed to make the approximate solutions conserve current.

A rational Lanczos algorithm for model reduction

Abstract: This paper presents a model reduction method for large-scale linear systems that is based on a Lanczos-type approach. A variant of the nonsymmetric Lanczos method, rational Lanczos, is shown to yield a rational interpolant (multi-point Padi approximant) for the large-scale system. An exact expression for the error in the interpolant is derived. Examples are utilized to demonstrate that the rational Lanczos method provides opportunities for significant improvements in the rate of convergence over single-point Lanczos approaches.

Numerical methods for forward-backward stochastic differential equations

Abstract: In this paper we study numerical methods to approximate the adapted solutions to a class of forward-backward stochastic differential equations (FBSDE's). The almost sure uniform convergence as well as the weak convergence of the scheme are proved, and the rate of convergence is proved to be as good as the approximation for the corresponding forward SDE. The idea of the approximation is based on the four step scheme for solving such an FBSDE, developed by Ma, Protter and Yong. For the PDE part, the combined characteristics and finite difference method is used, while for the forward SDE part, we use the first order Euler scheme.

Data-parallel lower-upper relaxation method for the navier-stokes equations

Abstract: The lower-upper symmetric Gauss-Seidel method is modified for the simulation of viscous flows on massively parallel computers. The resulting diagonal data-parallel lower-upper relaxation (DP-LUR) method is shown to have good convergence properties on many problems. However, the convergence rate decreases on the high cell aspect ratio grids required to simulate high Reynolds number flows. Therefore, the diagonal approximation is relaxed, and a full matrix version of the DP-LUR method is derived. The full matrix method retains the data-parallel properties of the original and reduces the sensitivity of the convergence rate to the aspect ratio of the computational grid. Both methods are implemented on the Thinking Machines CM-5, and a large fraction of the peak theoretical performance of the machine is obtained. The low memory use and high parallel efficiency of the methods make them attractive for large-scale simulation of viscous flows.

A note on the use of Laplace's approximation for nonlinear mixed-effects models

Abstract: The asymptotic properties of estimates obtained using Laplace's approximation for nonlinear mixed-effects models are investigated. Unlike the restricted maximum likelihood approach, e.g. Wolfinger (1993), here the Laplace approximation is applied only to the random effects of the integrated likelihood. This results in approximate maximum likelihood estimation. The resulting estimates are shown to be consistent with the rate of convergence depending on both the number of individuals and the number of observations per individual. Conditions under which the leading term Laplace approximation should be avoided are discussed.

The p and hp versions of the finite element method for problems with boundary layers

Abstract: We study the uniform approximation of boundary layer functions exp(-x/d) for x ∈ (0,1), d ∈ (0,1], by the p and hp versions of the finite element method. For the p version (with fixed mesh), we prove super-exponential convergence in the range p + 1/2 > e/(2d). We also establish, for this version, an overall convergence rate of O(p -1 √ln p) in the energy norm error which is uniform in d, and show that this rate is sharp (up to the √ln p term) when robust estimates uniform in d ∈ (0,1] are considered. For the p version with variable mesh (i.e., the hp version), we show that exponential convergence, uniform in d ∈ (0,1], is achieved by taking the first element at the boundary layer to be of size O(pd). Numerical experiments for a model elliptic singular perturbation problem show good agreement with our convergence estimates, even when few degrees of freedom are used and when d is as small as, e.g., 10 -8 . They also illustrate the superiority of the hp approach over other methods, including a low-order h version with optimal exponential mesh refinement. The estimates established in this paper are also applicable in the context of corresponding spectral element methods.

Convergence properties of ordinal comparison in the simulation of discrete event dynamic systems

Abstract: Recent research has demonstrated that ordinal comparison has fast convergence despite the possible presence of large estimation noise in the design of discrete event dynamic systems. In this paper, we address the fundamental problem of characterizing the convergence of ordinal comparison. To achieve this goal, an indicator process is formulated and its properties are examined. For several performance measures frequently used in simulation, the rate of convergence for the indicator process is proven to be exponential for regenerative simulations. Therefore, the fast convergence of ordinal comparison is supported and explained in a rigorous framework. Many performance measures of averaging type have asymptotic normal distributions. The results of this paper show that ordinal comparison converges monotonically in the case of averaging normal random variables. Such monotonicity is useful in simulation planning.

Binomial models for option valuation - examining and improving convergence

Abstract: Binomial models, which describe the asset price dynamics of the continuous-time model in the limit, serve for approximate valuation of options, especially where formulas cannot be derived analytically due to properties of the considered option type. To evaluate results, one inevitably must understand the convergence properties. In the literature we find various contributions proving convergence of option prices. We examine convergence behaviour and convergence speed. Unfortunately, even in the case of European call options, distorted results occur when calculating prices along the iteration of tree refinements. These convergence patterns are examined and order of convergence one is proven for the Cox-Ross-Rubinstein model as well as for two alternative tree parameter selections from the literature. Furthermore, we define new binomial models, where the calculated option prices converge smoothly to the Black-Scholes solution, and we achieve order of convergence two with much smaller initial error. Notably, ...

Time-varying sliding modes for second-order systems

Abstract: Three types of optimal, continuously time-varying sliding mode for robust control of second-order uncertain dynamic systems subject to input constraint are presented. Two of the modes incorporate straight sliding lines, and the third uses the so-called terminal slider, that is a curve that guarantees system error convergence to zero in finite time. At first, all three lines adapt themselves to the initial conditions of the system, and afterwards they move in such a way that, for each of them, the integral of the absolute value of the systems error is minimised over the whole period of the control action. By this means, insensitivity of the system to external disturbances and parameter uncertainties is guaranteed from the very beginning of the proposed control action, and the system error convergence rate can be increased. Performance of the three control algorithms is compared, and the Lyapunov theory is used to prove the existence of a sliding mode on the lines.

Computing the Extremal Positive Definite Solutions of a Matrix Equation

Abstract: An efficient and numerically stable implementation of a known algorithm is suggested for finding the extremal positive definite solutions of the matrix equation $X+A^*X^{-1}A=I$, if such solutions exist. The convergence rate is analyzed. A new algorithm that avoids matrix inversion is presented. Numerical examples are given to illustrate the effectiveness of the algorithms.

Trust-region interior-point algorithms for a class of nonlinear programming problems

Abstract: This thesis introduces and analyzes a family of trust-region interior-point (TRIP) reduced sequential quadratic programming (SQP) algorithms for the solution of minimization problems with nonlinear equality constraints and simple bounds on some of the variables. These nonlinear programming problems appear in applications in control, design, parameter identification, and inversion. In particular they often arise in the discretization of optimal control problems. The TRIP reduced SQP algorithms treat states and controls as independent variables. They are designed to take advantage of the structure of the problem. In particular they do not rely on matrix factorizations of the linearized constraints, but use solutions of the linearized state and adjoint equations. These algorithms result from a successful combination of a reduced SQP algorithm, a trust-region globalization, and a primal-dual affine scaling interior-point method. The TRIP reduced SQP algorithms have very strong theoretical properties. It is shown in this thesis that they converge globally to points satisfying first and second order necessary optimality conditions, and in a neighborhood of a local minimizer the rate of convergence is quadratic. Our algorithms and convergence results reduce to those of Coleman and Li for box-constrained optimization. An inexact analysis is presented to provide a practical way of controlling residuals of linear systems and directional derivatives. Complementing this theory, numerical experiments for two nonlinear optimal control problems are included showing the robustness and effectiveness of these algorithms. Another topic of this dissertation is a specialized analysis of these algorithms for equality-constrained optimization problems. The important feature of the way this family of algorithms specializes for these problems is that they do not require the computation of normal components for the step and an orthogonal basis for the null space of the Jacobian of the equality constraints. An extension of More and Sorensen's result for unconstrained optimization is presented, showing global convergence for these algorithms to a point satisfying the second-order necessary optimality conditions.

Second-order regular variation and rates of convergence in extreme-value theory

Abstract: Rates of convergence of the distribution of the extreme order statistic to its limit distribution are given in the uniform metric and the total variation metric. A second-order regular variation condition is imposed by supposing a von Mises type condition which allows a unified treatment. Rates are constructed from the parameters of the second-order regular variation condition. Some connections with Poisson processes are discussed.

Convergence Rate for the Approximation of the Limit Law of Weakly Interacting Particles 2: Application to the Burgers Equation

Abstract: In this paper, we construct a stochastic particles method for the Burgers equation with a monotonic initial condition; we prove that the convergence rate is $\displaystyleO\left(\frac1\sqrtN +\sqrt\D\right)$ for the $L^1(I\!\!R \times \Omega)$-norm of the error. To obtain that result, we link the PDE and the algorithm to a system of weakly interacting stochastic particles; the difficulty of the analysis comes from the discontinuity of the interaction kernel, equal to the Heaviside function. In~\citebossy_talay-93, we show how the algorithm and the result extend to the case of non monotonic initial conditions for the Burgers equation; we also treat the case of nonlinear PDE's related to particles systems with Lipschitz interaction kernels. Our next objective is to adapt our methodology to the (more difficult) case of the 2-D inviscid Navier-Stokes equation.

An iterative solution method for dynamic response of bridge–vehicles systems

Abstract: An iterative solution method is presented and illustrated to analyse the dynamic response of bridge–vehicle systems. The method consists in dividing the whole system into 2 subsystems at the interface of the bridge and vehicles; these 2 subsystems are solved separately; their compatibility at the interface is achieved by an iterative procedure with under-relaxation or with Aitken acceleration. The characteristics of this method are explained on a simplified system with 2 degrees of freedom (DOF). The numerical results for a simple example demonstrate the high performances of the proposed method: good convergence rate and high accuracy. Finally, the method is applied to a practical example: the linear dynamic response of the Yangtze-River Bridge at Wuhan under a moving train with 2 locomotives and 4 freight cars. The efficiency is attained because neither formation nor factorisation of the coefficient matrices for the equations of the system are needed at every time step in linear analysis. The Aitken acceleration technique is more efficient in systems with multi-degrees of freedom than the relaxation technique. The proposed method will be even more efficient in non-linear dynamic response because, in this case, the iterations are necessary whether the system is solved as a whole or not.