Rate of convergence

About: Rate of convergence is a research topic. Over the lifetime, 31257 publications have been published within this topic receiving 795334 citations. The topic is also known as: convergence rate.

Ensemble Nystrom Method

Abstract: A crucial technique for scaling kernel methods to very large data sets reaching or exceeding millions of instances is based on low-rank approximation of kernel matrices We introduce a new family of algorithms based on mixtures of Nystrom approximations, ensemble Nystrom algorithms, that yield more accurate low-rank approximations than the standard Nystrom method We give a detailed study of variants of these algorithms based on simple averaging, an exponential weight method, or regression-based methods We also present a theoretical analysis of these algorithms, including novel error bounds guaranteeing a better convergence rate than the standard Nystrom method Finally, we report results of extensive experiments with several data sets containing up to 1M points demonstrating the significant improvement over the standard Nystrom approximation

Linear Convergence with Condition Number Independent Access of Full Gradients

Abstract: For smooth and strongly convex optimizations, the optimal iteration complexity of the gradient-based algorithm is O(√k log 1/e), where k is the condition number. In the case that the optimization problem is ill-conditioned, we need to evaluate a large number of full gradients, which could be computationally expensive. In this paper, we propose to remove the dependence on the condition number by allowing the algorithm to access stochastic gradients of the objective function. To this end, we present a novel algorithm named Epoch Mixed Gradient Descent (EMGD) that is able to utilize two kinds of gradients. A distinctive step in EMGD is the mixed gradient descent, where we use a combination of the full and stochastic gradients to update the intermediate solution. Theoretical analysis shows that EMGD is able to find an e-optimal solution by computing O(log 1/e) full gradients and O(k2 log 1/e) stochastic gradients.

Improvement of Stability and Accuracy for Weighted Essentially Nonoscillatory Scheme

Abstract: This paper studies the weights stability and accuracy of the implicit fifth-order weighted essentially nonoscillatory finite difference scheme. It is observed that the weights of the Jiang-Shu weighted essentially nonoscillatory scheme oscillate even for smooth flows. An increased e value of 10 -2 is suggested for the weighted essentially nonoscillatory smoothness factors, which removes the weights oscillation and significantly improves the accuracy of the weights and solution convergence. With the improved e value, the weights achieve the optimum value with minimum numerical dissipation in smooth regions and maintain the sensitivity to capture nonoscillatory shock profiles for the transonic flows. The theoretical justification of this treatment is given in the paper. The wall surface boundary condition uses a half-point mesh so that the conservative differencing can be enforced. A third-order accurate finite difference scheme is given to treat wall boundary conditions. The implicit time-marching method with unfactored Gauss-Seidel line relaxation is used with the high-order schemes to achieve a high convergence rate. Several transonic cases are calculated to demonstrate the robustness, efficiency, and accuracy of the methodology.

Weak Galerkin finite element methods for the biharmonic equation on polytopal meshes

Abstract: A new weak Galerkin (WG) finite element method is introduced and analyzed in this article for the biharmonic equation in its primary form. This method is highly robust and flexible in the element construction by using discontinuous piecewise polynomials on general finite element partitions consisting of polygons or polyhedra of arbitrary shape. The resulting WG finite element formulation is symmetric, positive definite, and parameter-free. Optimal order error estimates in a discrete H2 norm is established for the corresponding WG finite element solutions. Error estimates in the usual L2 norm are also derived, yielding a suboptimal order of convergence for the lowest order element and an optimal order of convergence for all high order of elements. Numerical results are presented to confirm the theory of convergence under suitable regularity assumptions. © 2014 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq 30: 1003–1029, 2014

A simplified view of first order methods for optimization

Abstract: We discuss the foundational role of the proximal framework in the development and analysis of some iconic first order optimization algorithms, with a focus on non-Euclidean proximal distances of Bregman type, which are central to the analysis of many other fundamental first order minimization relatives. We stress simplification and unification by highlighting self-contained elementary proof-patterns to obtain convergence rate and global convergence both in the convex and the nonconvex settings, which in turn also allows to present some novel results.