Topic
Rate of convergence
About: Rate of convergence is a research topic. Over the lifetime, 31257 publications have been published within this topic receiving 795334 citations. The topic is also known as: convergence rate.
Papers published on a yearly basis
Papers
More filters
••
TL;DR: In the practically important case of logically cubic meshes with randomly perturbed nodes, the mixed finite element with the lowest order Raviart–Thomas elements does not converge while the proposed mimetic method has the optimal convergence rate.
147 citations
••
TL;DR: In this paper, the authors derived necessary and sufficient conditions on the range of one such parameter to guarantee stability of the method, and showed that the parameter effects only the length, not the direction, of the search vector at each step, and used this result to derive several computational algorithms.
Abstract: Quasi-Newton methods accelerate gradient methods for minimizing a function by approximating the inverse Hessian matrix of the function. Several papers in recent literature have dealt with the generation of classes of approximating matrices as a function of a scalar parameter. This paper derives necessary and sufficient conditions on the range of one such parameter to guarantee stability of the method. It further shows that the parameter effects only the length, not the direction, of the search vector at each step, and uses this result to derive several computational algorithms. The algorithms are evaluated on a series of test problems.
147 citations
••
TL;DR: In this paper, a simultaneous multifrequency inversion approach for seismic data interpretation is presented, where a data weighting scheme balances the contributions from different frequency data components so the inversion process does not become dominated by highfrequency data components, which produces a velocity image with many artifacts.
Abstract: We present a simultaneous multifrequency inversion approach for seismic data interpretation. This algorithm inverts all frequency data components simultaneously. A data-weighting scheme balances the contributions from different frequency data components so the inversion process does not become dominated by high-frequency data components, which produce a velocity image with many artifacts. A Gauss-Newton minimization approach achieves a high convergence rate and an accurate reconstructed velocity image. By introducing a modified adjoint formulation, we can calculate the Jacobian matrix efficiently, allowing the material properties in the perfectly matched layers (PMLs) to be updated automatically during the inversion process. This feature ensures the correct behavior of the inversion and implies that the algorithm is appropriate for realistic applications where a priori information of the background medium is unavailable. Two different regularization schemes, an L2 -norm and a weighted L2 -norm function, a...
147 citations
••
TL;DR: This paper is essentially expository but it presents however new results concerning some problems of rate of convergence in Finance theory.
Abstract: We present several results and methods concerning the convergence of numerical schemes for problems arising in Finance theory. This paper is essentially expository but we present however new results concerning some problems of rate of convergence.
147 citations
•
03 Dec 2018TL;DR: This paper studies the convergence rate of distributed SGD for non-convex optimization with two communication reducing strategies: sparse parameter averaging and gradient quantization and proposes a strategy called periodic quantized averaging (PQASGD) that further reduces the communication cost while preserving the O(1/√MK) convergence rate.
Abstract: The large communication overhead has imposed a bottleneck on the performance of distributed Stochastic Gradient Descent (SGD) for training deep neural networks. Previous works have demonstrated the potential of using gradient sparsification and quantization to reduce the communication cost. However, there is still a lack of understanding about how sparse and quantized communication affects the convergence rate of the training algorithm. In this paper, we study the convergence rate of distributed SGD for non-convex optimization with two communication reducing strategies: sparse parameter averaging and gradient quantization. We show that O(1/√MK) convergence rate can be achieved if the sparsification and quantization hyperparameters are configured properly. We also propose a strategy called periodic quantized averaging (PQASGD) that further reduces the communication cost while preserving the O(1/√MK) convergence rate. Our evaluation validates our theoretical results and shows that our PQASGD can converge as fast as full-communication SGD with only 3% - 5% communication data size.
147 citations