scispace - formally typeset
Search or ask a question
Topic

Rate of convergence

About: Rate of convergence is a research topic. Over the lifetime, 31257 publications have been published within this topic receiving 795334 citations. The topic is also known as: convergence rate.


Papers
More filters
Journal ArticleDOI
TL;DR: In the practically important case of logically cubic meshes with randomly perturbed nodes, the mixed finite element with the lowest order Raviart–Thomas elements does not converge while the proposed mimetic method has the optimal convergence rate.

147 citations

Journal ArticleDOI
TL;DR: In this paper, the authors derived necessary and sufficient conditions on the range of one such parameter to guarantee stability of the method, and showed that the parameter effects only the length, not the direction, of the search vector at each step, and used this result to derive several computational algorithms.
Abstract: Quasi-Newton methods accelerate gradient methods for minimizing a function by approximating the inverse Hessian matrix of the function. Several papers in recent literature have dealt with the generation of classes of approximating matrices as a function of a scalar parameter. This paper derives necessary and sufficient conditions on the range of one such parameter to guarantee stability of the method. It further shows that the parameter effects only the length, not the direction, of the search vector at each step, and uses this result to derive several computational algorithms. The algorithms are evaluated on a series of test problems.

147 citations

Journal ArticleDOI
TL;DR: In this paper, a simultaneous multifrequency inversion approach for seismic data interpretation is presented, where a data weighting scheme balances the contributions from different frequency data components so the inversion process does not become dominated by highfrequency data components, which produces a velocity image with many artifacts.
Abstract: We present a simultaneous multifrequency inversion approach for seismic data interpretation. This algorithm inverts all frequency data components simultaneously. A data-weighting scheme balances the contributions from different frequency data components so the inversion process does not become dominated by high-frequency data components, which produce a velocity image with many artifacts. A Gauss-Newton minimization approach achieves a high convergence rate and an accurate reconstructed velocity image. By introducing a modified adjoint formulation, we can calculate the Jacobian matrix efficiently, allowing the material properties in the perfectly matched layers (PMLs) to be updated automatically during the inversion process. This feature ensures the correct behavior of the inversion and implies that the algorithm is appropriate for realistic applications where a priori information of the background medium is unavailable. Two different regularization schemes, an L2 -norm and a weighted L2 -norm function, a...

147 citations

Journal ArticleDOI
TL;DR: This paper is essentially expository but it presents however new results concerning some problems of rate of convergence in Finance theory.
Abstract: We present several results and methods concerning the convergence of numerical schemes for problems arising in Finance theory. This paper is essentially expository but we present however new results concerning some problems of rate of convergence.

147 citations

Proceedings Article
03 Dec 2018
TL;DR: This paper studies the convergence rate of distributed SGD for non-convex optimization with two communication reducing strategies: sparse parameter averaging and gradient quantization and proposes a strategy called periodic quantized averaging (PQASGD) that further reduces the communication cost while preserving the O(1/√MK) convergence rate.
Abstract: The large communication overhead has imposed a bottleneck on the performance of distributed Stochastic Gradient Descent (SGD) for training deep neural networks. Previous works have demonstrated the potential of using gradient sparsification and quantization to reduce the communication cost. However, there is still a lack of understanding about how sparse and quantized communication affects the convergence rate of the training algorithm. In this paper, we study the convergence rate of distributed SGD for non-convex optimization with two communication reducing strategies: sparse parameter averaging and gradient quantization. We show that O(1/√MK) convergence rate can be achieved if the sparsification and quantization hyperparameters are configured properly. We also propose a strategy called periodic quantized averaging (PQASGD) that further reduces the communication cost while preserving the O(1/√MK) convergence rate. Our evaluation validates our theoretical results and shows that our PQASGD can converge as fast as full-communication SGD with only 3% - 5% communication data size.

147 citations


Network Information
Related Topics (5)
Partial differential equation
70.8K papers, 1.6M citations
89% related
Markov chain
51.9K papers, 1.3M citations
88% related
Optimization problem
96.4K papers, 2.1M citations
88% related
Differential equation
88K papers, 2M citations
88% related
Nonlinear system
208.1K papers, 4M citations
88% related
Performance
Metrics
No. of papers in the topic in previous years
YearPapers
20241
2023693
20221,530
20212,129
20202,036
20191,995