A Randomized Kaczmarz Algorithm with Exponential Convergence
Thomas Strohmer,Roman Vershynin +1 more
Reads0
Chats0
TLDR
In this paper, a randomized version of the Kaczmarz method for consistent, overdetermined linear systems is introduced and it is shown that it converges with expected exponential rate.Abstract:
The Kaczmarz method for solving linear systems of equations is an iterative algorithm that has found many applications ranging from computer tomography to digital signal processing. Despite the popularity of this method, useful theoretical estimates for its rate of convergence are still scarce. We introduce a randomized version of the Kaczmarz method for consistent, overdetermined linear systems and we prove that it converges with expected exponential rate. Furthermore, this is the first solver whose rate does not depend on the number of equations in the system. The solver does not even need to know the whole system but only a small random part of it. It thus outperforms all previously known methods on general extremely overdetermined systems. Even for moderately overdetermined systems, numerical simulations as well as theoretical analysis reveal that our algorithm can converge faster than the celebrated conjugate gradient algorithm. Furthermore, our theory and numerical simulations confirm a prediction of Feichtinger et al. in the context of reconstructing bandlimited functions from nonuniform sampling.read more
Citations
More filters
Journal ArticleDOI
Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions
TL;DR: This work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low-rank matrix approximation, and presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions.
Journal ArticleDOI
Coordinate descent algorithms
TL;DR: A certain problem structure that arises frequently in machine learning applications is shown, showing that efficient implementations of accelerated coordinate descent algorithms are possible for problems of this type.
Journal ArticleDOI
Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function
Peter Richtárik,Martin Takáč +1 more
TL;DR: In this paper, a randomized block-coordinate descent method for minimizing the sum of a smooth and a simple nonsmooth block-separable convex function was developed, and it was shown that the algorithm converges linearly.
Journal ArticleDOI
Minimizing finite sums with the stochastic average gradient
TL;DR: In this paper, the stochastic average gradient (SAG) method is used to optimize the sum of a finite number of smooth convex functions, which achieves a faster convergence rate than black-box SG methods.
Posted Content
Coordinate Descent Algorithms
TL;DR: Coordinate descent algorithms solve optimization problems by successively performing approximate minimization along coordinate directions or coordinate hyperplanes as mentioned in this paper, and they have been used in many applications, such as data analysis, machine learning, and other areas of current interest.
References
More filters
Book
Numerical Analysis
TL;DR: This report contains a description of the typical topics covered in a two-semester sequence in Numerical Analysis, and describes the accuracy, efficiency and robustness of these algorithms.
Book
The Mathematics of Computerized Tomography
TL;DR: In this paper, the Radon transform and related transforms have been studied for stability, sampling, resolution, and accuracy, and quite a bit of attention is given to the derivation, analysis, and practical examination of reconstruction algorithm, for both standard problems and problems with incomplete data.
Journal ArticleDOI
Computerized transverse axial scanning (tomography): Part I. Description of system. 1973.
TL;DR: A technique in which X-ray transmission readings are taken through the head at a multitude of angles: from these data, absorption values of the material contained within the head are calculated on a computer and presented as a series of pictures of slices of the cranium.