S
Sarit Khirirat
Researcher at Royal Institute of Technology
Publications - 17
Citations - 481
Sarit Khirirat is an academic researcher from Royal Institute of Technology. The author has contributed to research in topics: Computer science & Stochastic gradient descent. The author has an hindex of 6, co-authored 12 publications receiving 404 citations.
Papers
More filters
Proceedings Article
The Convergence of Sparsified Gradient Methods
Dan Alistarh,Torsten Hoefler,Mikael Johansson,Nikola Konstantinov,Sarit Khirirat,Cedric Renggli +5 more
TL;DR: The authors showed that sparsifying gradients by magnitude with local error correction provides convergence guarantees, for both convex and non-convex smooth objectives, for data-parallel SGD.
Posted Content
The Convergence of Sparsified Gradient Methods
Dan Alistarh,Torsten Hoefler,Mikael Johansson,Sarit Khirirat,Nikola Konstantinov,Cedric Renggli +5 more
TL;DR: This article showed that sparsifying gradients by magnitude with local error correction provides convergence guarantees, for both convex and non-convex smooth objectives, for data-parallel SGD.
Posted Content
Distributed learning with compressed gradients
TL;DR: A unified analysis framework for distributed gradient methods operating with staled and compressed gradients is presented and non-asymptotic bounds on convergence rates and information exchange are derived for several optimization algorithms.
Proceedings ArticleDOI
Mini-batch gradient descent: Faster convergence under data sparsity
TL;DR: This paper derives explicit expressions for how data sparsity affects the range of admissible step-sizes and the convergence factors of minibatch gradient descent and demonstrates improved performance of the update rules compared to the traditional mini-batch gradient descent algorithm.
Proceedings ArticleDOI
Gradient compression for communication-limited convex optimization
TL;DR: This paper establishes and strengthens the convergence guarantees for gradient descent under a family of gradient compression techniques, and derives admissible step sizes and quantifies both the number of iterations and the numbers of bits that need to be exchanged to reach a target accuracy.