S
Satyen Kale
Researcher at Google
Publications - 129
Citations - 11789
Satyen Kale is an academic researcher from Google. The author has contributed to research in topics: Regret & Convex optimization. The author has an hindex of 42, co-authored 119 publications receiving 9188 citations. Previous affiliations of Satyen Kale include Yahoo! & IBM.
Papers
More filters
Proceedings Article
On the Convergence of Adam and Beyond
TL;DR: It is shown that one cause for such failures is the exponential moving average used in the algorithms, and suggested that the convergence issues can be fixed by endowing such algorithms with `long-term memory' of past gradients.
Journal ArticleDOI
Logarithmic regret algorithms for online convex optimization
TL;DR: Several algorithms achieving logarithmic regret are proposed, which besides being more general are also much more efficient to implement, and give rise to an efficient algorithm based on the Newton method for optimization, a new tool in the field.
Journal ArticleDOI
The Multiplicative Weights Update Method: A Meta-Algorithm and Applications
TL;DR: A simple meta-algorithm is presented that unifies many of these disparate algorithms and derives them as simple instantiations of the meta-Algorithm.
Proceedings Article
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
Sai Praneeth Karimireddy,Satyen Kale,Mehryar Mohri,Sashank J. Reddi,Sebastian U. Stich,Ananda Theertha Suresh +5 more
TL;DR: This work obtains tight convergence rates for FedAvg and proves that it suffers from `client-drift' when the data is heterogeneous (non-iid), resulting in unstable and slow convergence, and proposes a new algorithm (SCAFFOLD) which uses control variates (variance reduction) to correct for the ` client-drifts' in its local updates.
Proceedings ArticleDOI
Privacy, accuracy, and consistency too: a holistic solution to contingency table release
TL;DR: This work proposes a solution that provides strong guarantees for all three desiderata simultaneously, privacy, accuracy, and consistency among the tables, and applies equally well to the logical cousin of the contingency table, the OLAP cube.