Exact Diffusion for Distributed Optimization and Learning—Part I: Algorithm Development
TLDR
In this paper, a distributed optimization strategy with guaranteed exact convergence for a broad class of left-stochastic combination policies was developed, which is applicable to locally balanced combination matrices which are more general and able to endow the algorithm with faster convergence rates, more flexible step-size choices, and improved privacy-preserving properties.Abstract:
This paper develops a distributed optimization strategy with guaranteed exact convergence for a broad class of left-stochastic combination policies. The resulting exact diffusion strategy is shown in Part II of this paper to have a wider stability range and superior convergence performance than the EXTRA strategy. The exact diffusion method is applicable to locally balanced left-stochastic combination matrices which, compared to the conventional doubly stochastic matrix, are more general and able to endow the algorithm with faster convergence rates, more flexible step-size choices, and improved privacy-preserving properties. The derivation of the exact diffusion strategy relies on reformulating the aggregate optimization problem as a penalized problem and resorting to a diagonally weighted incremental construction. Detailed stability and convergence analyses are pursued in Part II of this paper and are facilitated by examining the evolution of the error dynamics in a transformed domain. Numerical simulations illustrate the theoretical conclusions.read more
Citations
More filters
Journal ArticleDOI
A Decentralized Proximal-Gradient Method With Network Independent Step-Sizes and Separated Convergence Rates
TL;DR: This paper proposes a novel proximal-gradient algorithm for a decentralized optimization problem with a composite objective containing smooth and nonsmooth terms that is as good as one of the two convergence rates that match the typical rates for the general gradient descent and the consensus averaging.
Journal ArticleDOI
Decentralized Stochastic Optimization and Machine Learning: A Unified Variance-Reduction Framework for Robust Performance and Fast Convergence
TL;DR: A unified algorithmic framework that combines variance reduction with gradient tracking to achieve robust performance and fast convergence and provides explicit theoretical guarantees of the corresponding methods when the objective functions are smooth and strongly convex.
Journal ArticleDOI
Decentralized Optimization Over Time-Varying Directed Graphs With Row and Column-Stochastic Matrices
TL;DR: A distributed optimization algorithm that minimizes a sum of convex functions over time-varying, random directed graphs that relies on a novel information mixing approach that exploits both row- and column-stochastic weights to achieve agreement toward the optimal solution when the underlying graph is directed.
Journal ArticleDOI
Variance-Reduced Decentralized Stochastic Optimization With Accelerated Convergence
TL;DR: The GT-VR framework as discussed by the authors is a stochastic and decentralized framework to minimize a finite-sum of functions available over a network of nodes, which is particularly suitable for problems where large-scale, potentially private data, cannot be collected or processed at a centralized server.
Journal ArticleDOI
FROST—Fast row-stochastic optimization with uncoordinated step-sizes
TL;DR: Fast Row-stochastic-Optimization with uncoordinated STep-sizes (FROST) as mentioned in this paper is an optimization algorithm applicable to directed graphs that does not require the knowledge of out-degrees.
References
More filters
Journal ArticleDOI
Monte Carlo Sampling Methods Using Markov Chains and Their Applications
TL;DR: A generalization of the sampling method introduced by Metropolis et al. as mentioned in this paper is presented along with an exposition of the relevant theory, techniques of application and methods and difficulties of assessing the error in Monte Carlo estimates.
Journal ArticleDOI
A survey on sensor networks
TL;DR: The current state of the art of sensor networks is captured in this article, where solutions are discussed under their related protocol stack layer sections.
Journal ArticleDOI
Information consensus in multivehicle cooperative control
TL;DR: Theoretical results regarding consensus-seeking under both time invariant and dynamically changing communication topologies are summarized in this paper, where several specific applications of consensus algorithms to multivehicle coordination are described.
Journal ArticleDOI
Toward a smart grid: power delivery for the 21st century
S. Massoud Amin,Bruce Wollenberg +1 more
TL;DR: The security, agility, and robustness/survivability of a large-scale power delivery infrastructure that faces new threats and unanticipated conditions is presented.