Proceedings ArticleDOI
Proximal diffusion for stochastic costs with non-differentiable regularizers
Stefan Vlaski,Ali H. Sayed +1 more
- pp 3352-3356
Reads0
Chats0
TLDR
This work represents the implementation as the cascade of three operators and invoke Banach's fixed-point theorem to establish that, despite gradient noise, the stochastic implementation is able to converge in the mean-square-error sense within O(μ) from the optimal solution, for a sufficiently small step-size parameter, μ.Abstract:
We consider networks of agents cooperating to minimize a global objective, modeled as the aggregate sum of regularized costs that are not required to be differentiable. Since the subgradients of the individual costs cannot generally be assumed to be uniformly bounded, general distributed subgradient techniques are not applicable to these problems. We isolate the requirement of bounded subgradients into the regularizer and use splitting techniques to develop a stochastic proximal diffusion strategy for solving the optimization problem by continuously learning from streaming data. We represent the implementation as the cascade of three operators and invoke Banach's fixed-point theorem to establish that, despite gradient noise, the stochastic implementation is able to converge in the mean-square-error sense within O(μ) from the optimal solution, for a sufficiently small step-size parameter, μ.read more
Citations
More filters
Journal ArticleDOI
Decentralized Consensus Optimization With Asynchrony and Delays
TL;DR: An asynchronous, decentralized algorithm for consensus optimization that involves both primal and dual variables, uses fixed step-size parameters, and provably converges to the exact solution under a random agent assumption and both bounded and unbounded delay assumptions.
Journal ArticleDOI
Proximal Multitask Learning Over Networks With Sparsity-Inducing Coregularization
TL;DR: This work considers multitask learning problems where clusters of nodes are interested in estimating their own parameter vector and proposes a fully distributed algorithm that relies on minimizing a global mean-square error criterion regularized by nondifferentiable terms to promote cooperation among neighboring clusters.
Posted Content
Decentralized Consensus Optimization with Asynchrony and Delays
TL;DR: In this paper, the authors proposed an asynchronous, decentralized algorithm for consensus optimization, where each agent can compute and communicate independently at different times, for different durations, with the information it has even if the latest information from its neighbors is not yet available.
Proceedings ArticleDOI
Diffusion stochastic optimization with non-smooth regularizers
TL;DR: It is shown how the regularizers can be smoothed and how the Pareto solution can be sought by appealing to a multi-agent diffusion strategy under conditions that are weaker than assumed earlier in the literature.
Proceedings ArticleDOI
Decentralized consensus optimization with asynchrony and delays
TL;DR: An asynchronous, decentralized algorithm for consensus optimization that involves both primal and dual variables, uses fixed step-size parameters, and provably converges to the exact solution under a random agent assumption and both bounded and unbounded delay assumptions.
References
More filters
Book
Matrix Analysis
Roger A. Horn,Charles R. Johnson +1 more
TL;DR: In this article, the authors present results of both classic and recent matrix analyses using canonical forms as a unifying theme, and demonstrate their importance in a variety of applications, such as linear algebra and matrix theory.
Journal ArticleDOI
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Amir Beck,Marc Teboulle +1 more
TL;DR: A new fast iterative shrinkage-thresholding algorithm (FISTA) which preserves the computational simplicity of ISTA but with a global rate of convergence which is proven to be significantly better, both theoretically and practically.
Journal ArticleDOI
Consensus and Cooperation in Networked Multi-Agent Systems
TL;DR: A theoretical framework for analysis of consensus algorithms for multi-agent networked systems with an emphasis on the role of directed information flow, robustness to changes in network topology due to link/node failures, time-delays, and performance guarantees is provided.
Book
Parallel and Distributed Computation: Numerical Methods
TL;DR: This work discusses parallel and distributed architectures, complexity measures, and communication and synchronization issues, and it presents both Jacobi and Gauss-Seidel iterations, which serve as algorithms of reference for many of the computational approaches addressed later.
Book
Proximal Algorithms
Neal Parikh,Stephen Boyd +1 more
TL;DR: The many different interpretations of proximal operators and algorithms are discussed, their connections to many other topics in optimization and applied mathematics are described, some popular algorithms are surveyed, and a large number of examples of proxiesimal operators that commonly arise in practice are provided.