We study a distributed computation model for optimizing a sum of convex objective functions corresponding to multiple agents. For solving this (not necessarily smooth) optimization problem, we consider a subgradient method that is distributed among the agents. The method involves every agent minimizing his/her own objective function while exchanging information locally with other agents in the network over a time-varying topology. We provide convergence results and convergence rate estimates for the subgradient method. Our convergence rate results explicitly characterize the tradeoff between a desired accuracy of the generated approximate optimal solutions and the number of iterations needed to achieve the accuracy.

http://www.ifp.illinois.edu/~angelia/distributed_journal_final.pdf

Distributed Subgradient Methods for Multi-Agent Optimization

This paper reviews some main results and progress in distributed multi-agent coordination, focusing on papers published in major control systems and robotics journals since 2006. Distributed coordination of multiple vehicles, including unmanned aerial vehicles, unmanned ground vehicles, and unmanned underwater vehicles, has been a very active research subject studied extensively by the systems and control community. The recent results in this area are categorized into several directions, such as consensus, formation control, optimization, and estimation. After the review, a short discussion section is included to summarize the existing research and to propose several promising research directions along with some open problems that are deemed important for further investigations.

/pdf/an-overview-of-recent-progress-in-the-study-of-distributed-1xnb6e12se.pdf

An Overview of Recent Progress in the Study of Distributed Multi-Agent Coordination

This article reviews some main results and progress in distributed multi-agent coordination, focusing on papers published in major control systems and robotics journals since 2006. Distributed coordination of multiple vehicles, including unmanned aerial vehicles, unmanned ground vehicles and unmanned underwater vehicles, has been a very active research subject studied extensively by the systems and control community. The recent results in this area are categorized into several directions, such as consensus, formation control, optimization, task assignment, and estimation. After the review, a short discussion section is included to summarize the existing research and to propose several promising research directions along with some open problems that are deemed important for further investigations.

An Overview of Recent Progress in the Study of Distributed Multi-agent Coordination

We study learning in a setting where agents receive independent noisy signals about the true value of a variable and then communi cate in a network. They naively update beliefs by repeatedly taking weighted averages of neighbors' opinions. We show that all opinions in a large society converge to the truth if and only if the influence of the most influential agent vanishes as the society grows. We also identify obstructions to this, including prominent groups, and pro vide structural conditions on the network ensuring efficient learn ing. Whether agents converge to the truth is unrelated to how quickly consensus is approached. (JEL D83, D85, Z13)

/pdf/naive-learning-in-social-networks-and-the-wisdom-of-crowds-15d43bhpge.pdf

Naïve Learning in Social Networks and the Wisdom of Crowds

We consider distributed optimization by a collection of nodes, each having access to its own convex function, whose collective goal is to minimize the sum of the functions. The communications between nodes are described by a time-varying sequence of directed graphs, which is uniformly strongly connected. For such communications, assuming that every node knows its out-degree, we develop a broadcast-based algorithm, termed the subgradient-push, which steers every node to an optimal value under a standard assumption of subgradient boundedness. The subgradient-push requires no knowledge of either the number of agents or the graph sequence to implement. Our analysis shows that the subgradient-push algorithm converges at a rate of O (ln t/√t), where the constant depends on the initial values at the nodes, the subgradient norms, and, more interestingly, on both the consensus speed and the imbalances of influence among the nodes.

Distributed optimization over time-varying directed graphs

We study the (perfect Bayesian) equilibrium of a sequential learning model over a general social network. Each individual receives a signal about the underlying state of the world, observes the past actions of a stochastically generated neighbourhood of individuals, and chooses one of two possible actions. The stochastic process generating the neighbourhoods defines the network topology. We characterize pure strategy equilibria for arbitrary stochastic and deterministic social networks and characterize the conditions under which there will be asymptotic learning—convergence (in probability) to the right action as the social network becomes large. We show that when private beliefs are unbounded (meaning that the implied likelihood ratios are unbounded), there will be asymptotic learning as long as there is some minimal amount of “expansion in observations”. We also characterize conditions under which there will be asymptotic learning when private beliefs are bounded.

/pdf/bayesian-learning-in-social-networks-1uunyb0q8v.pdf

Bayesian Learning in Social Networks

We consider the problem of cooperatively minimizing the sum of convex functions, where the functions represent local objective functions of the agents. We assume that each agent has information about his local function, and communicate with the other agents over a time-varying network topology. For this problem, we propose a distributed subgradient method that uses averaging algorithms for locally sharing information among the agents. In contrast to previous works on multi-agent optimization that make worst-case assumptions about the connectivity of the agents (such as bounded communication intervals between nodes), we assume that links fail according to a given stochastic process. Under the assumption that the link failures are independent and identically distributed over time (possibly correlated across links), we provide almost sure convergence results for our subgradient algorithm.

https://pages.stern.nyu.edu/~ilobel/dist-stoch-opt.pdf

Distributed Subgradient Methods for Convex Optimization Over Random Networks

We study distributed algorithms for solving global optimization problems in which the objective function is the sum of local objective functions of agents and the constraint set is given by the intersection of local constraint sets of agents. We assume that each agent knows only his own local objective function and constraint set, and exchanges information with the other agents over a randomly varying network topology to update his information state. We assume a state-dependent communication model over this topology: communication is Markovian with respect to the states of the agents and the probability with which the links are available depends on the states of the agents. We study a projected multi-agent subgradient algorithm under state-dependent communication. The state-dependence of the communication introduces significant challenges and couples the study of information exchange with the analysis of subgradient steps and projection errors. We first show that the multi-agent subgradient algorithm when used with a constant stepsize may result in the agent estimates to diverge with probability one. Under some assumptions on the stepsize sequence, we provide convergence rate bounds on a “disagreement metric” between the agent estimates. Our bounds are time-nonhomogeneous in the sense that they depend on the initial starting time. Despite this, we show that agent estimates reach an almost sure consensus and converge to the same optimal solution of the global optimization problem with probability one under different assumptions on the local constraint sets and the stepsize sequence.

https://pages.stern.nyu.edu/~ilobel/state-dep-optimization.pdf

Distributed multi-agent optimization with state-dependent communication

We study distributed algorithms for solving global optimization problems in which the objective function is the sum of local objective functions of agents and the constraint set is given by the intersection of local constraint sets of agents. We assume that each agent knows only his own local objective function and constraint set, and exchanges information with the other agents over a randomly varying network topology to update his information state. We assume a state-dependent communication model over this topology: communication is Markovian with respect to the states of the agents and the probability with which the links are available depends on the states of the agents. In this paper, we study a projected multi-agent subgradient algorithm under state-dependent communication. The algorithm involves each agent performing a local averaging to combine his estimate with the other agents' estimates, taking a subgradient step along his local objective function, and projecting the estimates on his local constraint set. The state-dependence of the communication introduces significant challenges and couples the study of information exchange with the analysis of subgradient steps and projection errors. We first show that the multi-agent subgradient algorithm when used with a constant stepsize may result in the agent estimates to diverge with probability one. Under some assumptions on the stepsize sequence, we provide convergence rate bounds on a "disagreement metric" between the agent estimates. Our bounds are time-nonhomogeneous in the sense that they depend on the initial starting time. Despite this, we show that agent estimates reach an almost sure consensus and converge to the same optimal solution of the global optimization problem with probability one under different assumptions on the local constraint sets and the stepsize sequence.

Distributed Multi-Agent Optimization with State-Dependent Communication

We consider the problem of designing optimal mechanisms for settings where agents have dynamic private information. We present the virtual-pivot mechanism, which is optimal in a large class of environments that satisfy a separability condition. The mechanism satisfies a rather strong equilibrium notion (it is periodic ex post incentive compatible and individually rational). We provide both necessary and sufficient conditions for immediate incentive compatibility for mechanisms that satisfy periodic ex post incentive compatibility in future periods. The result also yields a strikingly simple mechanism for selling a sequence of items to a single buyer. We also show that the allocation rule of the virtual-pivot mechanism has a very simple structure (a virtual index) in multiarmed bandit settings. Finally, we show through examples that the relaxation technique we use does not produce optimal dynamic mechanisms in general nonseparable environments.

/pdf/optimal-dynamic-mechanism-design-and-the-virtual-pivot-3d79ov4d2f.pdf

Ilan Lobel

Papers

Bayesian Learning in Social Networks

Distributed Subgradient Methods for Convex Optimization Over Random Networks

Distributed multi-agent optimization with state-dependent communication

Distributed Multi-Agent Optimization with State-Dependent Communication

Optimal Dynamic Mechanism Design and the Virtual-Pivot Mechanism