Local divergence of Markov chains and the analysis of iterative load-balancing schemes
08 Nov 1998-pp 694-703
TL;DR: This work develops a general technique for the quantitative analysis of iterative distributed load balancing schemes, and applies this technique to obtain bounds on the number of rounds required to achieve coarse balancing in general networks, cycles and meshes in these models.
Abstract: We develop a general technique for the quantitative analysis of iterative distributed load balancing schemes. We illustrate the technique by studying two simple, intuitively appealing models that are prevalent in the literature: the diffusive paradigm, and periodic balancing circuits (or the dimension exchange paradigm). It is well known that such load balancing schemes can be roughly modeled by Markov chains, but also that this approximation can be quite inaccurate. Our main contribution is an effective way of characterizing the deviation between the actual loads and the distribution generated by a related Markov chain, in terms of a natural quantity which we call the local divergence. We apply this technique to obtain bounds on the number of rounds required to achieve coarse balancing in general networks, cycles and meshes in these models. For balancing circuits, we also present bounds for the stronger requirement of perfect balancing, or counting.
Citations
More filters
TL;DR: This work analyzes the averaging problem under the gossip constraint for an arbitrary network graph, and finds that the averaging time of a gossip algorithm depends on the second largest eigenvalue of a doubly stochastic matrix characterizing the algorithm.
Abstract: Motivated by applications to sensor, peer-to-peer, and ad hoc networks, we study distributed algorithms, also known as gossip algorithms, for exchanging information and for computing in an arbitrarily connected network of nodes. The topology of such networks changes continuously as new nodes join and old nodes leave the network. Algorithms for such networks need to be robust against changes in topology. Additionally, nodes in sensor networks operate under limited computational, communication, and energy resources. These constraints have motivated the design of "gossip" algorithms: schemes which distribute the computational burden and in which a node communicates with a randomly chosen neighbor. We analyze the averaging problem under the gossip constraint for an arbitrary network graph, and find that the averaging time of a gossip algorithm depends on the second largest eigenvalue of a doubly stochastic matrix characterizing the algorithm. Designing the fastest gossip algorithm corresponds to minimizing this eigenvalue, which is a semidefinite program (SDP). In general, SDPs cannot be solved in a distributed fashion; however, exploiting problem structure, we propose a distributed subgradient method that solves the optimization problem over the network. The relation of averaging time to the second largest eigenvalue naturally relates it to the mixing time of a random walk with transition probabilities derived from the gossip algorithm. We use this connection to study the performance and scaling of gossip algorithms on two popular networks: Wireless Sensor Networks, which are modeled as Geometric Random Graphs, and the Internet graph under the so-called Preferential Connectivity (PC) model.
2,634 citations
Cites background from "Local divergence of Markov chains a..."
...Distributed averaging has also been studied in the context of distributed load balancing ([43]), where nodes (processors) exchange tokens in order to uniformly distribute tokens over all the processors in the network (the number of tokens is constrained to be integral, so exact averaging is not possible)....
[...]
09 Aug 2010
TL;DR: An overview of recent gossip algorithms work, including convergence rate results, which are related to the number of transmitted messages and thus the amount of energy consumed in the network for gossiping, and the use of gossip algorithms for canonical signal processing tasks including distributed estimation, source localization, and compression.
Abstract: Gossip algorithms are attractive for in-network processing in sensor networks because they do not require any specialized routing, there is no bottleneck or single point of failure, and they are robust to unreliable wireless network conditions. Recently, there has been a surge of activity in the computer science, control, signal processing, and information theory communities, developing faster and more robust gossip algorithms and deriving theoretical performance guarantees. This paper presents an overview of recent work in the area. We describe convergence rate results, which are related to the number of transmitted messages and thus the amount of energy consumed in the network for gossiping. We discuss issues related to gossiping over wireless links, including the effects of quantization and noise, and we illustrate the use of gossip algorithms for canonical signal processing tasks including distributed estimation, source localization, and compression.
868 citations
[...]
TL;DR: This work describes simple randomized distributed algorithms which achieve consensus to the extent that the discrete nature of the problem permits.
773 citations
Cites background or result from "Local divergence of Markov chains a..."
...It might also be fruitful to bound the deviation of the discrete valued system from a real valued system, similar to the work of Rabani et al. (1998)....
[...]
...Key words: Sensor fusion, quantization, distributed detection, estimation algorithms, stochastic systems, random processes, clock synchronization....
[...]
...If the tasks are indivisible and of equal size, then a quantized consensus distribution represents such an equalized distribution of tasks....
[...]
...…2005; Boyd et al., 2005), load balancing in processor networks (Bertsekas and Tsitsiklis, 1997; Ghosh and Muthukrishnan, 1996; Ghosh et al., 1999; Rabani et al., 1998; Subramanian and Scherson, 1994), clock synchronization (Giridhar and Kumar, 2006; Akar and Shorten, 2006), and multi-agent…...
[...]
...Rabani et al. (1998) obtain a bound on the deviation of a particular discretization of diffusion from its real valued approximation....
[...]
744 citations
13 Mar 2005
TL;DR: This work analyzes the averaging problem under the gossip constraint for arbitrary network, and finds that the averaging time of a gossip algorithm depends on the second largest eigenvalue of a doubly stochastic matrix characterizing the algorithm.
Abstract: Motivated by applications to sensor, peer-to-peer and ad hoc networks, we study distributed asynchronous algorithms, also known as gossip algorithms, for computation and information exchange in an arbitrarily connected network of nodes. Nodes in such networks operate under limited computational, communication and energy resources. These constraints naturally give rise to "gossip" algorithms: schemes which distribute the computational burden and in which a node communicates with a randomly chosen neighbor. We analyze the averaging problem under the gossip constraint for arbitrary network, and find that the averaging time of a gossip algorithm depends on the second largest eigenvalue of a doubly stochastic matrix characterizing the algorithm. Using recent results of Boyd, Diaconis and Xiao (2003), we show that minimizing this quantity to design the fastest averaging algorithm on the network is a semi-definite program (SDP). In general, SDPs cannot be solved distributedly; however, exploiting problem structure, we propose a subgradient method that distributedly solves the optimization problem over the network. The relation of averaging time to the second largest eigenvalue naturally relates it to the mixing time of a random walk with transition probabilities that are derived from the gossip algorithm. We use this connection to study the performance of gossip algorithm on two popular networks: wireless sensor networks, which are modeled as geometric random graphs, and the Internet graph under the so-called preferential connectivity model.
627 citations
Cites methods from "Local divergence of Markov chains a..."
...Distributed averaging has also been studied in the context of distributed load balancing ([ RSW98 ]), where an analysis based on Markov chains is used to obtain bounds on the time required to achieve averaging (upto the integer constraint) upto a certain accuracy....
[...]
References
More filters
11,456 citations
Book•
01 Jan 1973
4,243 citations
1,438 citations
Book•
01 Jan 1971
1,367 citations