Consensus and Cooperation in Networked
Multi-Agent Systems
Author: Reza Olfati-Saber
1
, J. Alex Fax
2
, and Richard M. Murray
3
Note: Proceedings of the IEEE, Feb 2006 (under review)
Technical Report Number:
TR06-004, Thayer School of Engineering, Hanover, NH, April 2006.
1
Dartmouth College, Thayer School of Engineering, Hanover, NH.
2
Northrop Grumman NSD, 21240 Burbank Blvd., Woodland Hills, CA 91367.
3
California Institute of Technology, Control and Dynamical Systems, Pasadena, CA
91125.
Consensus and Cooperation in Networked Multi-Agent Systems
?
Reza Olfati-Sab e r
1
, J. Alex Fax
2
, and Richard M. Murray
3
1
Thayer School of Engineering, Dartmouth College, Hanover, NH 03755, olfati@seas.ucla.edu
2
Northrop Grumman NSD, 21240 Burbank Blvd., Woodland Hills, CA 91367, Alex.Fax@ngc.com
3
Control and Dynamical Systems, California Institute of Technology, Pasadena, CA 91125, murray@caltech.edu
Summary. This paper provides a theoretical framework for analysis of consensus algorithms for multi-agent net-
worked systems with an emphasis on the role of directed information flow, robustness to changes in network topology
due to link/node failures, time-delays, and performance guarantees. An overview of basic concepts of information
consensus in networks and methods of convergence and performance analysis for the algorithms are provided. Our
analysis framework is based on tools from matrix theory, algebraic graph theory, and control theory. We discuss the
connections between consensus problems in networked dynamic systems and diverse applications including synchro-
nization of coupled oscillators, flocking, formation control, fast consensus in small-world networks, Markov pro cess es
and gossip-based algorithms, load balancing in networks, rendezvous in space, distributed sensor fusion in sensor
networks, and belief propagation. We establish direct connections between spectral and structural properties of com-
plex networks and the speed of information diffusion of consensus algorithms. A brief introduction is provided on
networked systems with nonlocal information flow that are considerably faster than distributed systems with lattice-
type nearest neighbor interactions. Simulation results are presented that demonstrate the role of small-world effects
on the sp eed of consensus algorithms and cooperative control of multi-vehicle formations.
Key words: multi-agent systems, consensus algorithms, information fusion, networked systems, cooperative
control, algebraic connectivity, graph Laplacians, flocking, synchronization of coupled oscillators, small-world
networks
1 Introduction: Consensus and Cooperation
Consensus problems have a long history in computer science and form the foundation of the field of distributed
computing [51]. Formal study of consensus problems in groups of experts originated in management science
and statistics in 1960’s (See DeGroot [19] and references therein). The ideas of statistical consensus theory by
DeGroot reappeared two decades later in aggregation of information with uncertainty obtained from multiple
sensors
4
[6] and medical experts [93].
Distributed computation over networks has a tradition in systems and control theory starting with the
pioneering work of Borkar and Varaiya [10] and Tsitsiklis and Athens [90] on asynchron ous asymptotic agree-
ment problem for distributed decision-making systems. This effort was summarized in [7] with applications
to parallel computing.
In networks of agents (or dynamic systems), “consensus” means to reach an agreement regarding a
certain quantity of interest that depends on the state of all agents. A “consensus algorithm” (or protocol) is
?
Submitted to IEEE Pro ceedings on August 5, 2005. Revised on Feb 2, 2006.
4
This is known as sensor fusion and is an important application of modern consensus algorithms that will be
discussed later.
2 Reza Olfati-Saber, J. Alex Fax, and Richard M. Murray
an interaction rule that specifies the information exchange between an agent and all of its neighbors on the
network
5
.
The theoretical framework for posing and solving consensus p roblems for networked dynamic systems was
introduced by Olfati-Saber and Murray in [76, 70] building on the earlier work of Fax and Murray [27, 28].
The study of the alignment problem involving reaching an agreement—without computing any objective
functions—appe ared in the work of Jadbabaie et al. [38]. Further theoretical extensions of this work were
presented in [59, 74] with a look toward treatme nt of directed information flow in networks as shown in
Fig. 1 (a).
The common motivation behind the work in [10, 90, 70] is the rich history of consensus protocols in
computer science [51], whereas Jadbabaie et al. [38] attempted to provide a formal analysis of emergence of
alignment in the simplified model of flocking by Viscek et al. [91]. The setup in [70] was originally created
with the vision of designing agent-based amorphous computers [1, 62] for collaborative information processing
in networks. Later, [70] was used in development of flocking algorithms with guaranteed convergence and
the c apability to deal with obstacles and adversarial agents [66].
Graph Laplacians and their spectral properties [29, 58, 55, 32] are important graph-related matrices that
play a crucial role in convergence analysis of consensus and alignment algorithms. Graph Laplacians are
an important point of focus of this paper. It is worth mentioning that the second smallest eigenvalue of
graph Laplacians called algebraic connectivity quantifies the speed of convergence of consensus algorithms.
The notion of algebraic c onnectivity of graphs has appeared in a variety of other areas including low-density
parity-check codes (LDPC) in information theory and communications [84], Ramanujan graphs [50] in number
theory and quantum chaos, and combinatorial optimization problems such as the max-cut problem [58].
More recently, there has been a tremendous surge of interest—among researchers from various disciplines
of engineering and science—in problems related to multi-agent networked systems with close ties to consensus
problems. This includes subjects such as consensus [47, 9, 5, 15, 54, 8, 79], collective behavior of flocks
and swarms [66, 80, 60, 95, 30], sensor fusion [64, 71, 33], random networks [34, 73], synchronization of
coupled oscillators [81, 39, 72, 73, 14], algebraic connectivity
6
of complex networks [65, 12, 43], asynchronous
distributed algorithms [54, 26], formation control for multi-robot systems [21, 68, 69, 24, 89, 88, 48, 96,
20], optimization-based cooperative control [75, 42, 37, 2], dynamic graphs [56, 61, 40, 99], complexity of
coordinated tasks [36, 44, 52, 53], and consensus-based belief propagation in Bayesian networks [78, 67]. A
detailed discussion of selected applications will be presented shortly.
In this paper, we focus on the work described in five key papers—namely, Jadbabaie, Lin, and Morse
[38], Olfati-Saber and Murray [70], Fax and Murray [28], Moreau [59], and Ren and Beard [74]— that have
been instrumental in paving the way for more recent advances in study of self-organizing networked systems,
or swarms. These networked systems are comprised of locally interacting mobile/static agents equipped with
dedicated sensing, computing, and communication devices. As a result, we now have a better understanding
of complex phenomena such as flocking [66], or design of novel information fusion algorithms for sensor
networks that are robust to node and link failures [64, 86, 98, 13, 78, 67].
Gossip-based algorithms such as the push-sum protocol [41] are important alternatives in computer
science to Laplacian-based consensus algorithms in this paper. Markov proc es se s establish an interesting
connection betwe en the information propagation speed in these two categories of algorithms proposed by
computer s cientists and control theorists [11].
The contribution of this paper is to present a cohesive overview of the key results on theory and appli-
cations of consensus problems in networked systems in a unified framework. This includes basic notions in
information consensus and control theoretic methods for convergence and performance analysis of consensus
protocols that heavily rely on matrix theory and spectral graph theory. A byproduct of this framework is
to demonstrate that seemingly different consensus algorithms in the literature [38, 28, 70, 59, 74] are closely
related. Applications of conse nsus problems in areas of interest to researchers in computer science, physics,
biology, mathematics, robotics, and control theory are discussed in this introduction.
5
The term “nearest neighbors” is more commonly used in physics than “neighbors” when applied to particle/spin
interactions over a lattice (e.g. Ising model).
6
To be defined in Section 2.1.
Consensus and Cooperation in Networked Multi-Agent Systems 3
1.1 Consensus in Networks
The interaction topology of a network of agents is represented using a directed graph G = (V, E) with the
set of nodes V = {1, 2, . . . , n} and edges E ⊆ V × V . The neighbors of agent i are denoted by N
i
= {j ∈ V :
(i, j) ∈ E}. According to [70], a simple consensus algorithm to reach an agreement regarding the state of n
integrator agents with dynamics ˙x
i
= u
i
can be expressed as an nth-order linear system on a graph:
˙x
i
(t) =
X
j∈N
i
(x
j
(t) − x
i
(t)) + b
i
(t), x
i
(0) = z
i
∈ R, b
i
(t) = 0 (1)
The collective dynamics of the group of agents following protocol (1) can be written as
˙x = −Lx (2)
where L = [l
ij
] is the graph Laplacian of the network and its elements are defined as follows:
l
ij
=
−1 i 6= j,
|N
i
| i = j.
(3)
Here, |N
i
| denotes the number of neighbors of node i (or out-degree of node i). Fig. 1 shows two equivalent
forms of the consensus algorithm in equations (1) and (2) for agents with a scalar state. The role of the input
bias b in Fig. 1 (b) is defined later.
According to the definition of graph Laplacian in (3), all row-sums of L are zero because of
P
j
l
ij
= 0.
Therefore, L always has a zero eigenvalue λ
1
= 0. This zero eigenvalues corresponds to the eigenvector
1 = (1, . . . , 1)
T
because 1 belongs to the null-space of L (L1 = 0). In other words, an equilibrium of
system (2) is a state in the form x
∗
= (α, . . . , α)
T
= α1 where all nodes agree. Based on analytical tools
from algebraic graph theory [32], we later show that x
∗
is a unique equilibrium of (2) (up to a constant
multiplicative factor) for connected graphs.
i
j
+
b
y = x
Collective System
Consensus Feedback
u
OutputControl
Input bias
!
!
(a) (b)
Fig. 1. Two equivalent forms of consensus algorithms: (a) a network of integrator agents in which agent i receives
the state x
j
of its neighbor, agent j, if there is a link (i, j) connecting the two nodes; and (b) the block diagram for a
network of interconnected dynamic systems all with identical transfer functions P (s) = 1/s. The collective networked
system has a diagonal transfer function and is a MIMO (multi-input multi-output) linear system.
One can show that for a connected network, the equilibrium x
∗
= (α, . . . , α)
T
is globally exponentially
stable. Moreover, the consensus value is α = 1/n
P
i
z
i
that is equal to the average of the initial values.
4 Reza Olfati-Saber, J. Alex Fax, and Richard M. Murray
This implies that irrespective of the initial value of the state of each agent, all agents reach an asymptotic
consensus regarding the value of the function f(z) = 1/n
P
i
z
i
.
While the calculation of f(z) is simple for small networks, its implications for very large networks is
more interesting. For example, if a network has n = 10
6
nodes and each node can only talk to log
10
(n) = 6
neighbors, finding the average value of the initial conditions of the nodes is more complicated. The role of
protocol (1) is to provide a systematic consensus mechanism in such a large network to compute the average.
There are a variety of functions that can be computed in a similar fashion using synchronous or asynchronous
distributed algorithms (See [70, 67, 5, 13, 54]).
1.2 The f -Consensus Problem and Meaning of Cooperation
To understand the role of cooperation in performing coordinated tasks, we need to distinguish between uncon-
strained and constrained consensus problems. An unconstrained consensus problem is simply the alignment
problem in which it suffices that the state of all agents asymptotically be the same. In contrast, in distributed
computation of a function f(z), the state of all agents has to asymptotically become equal to f(z), meaning
that the consensus problem is constrained. We refer to this constrained consensus problem as the f-consensus
problem.
Solving the f -consensus problem is a cooperative task and requires willing participation of all the agents.
To demonstrate this fact, suppose a single agent decides not to cooperate with the rest of the agents and keep
its state unchanged. Then, the overall task cannot be performed despite the fact that the rest of the agents
reach an agreement. Furthermore, there could be scenarios in which multiple agents that form a coalition do
not cooperate with the rest and removal of this coalition of agents and their links might render the network
disconnected. In a disconnected network, it is impossible for all nodes to reach an agreement (unless all nodes
initially agree which is a trivial case).
From the above discussion, cooperation can be informally interpreted as “giving consent to providing
one’s s tate and following a common protocol that serves the group objective.”
One might think that solving the alignment problem is not a cooperative task. The justification is that if
a single agent (called a leader) leaves its value unchanged, all others will asymptotically agree with the leader
according to the consensus proto c ol and an alignment is reached. However, if there are multiple leaders where
two of whom are in disagreement, then no consensus can be asymptotically reached. Therefore, alignment is
in general a cooperative task as well.
Formal analysis of the behavior of systems that involve more than one type of agent is more complicated,
particularly, in presence of adversarial agents in non-cooperative games [31, 82]. The focus of this paper is
on cooperative multi-agent systems.
1.3 Iterative Consensus Algorithms and Markov Chains
In Section 2, we show how an iterative consensus algorithm that corresponds to the discrete-time version of
system (1) is a Markov chain
π(k + 1) = π(k)P (4)
with P = I − L and a small > 0. Here, the ith element of the row vector π(k) denotes the probability of
being in state i at iteration k. It turns out that for any arbitrary graph G with Laplacian L and a sufficiently
small , the matrix P satisfies the property
P
j
p
ij
= 1 with p
ij
≥ 0, ∀i, j. Hence, P is a valid transition
probability matrix for the Markov chain in (4). The reason matrix theory [35] is s o widely used in analysis of
consensus algorithms [38, 28, 70, 59, 74, 56] is primarily due to the structure of P in equation (4) and its
connection to graphs
7
.
There are interesting connections between this Markov chain and the speed of information diffusion in
gossip-based averaging algorithms [41, 11].
7
In honor of the pioneering contributions of Oscar Perron (1907) to the theory of nonnegative matrices, were refer
to P as the Perron Matrix of graph G (See Section 2.3 for details).
