INVITED
PAPER
Market-Based Multirobot
Coordination: A Survey
and Analysis
When robots work together as a team, the members that perform each task should
be the ones that promise to use the least resources to do the job.
By M. Bernardine Dias, Robert Zlot, Nidhi Kalra, and Anthony Stentz
ABSTRACT
|
Market-based multirobot coordination ap-
proaches have received significant attention and are growing
in popularity within the robotics research community. They
have been successfully implemented in a variety of domains
ranging f rom mapping and exploration to robot soccer. The
research literature on market-based approaches to co ordina-
tion has now reached a critical mass that warrants a survey and
analysis. This paper addresses this need for a survey of the
relevant literature by providing an introduction to market-
based multirobot coordination, a review and analysis of the
state of the art in the field, and a discussion of remaining
research challenges.
KEYWORDS
|
Auctions; market-based coordination; multirobot
teams; resource allocation; task allocation
I. INTRODUCTION
As robots become an integral part of human life, we charge
them with increasingly varied and difficult tasks including
planetary exploration, manufacturing and construction,
medical assistance, search and rescue, and port and
warehouse automation. Like humans, robots working in
challenging domains can potentially perform better by
working together in teams than by working alone. Ideally,
robots will coordinate to redistribute resources amongst
themselves in a way that enables them to accomplish their
mission efficiently and reliably. Coordination can lead to
faster task completion, increased robustness, higher
quality solutions, and the completion of tasks impossible
for single robots. However, these domains simultaneously
present many obstacles to effective coordination, such as
dynamic events, changing task demands, resource failures,
the presence of adversaries, and limited time, energy, com-
putation, communication, sensing, and mobility. Therefore,
coordinating a multirobot team requires overcoming many
formidable research challenges.
Humans have met these coordination challenges for
thousands of years with increasingly sophisticated market
economies. In these economies, self-interested individuals
and groups trade goods and services to maximize their own
profit; simultaneously, this redistribution results in an
efficient production of output for the system as a whole.
Researchers have recently applied the principles of market
economies to multirobot coordination. In market-based
multirobot systems, robots are designed as self-interested
agents that operate in a virtual economy. Both the tasks
that must be completed and the available resources are
commodities of measurable worth that can be traded. For
example, tasks can be assigned to robots via market
mechanisms such as auctions. When a robot completes a
task,itreceivessomepaymentintheformofvirtualmoney
for providing a service to the team. However, the robot
must also pay for the resources it consumed to complete
the task. The essence of market-based approaches is that,
in a well-designed system, the process of robots trading
tasksandresourceswithoneanothertomaximize
individual profit simultaneously improves the efficiency
of the team.
To illustrate this more concretely, consider a team of
robots performing a distributed sensing mission on Mars.
As illustrated in Fig. 1, the robots must gather data from
specific sites of interest to scientists while consuming the
least amount of energy. One important aspect of complet-
ing the mission is to determine which robot should visit
each site. We can solve this problem using a market-based
Manuscript received June 1, 2005; revised June 1, 2006. This work was supported in
part by the Boeing Company under Grant CMU-BA-GTA-1, in part by the U.S. Army
Research Laboratory, Robotics Collaborative Technology Alliance, under Contract
DAAD19-01-2-0012), and in part by the Qatar Foundation for Education, Science and
Community Development.
The authors are with The Robotics Institute, Carnegie Mellon University, Pittsburgh,
PA 15213 USA, and with Carnegie Mellon University in Qatar, Doha, Qatar (e-mail:
mbdias@ri.cmu.edu; robz@ri.cmu.edu; nidhi@ri.cmu.edu; axs@ri.cmu.edu).
Digital Object Identifier: 10.1109/JPROC.2006.876939
Vol. 94, No. 7, July 2006 | Proceedings of the IEEE 12570018-9219/$20.00
2006 IEEE
approach in which robots compete in auctions for each
task of visiting a site. After estimating their resource usage
for an offered task and submitting bids based on those
expected costs, the robot with the best bid is awarded a
contract for that site.
Suppose that we offer a maximum reward of $50 for
each task and that robots incur a cost of $2 for each meter
of travel (since the resource of concern is energy con-
sumed). This $50 is a reserve price that essentially says that
the task should only be attempted if the site can be reached
by increasing one’s path length by less than 25 m. Further
suppose that a robot A is only 5 m from a site S.SinceA
would have to spend $10 to complete the task, it bids $10.
Meanwhile, a robot B thatis10mfromthesitebids$20.A
is awarded the contract because it can perform the task
more efficiently and for less than the reserve price.
This simple example illustrates the basic mechanism of
a market-based approach to coordination. As the problem
increases in complexity with the addition of more robots,
more resources (e.g., time, network bandwidth, computing
power, sensors, etc.), added constraints between the tasks,
dynamically changing tasks, and so forth, the coordination
approach requires added functionality to produce efficient
solutions. We use this distributed sensing scenario
throughout the remainder of the paper to illustrate the
complexities of coordination and the diversity of market-
based approaches.
The earliest examples of market-based multiagent
coordination appeared in the literature over 30 years ago
[1], [2] and have been modified and adopted for multirobot
coordination in more recent years. This paper is motivated
by the growing popularity of market-based approaches and
the lack of a comprehensive review of these approaches.
This paper makes three contributions to the robotics
literature. First, it provides a tutorial on market-based
approaches by discussing the motivating philosophy,
defining the requirements and tradeoffs inherent in such
approaches, analyzing their strengths and weaknesses, and
placing them appropriately in the context of the larger set
of approaches to multirobot coordination. Second, this
paper surveys and analyzes the relevant literature. Finally,
it inspires and directs future research on this topic through
a discussion of remaining challenges.
The scope of this paper is limited to market-based
approaches for coordinating teams that include robots.
Moreover, this review principally considers approaches
that actively reason about the existence of other agents
when coordinating the team, in contrast to approaches in
which agents coexist. Nevertheless, related publications
outside the stated scope of this paper are included as
necessary to augment the discussion.
The following section provides an introduction to
market-based mechanisms for readers less familiar with
the field. This overview is followed by a extensive review of
market-based multirobot coordination approaches to date,
categorized and analyzed across several relevant dimen-
sions: planning, solution quality, scalability, dynamic
events and environments, and heterogeneity. The paper
concludes with a summary of the survey and future
challenges in this research area.
II. OVERVIEW
In this section, we discuss key concepts that will provide a
foundation for the remainder of the paper, including a
definition of market-based approaches and an introduction
to auctions. We then place market-based approaches in the
larger spectrum of coordination approaches.
A. Definition of a Market-Based Approach
Most market-based multirobot and multiagent coordi-
nation approaches share a set of underlying elements.
Market theory provides precise definitions for several of
these elements. Borrowing from both bodies of literature,
we define a market-based multirobot coordination ap-
proach based on the following requirements.
• The team is given an objective that can be
decomposed into subcomponents achievable by
individuals or subteams. The team has access to a
limited set of resources with which to meet this
objective.
• A global objective function quantifies the system
designer’s preferences over all possible solutions.
• An individual utility function (or cost function)
specified for each robot quantifies that robot’s
preferences for its individual resource usage and
contributions towards the team objective given its
current state. Evaluating this function cannot
require global or perfect information about the
state of the team or team objective. Subteam
preferences can also be quantified through a
combination of individual utilities (or costs).
• A mapping is defined between the team objective
function and individual and subteam utilities (or
costs). This mapping addresses how the individual
Fig. 1. An illustration of three robots exploring Mars. The robots’ task
is to gather data around the four craters, which can be achieved by
visiting the highlighted target sites.
1258 Proceedings of the IEEE |Vol.94,No.7,July2006
Dias et al.: Market-Based Multirobot Coordination: A Survey and Analysis
production and consumption of resources and
individuals’ advancement of the team objective
affect the overall solution.
• Resources and individual or subteam objectives can
be redistributed using a mechanism such as an
auction. This mechanism accepts as input team-
mates’ bids, which are computed as a function of
their utilities (or costs), and determines an out-
come that maximizes the mechanism-controlling
agent’s utility (or minimizes the cost). In a well-
designed mechanism, maximizing the mechanism-
controlling agent’s utility (or minimizing cost)
results in improving the team objective function
value.
B. Auctions
Auctions are the most common mechanisms used in
market-based approaches. In an auction, a set of items is
offered by an auctioneer in an announcement phase, and
the participants can make an offer for these items by
submitting bids to the auctioneer. Once all bids are
received or a prespecified deadline has passed, the auction
is then cleared in the winner determination phase by the
auctioneer who decides which items to award and to
whom. In robotic applications, the items for sale are
typically tasks, roles, or resources. The bid prices reflect
the robots’ costs or utilities associated with completing a
task, satisfying a role, or utilizing a resource.
The simplest kind of auction is a single-item auction in
which only one item is offered. In such auctions, each
participant submits a bid, and the auctioneer awards the
item to the highest bidder.
1
Alternatively, the auctioneer
retains the item if no bid beats the auctioneer’s price
(called a reserve price). Bids are usually submitted only to
the auctioneer; such sealed-bid auctions are in contrast to
open-cry auctions where bidders have the benefit of
overhearing the other bids as they are made. There are
two common approaches to determining the sale price of
the auctioned item. In a first-price auction, the sale price is
the same as the winning bid; in a Vickrey auction,thesale
price is the value of the second-highest bid and is intended
to motivate truthful bids from the participants. Some
multirobot systems have used Vickrey auctions (e.g., [3]),
though the resulting allocations are equivalent to first-
price auctions if the robots are designed to behave
truthfully. Wolfstetter provides an excellent introductory
survey into single-item auction theory [4].
Combinatorial auctions are more complex: multiple
items are offered and each participant can bid on any
combination of bundles (i.e., subsets) of these items. This
allows the bidder to explicitly express the synergies
between items. In the context of the Mars distributed
sensing scenario, a bidder can express the positive synergy
between two sites that are close together by bidding only
slightly higher for the bundle containing these tasks than
for either task individually. To express the negative
synergy between two tasks located far from one another,
the bid for the bundle would be much higher than the sum
of the individual costs of the tasks. In general, there are an
exponential number of bundles to consider which makes
bid valuation, communication, and auction clearing
intractable if all bundles are considered [5].
In between these two extremes are multi-item auctions
in which multiple items are offered but the participants
can win at most one item apiece. The maximum number of
awards per auction may also be limited. Multi-item
auctions are a special case of combinatorial auctions
where only bundles of cardinality one are considered;
bidding and clearing become tractable, but the resulting
solutions are generally much less efficient.
C. Costs, Utilities, and Valuation
The example scenario in Section I compares robots’
suitability for tasks in terms of cost. That is, the auction
allocates tasks to the robots with the lowest costs for
performing them and the overall goal is to minimize some
global cost function. As suggested in Section II-B, in some
systems bids are compared based on utilities, in which case
the highest bids win auctions and the system attempts to
maximize the global utility function. Utilities often encap-
sulate multiple factors, some representing the benefit or
expected quality of task execution and others representing
cost estimates. Cost estimates can also include diverse
factors such as the time taken to compute solutions and
the loss of efficiency caused by transitioning between tasks.
As an example of utility, Gerkey and Mataric
´
[6] propose
taking the difference of quality and cost to calculate utility,
assuming the units of cost and quality are directly
comparable. Thus, utility and cost functions that combine
multiple factors often require finding a reasonable set of
weights between the different components considered.
The process of estimating costs for bid valuation can
also be difficult. Though participants in the market may
have well-defined cost or utility functions, these functions
still rely on having accurate models of the world state and
may require computationally expensive operations. For
example, the cost to complete the task of driving to a goal
site depends on having an accurate map of the environ-
ment; however, the robots may be working in an unknown,
partially known, or changing environment. When there are
multiple goal locations, determining the cost to perform
even one task can require solving multiple path planning
problems and an instance of the traveling salesman
problem (TSP), the latter being NP-hard. Thus, heuristics
and approximation algorithms are commonly used, imply-
ing that bid prices may not always be entirely accurate.
Inaccurate bids can result in tasks not being awarded to the
robots best able to complete them. In this case reauction-
ing tasks can often improve solution quality.
1
We will assume utility maximization here; the case of cost
minimization is analogous, with awards going to the lowest cost bidders.
Vol. 94, No. 7, July 2006 | Proceedings of the IEEE 1259
Dias et al.: Market-Based Multirobot Coordination: A Survey and Analysis
D. The Range of Coordination Approaches
The goal in virtually all robotic application domains is to
generate optimal solutions in a timely manner. Unfortunate-
ly, many multirobot coordination problems are NP-hard.
The challenges are compounded by team considerations
that include operation in dynamic and uncertain environ-
ments, inconsistent information, unreliable and limited
communication, interaction with humans, and various
system and component failures. A spectrum of coordination
approaches has emerged to negotiate these demands.
At one end of the spectrum, fully centralized ap-
proaches employ a single agent to coordinate the entire
team. In theory, this agent can produce optimal solutions
by gathering all relevant information and planning
for the entire team. In reality, fully centralized
approaches are rarely tractable for large teams, can
suffer from a single point of failure, have high
communication demands, and are usually sluggish
to respond to local changes. Thus, centralized ap-
proaches are most suited for applications involving
small teams and static environments or easily
available global information.
At the other end of the spectrum, in fully
distributed systems, robots rely solely on local
knowledge. Such approaches are typically very fast,
flexible to change, and robust to failures, but can produce
highly suboptimal solutions, since good local solutions
may not necessarily aggregate to a good global solution.
Applications where large teams carry out relatively simple
tasks with no strict requirements for efficiency are best
served by fully distributed coordination schemes.
A vast majority of coordination approaches have
elements that are centralized and distributed and thus
reside in the middle of the spectrum. Market-based
approaches fall into this hybrid category, and, in some
instances, they can opportunistically adapt to dynamic
conditions to produce more centralized or more distrib-
uted solutions. Market mechanisms can distribute much of
the planning and execution over the team and thereby
retain the benefits of distributed approaches, including
robustness, flexibility, and speed [7], [8]. Auctions quickly
and concisely assemble team information in a single
location to make decisions about distributing resources;
in some cases they provide guarantees of solution quality
[5], [9]. Market-based approaches may also incorporate
methods of opportunistically coordinating subteams in a
centralized manner [10], [11]. Nevertheless, market-based
approaches are not without their weaknesses. In domains
where fully centralized approaches are feasible, market-
based approaches can be more complex to implement and
produce poorer solutions. In domains where fully dis-
tributed approaches suffice, market approaches can be
unnecessarily complex in design and have greater com-
munication and computation requirements.
The sections that follow discuss market-based multi-
robot coordination in greater detail along the dimensions
mentioned in the introduction. Each section introduces
the topic and its challenges, defines the goals and
appropriate evaluation metrics, reviews the relevant
literature, and identifies remaining research challenges.
III. PLANNING
In multirobot teams, planning can be required to
coordinate robots to accomplish the team mission.
Unfortunately, optimal planning problems for multirobot
systems are typically NP-hard [12]. The challenge then is
to have tractable planning that produces efficient solu-
tions. Market-based approaches manage this by distribut-
ing planning over the entire team to produce solutions
quickly. When required or when resources permit, markets
can behave in a more centralized fashion and plan over
larger portions of the team to improve solution quality.
Here, we consider different layers at which planning arises
in a multirobot system and how these planning problems
are handled by various market-based approaches.
A. Related Work
1) Planning and Task Allocation: Task allocation is the
problem of feasibly assigning a set of tasks to a team in a
way that optimizes a global objective function. Many
special cases of task allocation appear frequently in the
literature; here, we offer a general and formal definition
that allows us to discuss and compare them.
Definition 1: Given a set of robots R,letR :¼ 2
R
be the
set of all possible robot subteams. An allocation of a set T
of tasks to R is a function, A : T !R, mapping each task to
a subset of robots responsible for completing it. Equiva-
lently, R
T
is the set of all possible allocations of the tasks T
to the team of robots R.LetT
r
ðAÞ be the set of tasks
allocated to subteam r in allocation A.
Definition 2: The Multirobot Task Allocation Prob-
lem:GivenasetoftasksT, a set of robots R,andacost
function for each subset of robots r 2Rspecifying the cost
of completing each subset of tasks, c
r
: 2
T
! R
þ
[f1g,
find the allocation A
2R
T
that minimizes a global
objective function C : R
T
! R
þ
[f1g.
Auctions quickly and concisely
assemble team information in a
single location to make decisions
about distributing resources;
in some cases they provide
guarantees of solution quality.
1260 Proceedings of the IEEE |Vol.94,No.7,July2006
Dias et al.: Market-Based Multirobot Coordination: A Survey and Analysis
Gerkey and Mataric
´
[6] provide a taxonomy for some
variants of the task allocation problem, distinguishing
between: single-task (ST) and multitask (MT) robots;
single-robot (SR) and multirobot (MR) tasks; and instan-
taneous (IA) and time-extended (TA) assignment. In
instantaneous assignment robots do not plan for future
allocations and are only concerned with the one task they
are carrying out at the moment or for which they are
bidding. In time-extended assignment robots have more
information and can come up with longer term plans
involving task sequences or schedules. Definition 2
encompasses each of the types of task allocation in the
taxonomy, but in general describes TA task allocation. IA
allocation can be represented as a special case where all
cost functions map to infinity for any subsets of tasks with
cardinality greater than one. Further, if we allow the sets
of tasks T and robots R to be time dependent (i.e., TðtÞ,
RðtÞ) and require the objective function be minimized at
every instant of time or over the entire history, then the
definition also covers online and dynamic domains where
tasks and robots may be added or removed over time (see
Section VI). This definition also implies that task allo-
cation is NP-hard in general, as the multidepot traveling
salesman problem is a special case [12].
Market-based approaches distribute planning required
for task allocation through the auction process: each
robot or group of robots locally plans the achievement of
the offered tasks, computes its costs, and encapsulates the
costs in its bids. This process is illustrated in the
introduction of this paper for a distributed sensing task
on Mars: each robot determined its own cost of visiting
different sites. Most existing market-based approaches
fall into the SR-ST category in the task allocation taxon-
omy. Several assume instantaneous assignment (IA) [7],
[13]–[15], while others allow for time-extended assign-
ment (TA), introducing an additional layer of planning
whereby robots sequence [9], [16]–[20] or schedule
[21]–[23] a list of tasks and can therefore explicitly rea-
son about the dependencies between multiple tasks and
upcoming commitments. More recently, market-based
systems have addressed the allocation of multiple-robot
tasks (MR-ST) [24], [25], including human–robot tasks [26].
Market-based mechanisms for task allocation can also
be differentiated as centralized or distributed. Centralized
mechanisms have the ability to find optimal solutions (e.g.,
through combinatorial auctions [5], [16]) or provide
bounds on solution quality [9], but in general can require
an exponential amount of computation and communica-
tion [5]. Distributed mechanisms [17], [18] act as anytime
algorithms and require less computation and communica-
tion resources, but are not guaranteed to find optimal
solutions and have no known approximation bounds.
TraderBots [17] attempts to find a balance between these
two approaches by opportunistically allowing Bpockets[ of
centralized optimization to emerge within subgroups of
the team when resources permit. In our distributed
sensing example, for instance, the team might begin with
a suboptimal allocation of sites, perhaps caused by an
inaccurate map of the environment resulting in inaccurate
bids. At some point during execution (perhaps when map
information is more accurate), a robot might find a better
distribution of sites for some subset of its teammates. The
robot’s motivation for group optimization is that it can
pocket the cost difference as profit by winning the tasks
from the original holders and subcontracting them to the
new holders. Simultaneously, this results in a better team
solution. Solution quality and scalability aspects of these
different approaches are discussed in more detail in
Sections IV and V, respectively.
Related problems of allocating constrained subtasks
and roles can have additional planning requirements.
a) Allocating Constrained Subtasks: In many domains,
tasks are temporally constrained with respect to one
another. They may be partially ordered or may need to
start or finish within a common time frame. For instance,
consistency may be important in our Mars distributed
sensing task, so we might require that samples from
particular sites be collected at the same time. In the case of
partially ordered tasks, one can use a central allocator to
auction only those tasks whose predecessors have been
completed [27]. Alternatively, during assignment, robots
can incorporate the cost of meeting constraints into their
bids [28]. In terms of Definition 2, a violation of
constraints can be modeled as infinite values for local
cost functions or the global objective function. Constraints
can add another dimension to the bid valuation and
auction clearing processes and may thus increase compu-
tation requirements. Often, robots must also coordinate
during execution to reschedule and accommodate team
and task changes that have occurred since the initial
allocation [21], [22]. In these cases, robots must be able to
determine when and how the rescheduling should occur.
b) Allocating Roles and Instantaneous Assignment: In
team games one usually assigns positions such as Bprimary
offense[ or Bsupporting defense[ instead of tasks such as
Bshoot the ball[ or Bcapture a rebound.[ These positions
can be classified as roles. More generally a role defines a
collection of related actions or behaviors. Indeed, in many
domains it is more natural to think of teammates playing
roles than completing distinct tasks. In market-based ap-
proaches, role allocation can use the same auction–bid–
award protocol as task allocation. However, robots can
usually take on only one role at any given time (SR-ST-IA or
MR-ST-IA) and generate bids by evaluating a fitness
function that reflects how well its current state matches
the requirements of the role. Once allocated, a robot locally
plans the execution of actions and behaviors specified by its
role. Market-based role allocation has been demonstrated in
robot soccer [13], [15] and treasure hunt [29] domains.
Instantaneous assignment (IA) also arises in cases
where the tasks being allocated are short-term partial
actions that bring the team goal closer to being realized.
Vol. 94, No. 7, July 2006 | Proceedings of the IEEE 1261
Dias et al.: Market-Based Multirobot Coordination: A Survey and Analysis
