$50<,890:?6- ,559?3<(50($50<,890:?6- ,559?3<(50(
"*/63(83?644659"*/63(83?644659
,7(8:4,5:(3 (7,89
,7(8:4,5:6-,*/(50*(35.05,,805.
7730,+,*/(50*9
;3?
09:80);:,+",(8*/(5+!,9*;,=0:/!6)6:(5+",5968#,(4909:80);:,+",(8*/(5+!,9*;,=0:/!6)6:(5+",5968#,(49
<,,2(9
$50<,890:?6- ,559?3<(50(
,68.,(5:68
(85,.0,,3365$50<,890:?
!%01(?;4(8
$50<,890:?6- ,559?3<(50(
2;4(8.8(97;7,55,+;
;03/,84," ,8,08(
$50<,890:?6- ,559?3<(50(
!65 ,:,8965
(8:46;:/$50<,890:?
",,5,>:7(.,-68(++0:065(3(;:/689
6336=:/09(5+(++0:065(3=6829(:/::798,7690:68?;7,55,+;4,(4'7(7,89
!,*644,5+,+0:(:065!,*644,5+,+0:(:065
(9<,,2(5:68,68.,;4(8!%01(? ,8,08(;03/,84," ,:,8965!65!;9(50,3("05./
"(510<(5+"73,:@,86/509:80);:,+",(8*/(5+!,9*;,=0:/!6)6:(5+",5968#,(49
,7(8:4,5:(3 (7,89
/::798,7690:68?;7,55,+;4,(4'7(7,89
69:7805:<,89065
86*,,+05.96-:/,:/5:,85(:065(365-,8,5*,650,3+(5+",8<0*,!6)6:0*9"!
/,3+;3?
#/097(7,809769:,+(:"*/63(83?644659/::798,7690:68?;7,55,+;4,(4'7(7,89
68468,05-684(:06573,(9,*65:(*:8,7690:68?76)6>;7,55,+;
09:80);:,+",(8*/(5+!,9*;,=0:/!6)6:(5+",5968#,(4909:80);:,+",(8*/(5+!,9*;,=0:/!6)6:(5+",5968#,(49
)9:8(*:)9:8(*:
&,*6590+,89,(8*/(5+8,9*;,(7730*(:065905=/0*//,:,86.,5,6;9.86;796-(.,5:9/;4(5986)6:9
9:(:0*(5+46)03,9,59689,5:,8(5;5256=5);03+05.(5++097,89,=/03,-6336=05..8(+0,5:905
:,47,8(:;8,(5+*65*,5:8(:0656-:6>059(5+366205.-680446)03,/;4(59#/,(.,5:9+,736?:/,9:(:0*
9,59689(5+4(05:(05305,6-90./:<090)030:?(5+*644;50*(:065*655,*:0<0:?=/,5,<,876990)3,"05*,
+0--,8,5:(.,5:9/(<,+0--,8,5:9,59689(5+:/,8,-68,+0--,8,5:70,*,96-05-684(:065*644;50*(:06509
5,*,99(8?-68:(9205.:/,5,:=6829/(805.05-684(:065(5+-68*65:863
5(+/6*5,:=68209-684,+)?(.86;76-46)03,/69:9;765(=08,3,9936*(35,:=68205:,8-(*,:09(
:,4768(8?5,:=682-684,+=0:/6;::/,(0+6-(5?,9:()309/,+05-8(9:8;*:;8,68*,5:8(30@,+(+40509:8(:065
9,59685,:=682*65909:96-(*633,*:0656-9,59689(5++09:80);:,+6<,8964,(8,(:/(:-684(5(+/6*
5,:=682;8/,:,86.,5,6;9:,(496-(.,5:99,5968986)6:9(5+/;4(59*659:0:;:,+09:80);:,+
(+(7:0<,9,59685,:=6829(5+(8,=,339;0:,+-68:(92905,>:8,4,,5<08654,5:9,97,*0(33?=/,5:/,
,5<08654,5:(346+,3(5+:/,:(9297,*0A*(:0659(8,;5*,8:(05(5+:/,9?9:,4/(9:6(+(7::60:
7730*(:06596-:/09=682*6<,89,(8*/(5+8,9*;,-68A89:8,9765+,894650:6805.(5+9;8<,033(5*,(5+
05-8(9:8;*:;8,786:,*:065
&,*64)05,5,:=68205.9,5905.(5+*65:863:6*65:863:/,B6=6-05-684(:065059,(8*/(5+8,9*;,05
;5256=5,5<08654,5:9"7,*0A*(33?:/098,9,(8*/,>(405,936*(30@(:06505(5,5<08654,5:=0:/56
05-8(9:8;*:;8,9;*/(9();8505.);03+05.-68)6:/9,59689(5+86)6:905-684(:065B6=(*8699(
9,59685,:=682:/(:*(536*(30@,65:/,B?-68+,30<,805.:/,469:8,3,<(5:(5+*;88,5:05-684(:065:60:9
*659;4,84(05:(0505.*;88,5:4(79(5+(;:64(:05.36*(30@(:065;905.-,,+)(*2-864:/,9,5968
5,:=682:6*65:863:/,(;:65646;986)6:9-6873(*05.9,59689*633,*:05.+(:(-8649,59689(5+36*(:05.
:(8.,:9(5++,30<,805.:/,05-684(:065.(:/,8,+-864:/,9,59685,:=68205:,.8(:,+(9(.36)(3
70*:;8,:6/;4(5;9,89#/,7(7,8=033+,:(036;8:,*/50*(38,9;3:905:/,9,(8,(9(5++,9*80),(5
05:,.8(:,+,>7,804,5:-685(<0.(:06505);8505.);03+05.9
644,5:9644,5:9
69:7805:<,89065
86*,,+05.96-:/,:/5:,85(:065(365-,8,5*,650,3+(5+",8<0*,!6)6:0*9
"!
/,3+;3?
;:/689;:/689
<,,2(9,68.,(5:68!%01(?;4(8;03/,84," ,8,08(!65 ,:,8965(50,3(!;9"(510<
"05./(5+6/5"73,:@,8
#/09*65-,8,5*,7(7,809(<(03()3,(:"*/63(83?644659/::798,7690:68?;7,55,+;4,(4'7(7,89
Distributed Search and Rescue with Robot and Sensor Teams
Aveek Das
∗
, George Kantor
‡
, Vijay Kumar
∗
, Guilherme Pereira
∗
, Ron Peterson
†
, Daniela Rus
†
, Sanjiv Singh
‡
, John Spletzer
∗
1 Motivation
We consider search and rescue applications in which hetero-
geneous groups of agents (humans, robots, static and mobile
sensors) enter an unknown building and disperse while follow-
ing gradients in temperature and concentration of toxins, and
looking for immobile humans. The agents deploy the static
sensors and maintain line of sight visibility and communica-
tion connectivity whenever possible. Since different agents
have different sensors and therefore different pieces of infor-
mation, communication is necessary for tasking the network,
sharing information, and for control.
An ad-hoc network is formed by a group of mobile hosts
upon a wireless local network interface. It is a temporary net-
work formed without the aid of any established infrastructure
or centralized administration. A sensor network consists of a
collection of sensors and distributed over some area that form
an ad-hoc network. Our heterogeneous teams of agents (sen-
sors, robots, and humans) constitute distributed adaptive sen-
sor networks and are well-suited for tasks in extreme environ-
ments, especially when the environmental model and the task
specifications are uncertain and the system has to adapt to it.
Applications of this work cover search and rescue for first re-
sponders, monitoring and surveillance, and infrastructure pro-
tection.
We combine networking, sensing, and control to control the
flow of information in search and rescue in unknown environ-
ments. Specifically, this research examines (1) localization in
an environment with no infrastructure such as a burning build-
ing (for both sensors and robots) (2) information flow across
a sensor network that can localize on the fly for delivering the
most relevant and current information to its consumer, main-
taining current maps, and automating localization; (3) using
feedback from the sensor network to control the autonomous
robots for placing sensors, collecting data from sensors, and
locating targets; and (4) delivering the information gathered
from the sensor network (integrated as a global picture) to hu-
man users. The paper will detail our technical results in these
4 areas and describe an integrated experiment for navigation
in burning buildings.
2 Localization
Localization in dynamic environments such as posed by search
and rescue operations is difficult because no infrastructure can
be presumed and because simple assumptions such as line of
sight to known features can not be guaranteed. We have been
investigating the use of low cost radio beacons that can be
placed in the environment by rescue personnel or carried by
robots. These radio beacons provide range to a receiver and
since their position is unknown to start and can potentially
∗
Department of Computer Science, University of Pennsylvania
†
Department of Computer Science, Dartmouth
‡
Robotics Institute, Carnegie Mellon University
Sensors
Robots
Figure 1: (Left) An ad-hoc network of robots and Mote sensors
deployed in a burning building at the Allegheny Fire Academy,
Aug 23, 2002 (from an experimental exercise involving CMU,
Dartmouth, and U. Penn). (Right) The temperature gradient
graph collected using an ad-hoc network of Mote sensors.
change during operation, it is necessary to localize both the re-
ceiver and the beacons simultaneously. This problem is often
known as Simultaneous Localization and Mapping (SLAM)
although typically a receiver is able to measure both range and
bearing to features.
We have adapted the well-known estimation techniques of
Kalman filtering, Markov methods, and Monte Carlo local-
ization to solve the problem of robot localization from range-
only measurements [KS02] [SKS02]. All three of these meth-
ods estimate robot position as a distribution of probabilities
over the space of possible robot positions. In the same work
we presented an algorithm capable of solving SLAM in cases
where approximate a priori estimates of robot and landmark
locations exist. The primary difficulty stems from the annu-
lar distribution of potential relative locations that results from
a range only measurement. Since the distribution is highly
non-Gaussian, SLAM solutions based on Kalman filtering fal-
ter. In theory, Markov methods (probability grids) and Monte
Carlo methods (particle filtering) have the flexibility to handle
annular distributions. Unfortunately, the scaling properties of
these methods severely limit the number of landmarks that can
be mapped. In truth, Markov and Monte Carlo methods have
much more flexibility than we need; they can represent arbi-
trary distributions while we need only to deal with very well
structured annular distributions. What is needed is a compact
way to represent annular distributions together with a com-
putationally efficient way of combining annular distributions
with each other and with Gaussian distributions. In most cases,
we expect the results of these combinations to be well approx-
imated by mixtures of Gaussians so that standard techniques
such as Kalman filtering or multiple hypothesis tracking could
be applied to solve the remaining estimation problem.
3 Information Flow
Sensors detect information about the area they cover. They can
store this information locally or forward it to a base station for
further analysis and use. Sensors can also use communica-
tion to integrate their sensed values with the rest of the sensor
landscape. Users of the network (robots or people) can use this
information as they traverse the network.
We have developed distributed protocols for navigation
tasks in which a distributed sensor field guides a user across
the filed [LdRR03]. We use the localization techniques pre-
sented above to compute environmental maps and sensor
maps, such as temperature gradients. These maps are then
used for human and robot navigation to a target, while avoid-
ing danger (hot areas).
Figure 1(Right) shows the layout of a room in which a fire
was started. We have collected a temperature gradient map
during the fire burning experiment as shown in Figure 1. The
Mote sensors
1
were deployed by hand at the locations marked
in the figure. The sensors computed multi-hop communication
paths to a base station placed at the door. Data was sent to the
base station over a period of 30 minutes.
We used the structure of the data we collected during the
fire burning exercise to develop a navigation guidance algo-
rithm designed to guide a user to the door, in a hop-by-hop
fashion. We have deployed 12 Mote sensors along corridors in
our building and guide a human user out of the building. Us-
ing an interactive device that can transmit directional feedback
called a Flashlight [PR02] a human user was directed across
the field. For each interaction, the user did a rotation scan un-
til the Flashlight was pointed in the direction computed from
the sensor data. The user then walked in that direction to the
next sensor. Each time we recorded the correct direction and
the direction detected by the Flashlight.
4 Control of a Network of Robots
Robots augment the surveillance capabilities of a sensor net-
work by using mobility. Each robot must use partial state in-
formation derived from its sensors and from the communica-
tion network to control in cooperation with other robots the
distribution of robots and the motion of the team. We treat
this as a problem of formation control where the motion of the
team is modeled as an element of a Lie group, while the shape
of the formation is a point in shape space. We seek abstrac-
tions and control laws that allow partial state information to
be used effectively and in a scalable manner.
Our platforms are car-like robots equipped with omnidirec-
tional cameras as their primary sensors. The communication
among the robots relies on IEEE 802.11 networking. By using
information from its camera system each robot is only able to
estimate its distance and bearing from their teammates. How-
ever, if two robots exchange their bearing to each other, they
are also able to estimate their relative orientations [SDF
+
01].
We use this idea to combine the information of a group of two
or more robots in order to improve the knowledge of the group
about their relative position.
We have developed control protocols for using such a team
of robots in connection with a sensor network to explore a
known building. We assume that a network of Mote sensors
previously deployed in the environment guide the robots to-
wards the source of heat. The robots can modify their trajecto-
ries and still find the building exit. The robots can also switch
1
Each Mote sensor (http://today.CS.Berkeley.EDU/tos/) consists
of an Atmel ATMega128 microcontroller a 916 MHz RF transceiver
a UART and a 4Mbit serial flash. A Mote runs for approximately one
month on two AA batteries. It includes light, sound, and temperature
sensors, but other types of sensors may be added. Each Mote runs the
TinyOS operating system.
0 100 200 300 400 500 600 700 800 900
0
50
100
150
200
250
300
350
400
FIRE
EXIT
x (cm)
y (cm)
(a)
0 100 200 300 400 500 600 700 800 900
0
50
100
150
200
250
300
350
400
FIRE
EXIT
x (cm)
y (cm)
(b)
Figure 2: Three robots switching motion plans in real time in
order to get information from the hottest spot of the building.
In (b) a gradient of temperature is obtained from a network of
Mote sensors distributed on the ground.
between the potential fields (or temperature gradients) com-
puted and stored in the sensor network (see Figure 2). The
first switch occurs automatically when the first robot encoun-
ters a Mote sensor at a given location. The robots move toward
the fire and stop at a safer distance (given by the temperature
gradient). They stay there until they are asked to evacuate the
building, at which point they use the original potential field to
find the exit.
5 User Feedback
When robots or people interact with the sensor network, it
becomes an extension of their capabilities, basically extend-
ing their sensory systems and ability to act over a much large
range. We have developed software that allows an intuitive,
immersive display of environments. Using, panoramic imag-
ing sensors that can be carried by small robots into the heart
of a damaged structure, the display can be coupled to head
mounted, head tracking sensors that enable a remote operator
to look around in the environment without the delay associated
with mechanical pan and tilt mechanisms.
The data collected from imaging systems such as visible
cameras and IR cameras are displayed on a wearable computer
to give the responder the most accurate and current informa-
tion. Distributed protocols collect data from the geographi-
cally dispersed sensor network and integrate this data into a
global map such as a temperature gradient that can also be dis-
played on a wearable computer to the user.
References
G. Kantor and S. Singh. Preliminary results in range only lo-
calization and mapping. In IEEE Intl. Conf. on Robotics
and Automation, pages 1819–1825, 2002.
Q. Li, M. de Rosa, and D. Rus. Distributed algorithms for
guiding navigation across a sensor net. In submitted to Mo-
biHoc 2003, 2003.
R. Peterson and D. Rus. Interacting with a sensor network. In
Proc. of Australian Conf. on Robotics an Automation, 2002.
J. Spletzer, A. K. Das, R. Fierro, C. J. Taylor, V. Kumar, and
J. P. Ostrowski. Cooperative localization and control for
multi-robot manipulation. In IEEE/RSJ Intl. Conf. on Intel-
ligent Robots and Systems, 2001.
S. Singh, G. Kantor, and D. Strelow. Recent results in exten-
sions to simultaneous localization and mapping. In Proc. of
International Symposium of Experimental Robotics, 2002.
