448 IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 2, NO. 4, AUGUST 2008
Distributed Object Tracking Using a Cluster-Based
Kalman Filter in Wireless Camera Networks
Henry Medeiros, Johnny Park, Member, IEEE, and Avinash C. Kak
Abstract—Local data aggregation is an effective means to save
sensor node energy and prolong the lifespan of wireless sensor net-
works. However, when a sensor network is used to track moving
objects, the task of local data aggregation in the network presents
a new set of challenges, such as the necessity to estimate, usually in
real time, the constantly changing state of the target based on in-
formation acquired by the nodes at different time instants. To ad-
dress these issues, we propose a distributed object tracking system
which employs a cluster-based Kalman filter in a network of wire-
less cameras. When a target is detected, cameras that can observe
the same target interact with one another to form a cluster and elect
a cluster head. Local measurements of the target acquired by mem-
bers of the cluster are sent to the cluster head, which then estimates
the target position via Kalman filtering and periodically transmits
this information to a base station. The underlying clustering pro-
tocol allows the current state and uncertainty of the target posi-
tion to be easily handed off among clusters as the object is being
tracked. This allows Kalman filter-based object tracking to be car-
ried out in a distributed manner. An extended Kalman filter is nec-
essary since measurements acquired by the cameras are related to
the actual position of the target by nonlinear transformations. In
addition, in order to take into consideration the time uncertainty
in the measurements acquired by the different cameras, it is neces-
sary to introduce nonlinearity in the system dynamics. Our object
tracking protocol requires the transmission of significantly fewer
messages than a centralized tracker that naively transmits all of
the local measurements to the base station. It is also more accu-
rate than a decentralized tracker that employs linear interpolation
for local data aggregation. Besides, the protocol is able to perform
real-time estimation because our implementation takes into con-
sideration the sparsity of the matrices involved in the problem. The
experimental results show that our distributed object tracking pro-
tocol is able to achieve tracking accuracy comparable to the cen-
tralized tracking method, while requiring a significantly smaller
number of message transmissions in the network.
Index Terms—Cameras, clustering, distributed tracking,
Kalman filtering, sensor, wireless camera networks, wireless
sensor networks.
I. INTRODUCTION
I
T IS well known that local data aggregation is an effec-
tive means to save sensor node energy and prolong the
lifespan of wireless sensor networks. This has motivated many
Manuscript received October 30, 2007; revised May 19, 2008. Current ver-
sion published September 17, 2008. This work was supported by Olympus Cor-
poration. The associate editor coordinating the review of this manuscript and
approving it for publication was Prof. Hamid Aghajan.
The authors are with the School of Electrical and Computer Engineering,
Purdue University, Electrical Engineering Building, West Lafayette, IN 47907-
2035 USA (e-mail: hmedeiro@purdue.edu; jpark@purdue.edu; kak@purdue.
edu).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/JSTSP.2008.2001310
previous researchers to employ sensor clustering techniques
to enable local data aggregation for environment monitoring
applications [1]–[5]. However, when a sensor network is used
to track moving objects, the task of local data aggregation in
the network presents a new set of challenges. One challenge
is that the system must be able to estimate the current state
of the target based on information acquired by the nodes at
different time instants while the state of the target is constantly
changing. Another challenge comes from the fact that most
object-tracking systems demand the position of the target object
to be estimated in real time which puts heavy constraints on the
time it takes to carry out local data aggregation in the network.
The work presented in this paper attempts to address these
issues.
In our earlier work [6], we have presented a clustering pro-
tocol to allow for dynamic formation of camera clusters as a
target with specific visual features is detected in the network.
In this paper, we extend that work by employing the Kalman
filter [7]—one of the most commonly used and time-honored
techniques for reliable parameter estimation—to aggregate in-
formation collected by different nodes. We use our clustering
algorithm to manage a decentralized Kalman filter to locally ag-
gregate the data collected by the cameras. The information on a
target object is acquired by the cluster members and transmitted
to the cluster head. The cluster head then aggregates the data
and transmits the information to a base station at a predefined
rate. As the target moves in physical space, so does the corre-
sponding cluster in the network. During cluster propagation, the
state information regarding the target is handed off from cluster
head to cluster head. As we will demonstrate, it is possible for
a single target to result in multiple clusters—this is due to the
directional properties of the cameras. Multiple clusters will also
result from multiple targets executing motions in the physical
space. The results we show in this paper are limited to the case
of clusters formed by the motion of a single target.
This paper is organized as follows. In Section II, we present
an overview of some works related to both clustering in wireless
sensor networks and distributed Kalman filtering. In Section III,
we discuss some of the challenges involved in cluster-based ob-
ject tracking using wireless camera networks and present the
clustering protocol we have designed to cope with these chal-
lenges. Section IV presents the main contribution of this paper,
the cluster-based Kalman filter. In Section V, we present some
experimental results obtained using our wireless camera net-
work simulator and our network of wireless cameras. Finally,
we conclude this paper in Section VI.
1932-4553/$25.00 © 2008 IEEE
Authorized licensed use limited to: Purdue University. Downloaded on June 16, 2009 at 16:28 from IEEE Xplore. Restrictions apply.
MEDEIROS et al.: DISTRIBUTED OBJECT TRACKING USING A CLUSTER-BASED KALMAN FILTER 449
II. RELATED
WORK
A. Event-Driven Clustering Protocols
In environment monitoring applications, the nodes of a sensor
network are usually clustered using one of three different strate-
gies: 1) the nodes may be clustered only once at the system ini-
tialization time; 2) periodically based on some predefined net-
work-wide time interval; and 3) aperiodically on the basis of
some internal node parameter, such as the remaining energy re-
serve at the nodes [1]–[3]. However, in object tracking applica-
tions, clustering must be triggered by the detection of an event
of interest external to the network. This section presents some
of the works that take external events into consideration in the
cluster formation process.
Chen et al. [8] have proposed an algorithm for distributed
target tracking using acoustic information. Their system is
composed of sparsely placed high-capability nodes and densely
spaced low-end sensors. The high-capability nodes act as
cluster heads and the low-end sensors act as cluster members.
Cluster heads close to the detected event become active with
higher probability than cluster heads that are farther from the
event. Similarly, the probability that a cluster member sends
data to the cluster head is proportional to its distance to the
event.
Fang et al. [9] have proposed a distributed aggregate manage-
ment (DAM) protocol, in which nodes that detect energy peaks
become cluster heads, and a tree of cluster members is formed
by their neighbors that detect lower energy levels. When many
targets lie within the same cluster, they use their energy-based
activity monitoring (EBAM) algorithm to count the number of
targets. EBAM assumes that all of the targets are equally strong
emitters of energy and counts the number of targets within a
cluster based on the total energy detected by the cluster. To
drop this assumption, they proposed the expectation-maximiza-
tion-like activity monitoring (EMLAM) algorithm. This algo-
rithm assumes that the targets are initially well separated and
uses a motion prediction model along with message exchanges
among cluster leaders to keep track of the total number of ob-
jects.
Zhang and Cao [10] proposed the dynamic convoy tree-based
collaboration (DCTC) algorithm in which the nodes that can de-
tect an object create a tree rooted at a node near the detected ob-
ject. As the object moves, nodes are added to and pruned from
the tree, and the root moves to nodes closer to the object. This
work is similar to our clustering protocol in that it also provides
mechanisms for cluster propagation as the object moves. How-
ever, no mechanisms are provided for interaction among clusters
since a single cluster is formed to keep track of the target. As we
will see, in camera networks, it may be necessary to have mul-
tiple clusters simultaneously tracking the same target.
Blum et al. [11] proposed a middleware architecture to allow
for distributed applications to communicate with groups of sen-
sors assigned to track multiple events in the environment. Their
architecture is divided into two modules: the entity-management
module (EMM) and the entity connection module (ECM). The
EMM is responsible for creating unique groups of sensors to
track each event, keeping persistent identities to these groups,
and storing information about the state of the event. The ECM
provides end-to-end communication among different groups of
sensors.
All of these works have in common the fact that they are
designed for omnidirectional sensors. Therefore, they do not
account for challenges specific to directional sensors, such as
cameras. One challenge is the fact that physical proximity be-
tween a sensor and the target does not imply that the sensor is
able to acquire information about the target. Hence, distance-
based cluster formation protocols are not directly applicable to
camera networks. The challenges of sensor clustering in wire-
less camera networks will be addressed in detail in Section III.
B. Distributed Kalman Filtering
The idea of distributing the computations involved in esti-
mation problems using Kalman filters in sensor networks has
been a subject of research since the late 1970s [12]. This sec-
tion presents some of the recent contributions in this area.
Olfati–Saber [13] presented a distributed Kalman filter
wherein a system with an
-dimensional measurement vector
is first split into
subsystems of -dimensional measurement
vectors, then these subsystems are individually processed
by micro Kalman filters in the nodes of the network. In this
system, the sensors compute an average inverse covariance and
average measurements using consensus filters. These averaged
values are then used by each node to individually compute the
estimated state of the system using the information form of
the Kalman filter. Even though this approach is effective in an
environment monitoring application where the state vector is
partially known by each node in the network, it is not valid
for an object tracking application where, at a given time, each
node in a small number of nodes knows the entire state vector
(although possibly not accurately).
Nettleton et al. [14] proposed a tree-based architecture in
which each node computes the update equations of the Kalman
filter in its information form and sends the results to its imme-
diate predecessor in the tree. The predecessor then aggregates
the received data and computes a new update. Node asynchrony
is handled by predicting asynchronously received information
to the current time in the receiving node. This approach is scal-
able since the information transmitted between any pair of nodes
is fixed. However, the size of the information matrix is propor-
tional to
, where is the dimension of the state vector. In
a sensor network setting, this information may be too large to
be transmitted between nodes; therefore, methods to effectively
quantize this information may need to be devised.
Regarding quantization, the work by Ribeiro et al. [15],
studied a network environment wherein each node transmits
a single bit per observation, the sign of innovation (SOI), at
every iteration of the filter. The system assumes an underlying
sensor-scheduling mechanism so that only one node transmits
the information at a time. It also assumes the update information
(i.e., the signs of innovations) to be available to each node of
the network. They showed that the mean squared error of their
SOI Kalman filter is closely related to the error of a clairvoyant
Kalman filter, which has access to all of the data in analog form.
There is an interesting tradeoff between the works by Net-
tleton et al. and Ribeiro et al. The former presents a high level of
Authorized licensed use limited to: Purdue University. Downloaded on June 16, 2009 at 16:28 from IEEE Xplore. Restrictions apply.
450 IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 2, NO. 4, AUGUST 2008
(a) (b)
Fig. 1. (a) Multiple clusters tracking the same object in a wireless camera net-
work. Dotted circles represent the communication ranges of the clusters. (b) Two
single-hop clusters in a network of cameras that can communicate in multiple
hops. Blue (dark) circles represent cluster heads, green (light) circles represent
cluster members. The lines connecting the nodes correspond to communication
links among them.
locality (i.e., each node only needs information about its imme-
diate neighbors). On the other hand, a reasonably large amount
of information must be transmitted by each node. The later, by
its turn, requires the transmission of a very small amount of in-
formation by each node; however, the algorithm does not present
locality since the information must be propagated throughout
the network. This kind of tradeoff must be carefully considered
when designing an algorithm for real wireless sensor network
applications.
To the best of our knowledge, the only work that applies
Kalman filtering to a cluster-based architecture for object
tracking using camera networks is that proposed by Goshorn
et
al. [16]. Their system assumes that the network is previously
partitioned into clusters of cameras with similar fields of view.
As the target moves, information within a cluster is handed off
to a neighboring cluster.
III. C
LUSTER-BASED OBJECT TRACKING WITH
WIRELESS CAMERA NETWORKS
Most of the current event-driven clustering protocols assume
that sensors closest to an event-generating target can best ac-
quire information about the target. In wireless camera networks,
however, the distance-based criteria for sensor-node clustering
are not sufficient since, depending on their pointing directions,
physically proximal cameras may view segments of space that
are disjointed and even far from one another. What that means
is that even when only a single object is being tracked, a clus-
tering protocol must allow for the formation of multiple dis-
jointed clusters of cameras to track the same object. An example
is illustrated in Fig. 1(a) where, despite the fact that the cameras
in cluster A cannot communicate with the cameras in cluster B,
both clusters of cameras can track the object. Therefore, mul-
tiple clusters must be allowed to track the same target.
Even if all of the cameras that can detect a common object can
communicate with one another in multiple hops, the commu-
nication overhead involved in tracking the object using a large
Fig. 2. Fragmentation of a single cluster. As the cluster head in (a) leaves the
cluster, it is fragmented into two clusters as illustrated in (b).
cluster may be unacceptable as collaborative processing gen-
erally requires intensive message exchange among the cluster
members. Therefore, rather than creating a single large multihop
cluster to track an object, it is often desirable to have multiple
single-hop clusters that may interact as needed. An example is
illustrated in Fig. 1(b) where, whereas all of the cameras that
can see the same object may constitute a connected graph if we
allow multihop communications, it may be more efficient to re-
quire that two single-hop clusters be formed in this case.
Dynamic cluster formation requires all cluster members to in-
teract to select a cluster head. There are many algorithms avail-
able [17], [18] that could be used for electing a leader from
among all of the cameras that are able to see the same object.
Nevertheless, these algorithms would not work for us since we
must allow for the formation of multiple single-hop clusters (for
the reasons previously explained) and for the election of a sep-
arate leader for each cluster. Therefore, it is necessary to de-
vise a new leader election protocol suitable for the creation of
single-hop clusters in a wireless camera network setting.
After clusters are created to track specific targets, these clus-
ters must be allowed to propagate through the network as the
targets move. Cluster propagation refers to the process of ac-
cepting new members into the cluster as they identify the same
object, removing members that can no longer see the object, and
assigning new cluster heads as the current cluster head leaves the
cluster. Since cluster propagation in wireless camera networks
can be based on distinctive visual features of the target, it is pos-
sible for clusters tracking different objects to propagate inde-
pendently or even overlap if necessary. In other words, cameras
that can detect multiple targets may belong simultaneously to
multiple clusters. Including a new member into a cluster and re-
moving an existing member from a cluster are rather simple op-
erations. However, when a cluster head leaves the cluster, mech-
anisms must be provided to account for the possibility that the
cluster be fragmented into two or more clusters, as illustrated in
Fig. 2.
Since multiple clusters are allowed to track the same target,
if these clusters overlap, they must be able to coalesce into a
single cluster. Coalescence of clusters is made possible by per-
mitting the overhearing of intracluster communications as dif-
ferent clusters come into each other’s communication range.
Overhearing obviously implies intercluster communication. It
is important to note that intercluster communication can play a
Authorized licensed use limited to: Purdue University. Downloaded on June 16, 2009 at 16:28 from IEEE Xplore. Restrictions apply.
MEDEIROS et al.: DISTRIBUTED OBJECT TRACKING USING A CLUSTER-BASED KALMAN FILTER 451
Fig. 3. State transition diagram of a cluster-based object tracking system using
a wireless camera network.
role in intracluster computation of a parameter of the environ-
ment even when cluster merging is not an issue. For example, a
cluster composed of overhead cameras may request information
about the
coordinate of the target from a neighboring cluster
composed of wall-mounted cameras. Therefore, it is necessary
to provide mechanisms to allow intercluster interactions in wire-
less camera networks.
To summarize these points, Fig. 3 illustrates the state tran-
sition diagram of a cluster-based object tracking system using
a wireless camera network. The network initially monitors the
environment. As an object is detected, one or more clusters are
formed to track this object. To keep track of the object, these
clusters must propagate through the network as the object moves
and, if necessary, fragment themselves into smaller clusters. Fi-
nally, if two or more clusters tracking the same object meet each
other, they may interact to share information or coalesce into
larger clusters.
One of the primary contributions of the clustering protocol
we will present is that it does allow for the formation and prop-
agation of multiple clusters. When needed, the protocol also
allows for clusters to interact or coalesce into larger clusters
and for large clusters to fragment into smaller clusters. More-
over, our clustering protocol allows for distributed applications
to be easily implemented in wireless camera networks since it
releases the application of much of the burden of assigning roles
to the cameras (i.e., leader/member) and of the collection of the
data generated by the cameras.
A. Clustering Protocol
In this section, we present our clustering protocol. We believe
that the best way to present the protocol would be to show the
state transition diagram at each node. Such a diagram would
define all of the states of a node as it transitions from initial
object detection to participation in a cluster, to possibly its
role as a leader and, finally, to relinquishing its membership
in the cluster. Unfortunately, such a diagram would be much
too complex for a clear presentation. So instead, we have
opted to present this diagram in three pieces. The individual
pieces we will present in this section correspond to the cluster
formation and head election, cluster propagation, and inter-
cluster communications. The state transition diagram for cluster
propagation includes the transitions needed for cluster coa-
lescence and fragmentation. As the reader will note, our state
transitions allow for wireless camera networks to dynamically
create one or more clusters to track objects based on visual
features. Note that our protocol is lightweight in the sense
that it creates single-level clusters (i.e., clusters composed
only of cameras that can communicate in a single hop), rather
than multiple-level clusters, which incur large communication
overhead and latency during collaborative processing and
require complex cluster-management strategies. Cameras that
can communicate in multiple hops may share information as
needed by intercluster interactions.
1) Cluster Head Election: To select cluster heads for
single-hop clusters, we employ a two-phase cluster head elec-
tion algorithm. In the first phase, nodes compete to find a
node that minimizes (or maximizes) some criterion, such as
the distance from the camera center to the object center in the
image plane. By the end of this phase, at most one camera in a
single-hop neighborhood elects itself leader and its neighbors
join its cluster. During the second phase, cameras that were left
without a leader (because their leader candidate joined another
cluster) identify the next best leader candidate.
As illustrated by the state transition diagram on the left side
of Fig. 4, in the first phase of the cluster head election algorithm,
each camera that detects an object sends a message requesting
the creation of a cluster and includes itself in a list of cluster
head candidates sorted by the cluster head selection criteria.
The cluster creation message includes the values of the cluster
head selection criteria from the sender. After a camera sends a
cluster creation message, it waits for a predefined timeout pe-
riod for cluster creation messages from other cameras. When-
ever a camera receives a cluster creation message from another
camera, it updates the list of cluster head candidates. To make
sure that cameras that detect the object at later moments do not
lose information about the available cluster head candidates, all
of the cameras that can hear the create cluster messages update
their candidate lists. After the end of the timeout period, if the
camera finds itself in the first position of the candidate list, it
sends a message informing its neighbors that it is ready to be-
come the cluster head. If the camera does not decide to become
a cluster head, it proceeds to the second phase of the algorithm.
The first phase of the algorithm guarantees that a single
camera chooses to become a cluster head within its communi-
cation range. However, it might be the case that cameras that
can communicate with the cluster head in multiple hops are
left without a leader. Fig. 5 shows an example of this situation.
Cameras 1 and 2 decide that camera 3 is the best cluster head
candidate. However, camera 3 chooses to become a member
of the cluster headed by camera 4. Hence, cameras 1 and 2 are
left orphans after the first stage of the leader election and must
proceed to the second phase of the algorithm to choose their
cluster heads.
During the second phase of the cluster head election, cam-
eras that did not receive a cluster-ready message after a time
interval remove the first element of the cluster head candidate
list. If the camera then finds itself in the first position of the
Authorized licensed use limited to: Purdue University. Downloaded on June 16, 2009 at 16:28 from IEEE Xplore. Restrictions apply.
452 IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 2, NO. 4, AUGUST 2008
Fig. 4. Cluster head election state transition diagram.
Fig. 5. Orphan cameras after the first stage of the leader election algorithm.
candidate list, it sends a cluster-ready message and becomes a
cluster head. Otherwise, the camera waits for a timeout period
for a cluster-ready message from the next candidate in the list.
This process is illustrated in the right side of the state transition
diagram of Fig. 4. Eventually, the camera will either become
a cluster head or join a cluster from a neighboring camera. To
avoid multiple cameras deciding to become cluster heads simul-
taneously, it is important that the cluster head election criteria
impose strict ordering of the candidates (if it does not, ties must
be broken during the first phase).
The second phase of our leader election algorithm bears some
similarities with Garcia–Molina’s bully election algorithm [19].
As a consequence, the algorithm is not robust to communication
failures in the network. However, the consequences of commu-
nication failures are relatively mild in the sense that as the al-
gorithm terminates, every cluster will have exactly one cluster
head, even if more than one cluster is formed where a single
cluster should. This property holds because each camera even-
tually chooses a cluster head, even if it is itself, and after re-
ceiving a cluster-ready message from a cluster head, a camera
no longer accepts cluster-ready messages. Therefore, we believe
that the simplicity of the algorithm overcomes its relative lack
of robustness.
In the final step of the algorithm, to establish a bidirectional
connection among the cluster head and its members, each
member sends a message to report the cluster head that it has
joined the cluster. This step is not strictly necessary if the
cluster head does not need to know about the cluster members.
However, in general, for collaborative processing, the cluster
head needs to know its cluster members so that it can assign
them tasks and coordinate the distributed processing.
Authorized licensed use limited to: Purdue University. Downloaded on June 16, 2009 at 16:28 from IEEE Xplore. Restrictions apply.
