scispace - formally typeset
Search or ask a question
Book ChapterDOI

A Probabilistic Approach to Integrating Multiple Cues in Visual Tracking

12 Oct 2008-pp 225-238
TL;DR: This paper demonstrates empirically that the ordering of the cues is nearly inconsequential, and that the approach is superior to other approaches such as Independent Integration and Hierarchical Integration in terms of flexibility and robustness.
Abstract: This paper presents a novel probabilistic approach to integrating multiple cues in visual tracking. We perform tracking in different cues by interacting processes. Each process is represented by a Hidden Markov Model, and these parallel processes are arranged in a chain topology. The resulting Linked Hidden Markov Models naturally allow the use of particle filters and Belief Propagation in a unified framework. In particular, a target is tracked in each cue by a particle filter, and the particle filters in different cues interact via a message passing scheme. The general framework of our approach allows a customized combination of different cues in different situations, which is desirable from the implementation point of view. Our examples selectively integrate four visual cues including color, edges, motion and contours. We demonstrate empirically that the ordering of the cues is nearly inconsequential, and that our approach is superior to other approaches such as Independent Integration and Hierarchical Integration in terms of flexibility and robustness.

Summary (2 min read)

1 Introduction

  • Various types of cues have been used to characterize different object properties, including color [1], texture [2], points [3], edges [4], motion [5] and contours [6].
  • As no single cue remains reliable in all situations, the integration of multiple cues has proved successful at increasing the robustness of tracking algorithms.
  • One important issue is how to model the dependence between different cues, which in turn determines the manner in which the cues are combined.
  • The resulting Linked Hidden Markov Models 1 naturally allow the use of two powerful inference algorithms, particle filtering and Belief Propagation (BP).
  • Experimental results are presented in Section 5.

3 Linked Hidden Markov Models

  • Suppose M visual cues are used and each cue is associated with a different target state.
  • In multi-cue tracking, the dependencies between target states at different cues must be taken into consideration.
  • The authors use graphical models to capture these dependencies.
  • The authors choose to adopt a chain model (Fig. 1(b)) to reduce the model complexity and to simplify the inference algorithm.
  • Linking the HMMs into chains results in so-called Linked Hidden Markov Models or lHMMs, shown in Fig. 1(c).

4.1 Problem Formulation

  • The authors first consider the inference in the chain model in Fig. 1(b).
  • Due to the lack of analytic representations of the above formulations, Monte Carlo approximations are required and can be obtained by using importance sampling techniques.
  • The authors address this by adapting Sequential Auxiliary Particle Belief Propagation [20].

4.2 Sequential Auxiliary Particle Belief Propagation

  • Assume there are K terms in the product and each term consists of N particles.
  • Sampling θk from this importance function is analogous to an auxiliary particle filter and is thus computationally efficient.
  • These sampled particles are used to approximate the messages and beliefs in Eqs. 6–8.
  • The authors algorithm treats each cue equally with no explicit preference.

4.3 Cues and Inter-Cue Potentials

  • For this paper, the authors chose four simple and complementary cues including color, edges, motion and contours.
  • The detected edges are then histogrammed into orientation bins weighted by their strengths [4].
  • This procedure was implemented using integral histograms [23].
  • Representing the same target in different cues differently allows a separate and efficient implementation of each tracking process.
  • When different target representations are used in the two neighboring cues, i.e., when the contour cue is neighboring to the color, edge or motion cues, a similar potential function is defined by ignoring θt in xcontourt .

5 Results

  • The authors tested the performance of their approach on sequences of various objects taken in both indoor and outdoor environments.
  • The edge, color (color histogram) and motion cues were integrated in the given order for both sequences.
  • For hierarchical integration, all shown cue orderings failed because misleading evidence was propagated downstream at some point.
  • Moreover, no single cue was able to track the target by itself, except for the contour cue, which was inaccurate and brittle.
  • It can be seen that both methods produced almost identical results, but their approach runs on average about 15 times faster than the full-product NBP.

6 Conclusions

  • This paper presents a systematic approach to integrating multiple cues in visual tracking.
  • The strength and beauty of the approach lies in its unprejudiced treatment of each individual cue, which permits efficient inference based on linked HMMs and a Sequential Auxiliary Particle Belief Propagation algorithm.
  • The simultaneous cues are arranged in a simple chain topology.
  • It doesn’t have to know what other cues are being used and how they are implemented.
  • The authors experiments confirmed a robustness superior to two competing approaches.

Did you find this useful? Give us your feedback

Figures (10)

Content maybe subject to copyright    Report

A Probabilistic Approach to Integrating
Multiple Cues in Visual Tracking
Wei Du and Justus Piater
University of Li`ege, Department of Electrical Engineering and Computer Science
Montefiore Institute, B28, B-4000 Liege, Belgium
{wei.du,justus.piater}@ulg.ac.be
Abstract. This paper presents a novel probabilistic approach to inte-
grating multiple cues in visual tracking. We perform tracking in different
cues by interacting processes. Each process is represented by a Hidden
Markov Model, and these parallel processes are arranged in a chain topol-
ogy. The resulting Linked Hidden Markov Models naturally allow the use
of particle filters and Belief Propagation in a unified framework. In par-
ticular, a target is tracked in each cue by a particle filter, and the particle
filters in different cues interact via a message passing scheme. The general
framework of our approach allows a customized combination of different
cues in different situations, which is desirable from the implementation
point of view. Our examples selectively integrate four visual cues in-
cluding color, edges, motion and contours. We demonstrate empirically
that the ordering of the cues is nearly inconsequential, and that our ap-
proach is superior to other approaches such as Independent Integration
and Hierarchical Integration in terms of flexibility and robustness.
1 Introduction
From a Bayesian perspective, visual tracking is viewed as a problem of inferring
target states over time based on image features or cues. Various types of cues
have been used to characterize different object properties, including color [1],
texture [2], points [3], edges [4], motion [5] and contours [6]. As no single cue
remains reliable in all situations, the integration of multiple cues has proved suc-
cessful at increasing the robustness of tracking algorithms. For instance, color
and appearance are more sensitive to the lighting conditions than gradient fea-
tures such as edges and points. Therefore, when the scene is subject to fast
illumination changes, edges and points may provide complementary information
that helps localize the targets being tracked.
Basically, multi-cue tracking is a fusion problem. Each cue tells a story about
the targets of interest, and this information is processed and fused to estimate
target states. One important issue is how to model the dependence between
different cues, which in turn determines the manner in which the cues are com-
bined. Many methods assume that the cues are conditionally independent [7–10],
while more sophisticated methods model existing dependencies explicitly by e.g.
graphical models [11].

2 Wei Du and Justus Piater
This paper presents a novel probabilistic approach to integrating multiple
cues in visual tracking. In contrast to previous work, we perform tracking in dif-
ferent cues by individual but interacting processes, each of which is represented
by a Hidden Markov Model (HMM). Chain models are used to link these parallel
HMMs and to represent the dependence between these processes. The resulting
Linked Hidden Markov Models (lHMMs)
1
naturally allow the use of two pow-
erful inference algorithms, particle filtering and Belief Propagation (BP). By
combining them in a unified framework, a Sequential Auxiliary Particle Belief
Propagation algorithm is devised to perform inference in multi-cue tracking. In
particular, a target is tracked in each cue by a particle filter, and the particle
filters in different cues interact with each other via a message passing scheme.
Our approach has several advantages. First, it allows different target rep-
resentations to be used in different cues so that each tracking process can be
implemented separately and efficiently. Second, the approach is highly modular,
facilitating the combination, addition and removal of vastly different cues. One
only needs to define the potential function between each pair of neighboring
cues according to their target representations. Third, the chain topology of the
lHMMs reduces the complexity of the integration framework with respect to
more elaborate graphical models. Due to the bidirectional propagation of infor-
mation along the chain, the order of the cues in the chain is largely unimportant.
We confirmed empirically that changing this order hardly affects the tracking
results.
The rest of the paper is organized as follows. Section 2 discusses related work
and highlights our contributions. Section 3 describes the lHMMs that model the
multi-cue tracking problem. Section 4 formulates the problem and introduces
the inference algorithm. Experimental results are presented in Section 5.
2 Related Work
Numerous approaches to multi-cue tracking have been reported in the literature.
They differ in the way the cues are integrated and in the cues adopted. For ex-
ample, Birchfield combined gradient and color cues for head tracking [7]. Triesch
et al. used democratic integration to compute the consensus between the multi-
ple cues [8]. Taylor et al. fused color, edge and texture cues in a Kalman-filter
framework [2].
Particle filters, also known as Condensation [6] in the computer vision com-
munity, have achieved great success in solving tracking problems. Conventional
particle filters maintain the target distributions over time based on a single ob-
servation model such as color [1] or contours [6]. It has been shown that multiple
cues can easily be fused within a particle-filter framework. By assuming that the
cues are conditionally independent, multi-cue observations were integrated by
the product [9, 13, 14] or the sum [15] of the likelihoods in different cues. Based
1
The term Linked Hidden Markov Models was introduced by Brand as a way of
modeling two interacting processes [12]. Here, we borrow this term and extend it to
multiple interacting processes.

A Probabilistic Approach to Integrating Multiple cues in Visual Tracking 3
on the same independence assumption, Leichter et al. combined Condensation-
based or explicit PDF-yielding trackers by fusing only the trackers’ output esti-
mates [10]. Although not explicitly stated, the approach required each tracker to
integrate information from all the other trackers, implying a fully-connected ar-
chitecture. On the other hand, P´erez et al. propagated target distributions from
cue to cue in a fixed order, hoping that downstream cues resolve the ambiguities
introduced by the upstream cues [5]. Similar ideas were introduced under the
names of hierarchical particle filters [4] and cascades of cues [16]. One downside
of this strategy is that the performance depends on the order in which the cues
are incorporated. Generally, heuristics are required to design this order. Wu et
al. used factorial HMMs to model the dependency between color and contour
cues and proposed a so-called co-inference algorithm [11].
Inspired by some of the cited work [10, 5, 11], this paper presents a general
framework for multi-cue integration. Similarly to Leichter et al. [10], we combine
particle filter-based trackers, each of which exploits a different cue. However,
the dependencies between these trackers are explicitly modeled using lHMMs.
The lHMMs link a set of parallel HMMs in a chain topology, which largely
reduces both the architectural and algorithmic complexities. In doing so, we
synchronously infer the target states in different cues. Unlike P´erez et al. [5] who
propagated information from cue to cue in one direction, the undirected links in
the chains in our lHMMs enable bidirectional message passing between the cues,
relaxing the dependency on the ordering of the cues. The suggested approach is
more general than e.g. that of Wu et al. [11] as it allows the integration of an
arbitrary number of cues.
The combination of particle filters and BP was originally motivated for in-
ference under non-linear and non-Gaussian models, resulting in Nonparametric
Belief Propagation (NBP) [17, 18]. Hua et al. first formulated the inference in
temporally evolving graphical models and proposed a Sequential Belief Prop-
agation (SBP) algorithm [19]. Briers et al. addressed the computational issue
in the same sequential-inference context [20]. Auxiliary particle filters and the
unscented approximation were used to sample particles, avoiding the need for
Gibbs sampling. The resulting Sequential Auxiliary Particle Belief Propagation
(SAPBP) reduced the computation from quadratic in the number of particles,
as in NBP, to linear, which is desirable for online inference.
In this paper, we adopt a simplified SAPBP algorithm to solve the multi-
cue tracking problem. The algorithm integrates the temporal evolution of each
cue and the inter-cue correlations into a coherent framework. While indepen-
dent temporal transition kernels are used for each cue, target states in different
cues are related through messages passed along a chain. Contrary to the original
SAPBP algorithm, we do not use the unscented approximation, as the inter-cue
pairwise potentials are linear. Four visual cues are selected to demonstrate the
effectiveness of our approach including color, edges, motion and contours. The
general framework facilitates a customized combination of different cues in dif-
ferent situations, which is particularly desirable from the implementation point
of view. Extensive experiments on tracking various objects in both indoor and

4 Wei Du and Justus Piater
(a) HMM (b) Chain (c) lHMMs
Fig. 1. (a) HMM representing the tracking process in one cue. (b) Chain model rep-
resenting the dependencies between different cues at a time instant. (c) Linked HMMs
representing the interacting processes in different cues.
outdoor environments show that our approach is more flexible and robust than
other approaches such as independent integration and hierarchical integration.
3 Linked Hidden Markov Models
Suppose M visual cues are used and each cue is associated with a different target
state. Let x
t,i
be the target state in the ith cue at time t and z
t,i
the associated
image observation, i = 1, . . . , M . Given these definitions, tracking in the ith cue
is formulated as
p(x
t,i
|z
t
i
) p(z
t,i
|x
t,i
)
Z
p(x
t,i
|x
t1,i
)p(x
t1,i
|z
t1
i
)dx
t1,i
, (1)
where z
t
i
= {z
1,i
, . . . , z
t,i
}, p(z
t,i
|x
t,i
) is the image likelihood and p(x
t,i
|x
t1,i
)
is the temporal transition kernel in the cue. This Bayesian formulation can be
represented by a HMM, shown in Fig. 1(a).
In multi-cue tracking, the dependencies between target states at different
cues must be taken into consideration. We use graphical models to capture these
dependencies. Theoretically, fully-connected graphical models are required since
all states in different cues may depend on each other. However, we choose to
adopt a chain model (Fig. 1(b)) to reduce the model complexity and to simplify
the inference algorithm. Linking the HMMs into chains results in so-called Linked
Hidden Markov Models or lHMMs, shown in Fig. 1(c). The lHMMs represent
the interacting processes in different cues.
Let X
t
= {x
t,1
, . . . , x
t,M
} denote the multi-cue target state and Z
t
= {z
t,1
, . . . , z
t,M
}
the multi-cue image observation. Thus, tracking in the multiple cues amounts to
the recursive inference of X
t
, formulated as
p(X
t
|Z
t
) p(Z
t
|X
t
)
Z
p(X
t
|X
t1
)p(X
t1
|Z
t1
)dX
t1
, (2)
where Z
t
= {Z
1
, . . . , Z
t
}.

A Probabilistic Approach to Integrating Multiple cues in Visual Tracking 5
Direct inference using Eq. 2 is intractable due to the high dimensionality of
the state space and the non-Gaussian nature of the target distributions. There-
fore, we infer each p(x
t,i
|Z
t
), i = 1, . . . , M , collaboratively by a set of particle-
filter-based processes, as detailed below.
4 Multi-Cue Tracking by Interacting Processes
4.1 Problem Formulation
We first consider the inference in the chain model in Fig. 1(b). BP performs
inference in graphical models by first computing messages and then computing
beliefs. The chain topology admits a two-pass message-passing implementation.
Specifically, the local messages passed from top to bottom are defined by
m
i,i+1
(x
t,i+1
)
Z
p(z
t,i
|x
t,i
)m
i1,i
(x
t,i
)ψ
i,i+1
(x
t,i
, x
t,i+1
)dx
t,i
, (3)
where ψ
i,j
is the potential function that describes the dependency between node
i and j. The definition of this potential function depends on the target repre-
sentations in the two neighboring cues and will be discussed later. Likewise, the
messages passed from bottom to top have a symmetric form,
m
i,i1
(x
t,i1
)
Z
p(z
t,i
|x
t,i
)m
i+1,i
(x
t,i
)ψ
i1,i
(x
t,i1
, x
t,i
)dx
t,i
. (4)
Then, the belief of x
t,i
is obtained by
p(x
t,i
|Z
t
) p(z
t,i
|x
t,i
)m
i1,i
(x
t,i
)m
i+1,i
(x
t,i
). (5)
Note that Eqs. 3–5 are slightly different for the nodes at the ends of the chain.
In the sequential context, the message and belief equations for the lHMMs
in Fig. 1(c) have similar forms as Eqs. 3, 4 and 5, adding only the terms of the
temporal priors,
m
i,i+1
(x
t,i+1
)
Z
p(z
t,i
|x
t,i
)p(x
t,i
|Z
t1
)m
i1,i
(x
t,i
)ψ
i,i+1
(x
t,i
, x
t,i+1
)dx
t,i
,(6)
m
i,i1
(x
t,i1
)
Z
p(z
t,i
|x
t,i
)p(x
t,i
|Z
t1
)m
i+1,i
(x
t,i
)ψ
i1,i
(x
t,i1
, x
t,i
)dx
t,i
,(7)
p(x
t,i
|Z
t
) p(z
t,i
|x
t,i
)p(x
t,i
|Z
t1
)m
i,i1
(x
t,i
)m
i,i+1
(x
t,i
), (8)
where the temporal prior is
p(x
t,i
|Z
t1
) =
Z
p(x
t,i
|x
t1,i
)p(x
t1,i
|Z
t1
)dx
t1,i
.
Thus, the sequential inference in the lHMMs consists of passing messages
using Eqs. 6 and 7 followed by belief updating using Eq. 8. Due to the lack of
analytic representations of the above formulations, Monte Carlo approximations
are required and can be obtained by using importance sampling techniques. A
technical issue is the design of proper importance functions. We address this by
adapting Sequential Auxiliary Particle Belief Propagation [20].

Citations
More filters
Proceedings ArticleDOI
13 Jun 2010
TL;DR: A novel tracking algorithm that can work robustly in a challenging scenario such that several kinds of appearance and motion changes of an object occur at the same time is proposed.
Abstract: We propose a novel tracking algorithm that can work robustly in a challenging scenario such that several kinds of appearance and motion changes of an object occur at the same time. Our algorithm is based on a visual tracking decomposition scheme for the efficient design of observation and motion models as well as trackers. In our scheme, the observation model is decomposed into multiple basic observation models that are constructed by sparse principal component analysis (SPCA) of a set of feature templates. Each basic observation model covers a specific appearance of the object. The motion model is also represented by the combination of multiple basic motion models, each of which covers a different type of motion. Then the multiple basic trackers are designed by associating the basic observation models and the basic motion models, so that each specific tracker takes charge of a certain change in the object. All basic trackers are then integrated into one compound tracker through an interactive Markov Chain Monte Carlo (IMCMC) framework in which the basic trackers communicate with one another interactively while run in parallel. By exchanging information with others, each tracker further improves its performance, which results in increasing the whole performance of tracking. Experimental results show that our method tracks the object accurately and reliably in realistic videos where the appearance and motion are drastically changing over time.

1,234 citations


Cites methods from "A Probabilistic Approach to Integra..."

  • ...The method in [5] integrates multiple cues, edge, and color in a probabilistic framework while the method in [18] fuses multiple observation models with parallel and cascaded evaluation....

    [...]

Proceedings ArticleDOI
01 Dec 2013
TL;DR: This paper cast tracking as a novel multi-task multi-view sparse learning problem and exploit the cues from multiple views including various types of visual features, such as intensity, color, and edge, where each feature observation can be sparsely represented by a linear combination of atoms from an adaptive feature dictionary.
Abstract: Combining multiple observation views has proven beneficial for tracking. In this paper, we cast tracking as a novel multi-task multi-view sparse learning problem and exploit the cues from multiple views including various types of visual features, such as intensity, color, and edge, where each feature observation can be sparsely represented by a linear combination of atoms from an adaptive feature dictionary. The proposed method is integrated in a particle filter framework where every view in each particle is regarded as an individual task. We jointly consider the underlying relationship between tasks across different views and different particles, and tackle it in a unified robust multi-task formulation. In addition, to capture the frequently emerging outlier tasks, we decompose the representation matrix to two collaborative components which enable a more robust and accurate approximation. We show that the proposed formulation can be efficiently solved using the Accelerated Proximal Gradient method with a small number of closed-form updates. The presented tracker is implemented using four types of features and is tested on numerous benchmark video sequences. Both the qualitative and quantitative results demonstrate the superior performance of the proposed approach compared to several state-of-the-art trackers.

164 citations


Cites background from "A Probabilistic Approach to Integra..."

  • ...Exploiting these multiple sources of information can significantly improve tracking performance as a result of their complementary characteristics [2][14][7][18]....

    [...]

Book ChapterDOI
06 Sep 2014
TL;DR: Experimental results show that the proposed principled way to combine occlusion and motion reasoning with a tracking-by-detection approach obtains state-of-the-art results and handles occlusions and viewpoints changes better than competing tracking methods.
Abstract: Object tracking is a reoccurring problem in computer vision. Tracking-by-detection approaches, in particular Struck [20], have shown to be competitive in recent evaluations. However, such approaches fail in the presence of long-term occlusions as well as severe viewpoint changes of the object. In this paper we propose a principled way to combine occlusion and motion reasoning with a tracking-by-detection approach. Occlusion and motion reasoning is based on state-of-the-art long-term trajectories which are labeled as object or background tracks with an energy-based formulation. The overlap between labeled tracks and detected regions allows to identify occlusions. The motion changes of the object between consecutive frames can be estimated robustly from the geometric relation between object trajectories. If this geometric change is significant, an additional detector is trained. Experimental results show that our tracker obtains state-of-the-art results and handles occlusion and viewpoints changes better than competing tracking methods.

91 citations


Cites methods from "A Probabilistic Approach to Integra..."

  • ...The inference methods range from Kalman filtering techniques, to those that use multiple cues [4,5,13,34,37,38,43,44] and fuse their results with methods like particle filtering [22], error analysis [44], and Markov chain Monte Carlo schemes [37]....

    [...]

Journal ArticleDOI
TL;DR: This paper presents a tracking approach using an approximate least absolute deviation (LAD)-based multitask multiview sparse learning method to enjoy robustness of LAD and take advantage of multiple types of visual features, such as intensity, color, and texture.
Abstract: Various sparse-representation-based methods have been proposed to solve tracking problems, and most of them employ least squares (LSs) criteria to learn the sparse representation. In many tracking scenarios, traditional LS-based methods may not perform well owing to the presence of heavy-tailed noise. In this paper, we present a tracking approach using an approximate least absolute deviation (LAD)-based multitask multiview sparse learning method to enjoy robustness of LAD and take advantage of multiple types of visual features, such as intensity, color, and texture. The proposed method is integrated in a particle filter framework, where learning the sparse representation for each view of the single particle is regarded as an individual task. The underlying relationship between tasks across different views and different particles is jointly exploited in a unified robust multitask formulation based on LAD. In addition, to capture the frequently emerging outlier tasks, we decompose the representation matrix to two collaborative components that enable a more robust and accurate approximation. We show that the proposed formulation can be effectively approximated by Nesterov’s smoothing method and efficiently solved using the accelerated proximal gradient method. The presented tracker is implemented using four types of features and is tested on numerous synthetic sequences and real-world video sequences, including the CVPR2013 tracking benchmark and ALOV++ data set. Both the qualitative and quantitative results demonstrate the superior performance of the proposed approach compared with several state-of-the-art trackers.

78 citations

Book ChapterDOI
07 Oct 2012
TL;DR: A robust visual tracking method to track an object in dynamic conditions that include motion blur, illumination changes, pose variations, and occlusions is proposed, which retains the robustness and adaptability of the TLF and multiple trackers.
Abstract: In this paper, a robust visual tracking method is proposed to track an object in dynamic conditions that include motion blur, illumination changes, pose variations, and occlusions. To cope with these challenges, multiple trackers with different feature descriptors are utilized, and each of which shows different level of robustness to certain changes in an object's appearance. To fuse these independent trackers, we propose two configurations, tracker selection and interaction. The tracker interaction is achieved based on a transition probability matrix (TPM) in a probabilistic manner. The tracker selection extracts one tracking result from among multiple tracker outputs by choosing the tracker that has the highest tracker probability. According to various changes in an object's appearance, the TPM and tracker probability are updated in a recursive Bayesian form by evaluating each tracker's reliability, which is measured by a robust tracker likelihood function (TLF). When the tracking in each frame is completed, the estimated object's state is obtained and fed into the reference update via the proposed learning strategy, which retains the robustness and adaptability of the TLF and multiple trackers. The experimental results demonstrate that our proposed method is robust in various benchmark scenarios.

75 citations


Cites methods from "A Probabilistic Approach to Integra..."

  • ...The methods that integrate trackers or features have been pr oposed using Condensation [1] or other Bayesian filters; they can be categorized into th ree kinds: a single tracker with multiple observations [6–8, 14], and multiple tracker s in parallel [9, 10, 15] or in cascade[11,12]....

    [...]

  • ...[11] proposed using Linked Hidden Markov Models whi ch enable the conjunction of particle filters with a belief propagation....

    [...]

References
More filters
Journal ArticleDOI
TL;DR: The Condensation algorithm uses “factored sampling”, previously applied to the interpretation of static images, in which the probability distribution of possible interpretations is represented by a randomly generated set.
Abstract: The problem of tracking curves in dense visual clutter is challenging. Kalman filtering is inadequate because it is based on Gaussian densities which, being unimo dal, cannot represent simultaneous alternative hypotheses. The Condensation algorithm uses “factored sampling”, previously applied to the interpretation of static images, in which the probability distribution of possible interpretations is represented by a randomly generated set. Condensation uses learned dynamical models, together with visual observations, to propagate the random set over time. The result is highly robust tracking of agile motion. Notwithstanding the use of stochastic methods, the algorithm runs in near real-time.

5,804 citations


"A Probabilistic Approach to Integra..." refers background or methods in this paper

  • ...Particle filters, also known as Condensation [6] in the computer vision community, have achieved great success in solving tracking problems....

    [...]

  • ...Various types of cues have been used to characterize different object properties, including color [1], texture [2], points [3], edges [4], motion [5] and contours [6]....

    [...]

  • ...Conventional particle filters maintain the target distributions over time based on a single observation model such as color [1] or contours [6]....

    [...]

  • ...Here, we borrow this term and extend it to multiple interacting processes. on the same independence assumption, Leichter et al. combined Condensationbased or explicit PDF-yielding trackers by fusing only the trackers’ output estimates [10]....

    [...]

  • ...In the contour cue, the Condensation algorithm [6] was implemented using a distance transformation....

    [...]

Book ChapterDOI
28 May 2002
TL;DR: This work introduces a new Monte Carlo tracking technique based on the same principle of color histogram distance, but within a probabilistic framework, and introduces the following ingredients: multi-part color modeling to capture a rough spatial layout ignored by global histograms, incorporation of a background color model when relevant, and extension to multiple objects.
Abstract: Color-based trackers recently proposed in [3,4,5] have been proved robust and versatile for a modest computational cost They are especially appealing for tracking tasks where the spatial structure of the tracked objects exhibits such a dramatic variability that trackers based on a space-dependent appearance reference would break down very fast Trackers in [3,4,5] rely on the deterministic search of a window whose color content matches a reference histogram color modelRelying on the same principle of color histogram distance, but within a probabilistic framework, we introduce a new Monte Carlo tracking technique The use of a particle filter allows us to better handle color clutter in the background, as well as complete occlusion of the tracked entities over a few framesThis probabilistic approach is very flexible and can be extended in a number of useful ways In particular, we introduce the following ingredients: multi-part color modeling to capture a rough spatial layout ignored by global histograms, incorporation of a background color model when relevant, and extension to multiple objects

1,549 citations


"A Probabilistic Approach to Integra..." refers background in this paper

  • ...Various types of cues have been used to characterize different object properties, including color [1], texture [2], points [3], edges [4], motion [5] and contours [6]....

    [...]

  • ...Conventional particle filters maintain the target distributions over time based on a single observation model such as color [1] or contours [6]....

    [...]

Book
01 Jan 2008

1,473 citations

Journal ArticleDOI
TL;DR: This paper formulate the stereo matching problem as a Markov network and solve it using Bayesian belief propagation to obtain the maximum a posteriori (MAP) estimation in the Markovnetwork.
Abstract: In this paper, we formulate the stereo matching problem as a Markov network and solve it using Bayesian belief propagation. The stereo Markov network consists of three coupled Markov random fields that model the following: a smooth field for depth/disparity, a line process for depth discontinuity, and a binary process for occlusion. After eliminating the line process and the binary process by introducing two robust functions, we apply the belief propagation algorithm to obtain the maximum a posteriori (MAP) estimation in the Markov network. Other low-level visual cues (e.g., image segmentation) can also be easily incorporated in our stereo model to obtain better stereo results. Experiments demonstrate that our methods are comparable to the state-of-the-art stereo algorithms for many test cases.

1,272 citations

Book ChapterDOI
28 May 2002
TL;DR: This paper forms the stereo matching problem as a Markov network consisting of three coupled Markov random fields, and obtains the maximum a posteriori (MAP) estimation in the Markovnetwork by applying a Bayesian belief propagation (BP) algorithm.
Abstract: In this paper, we formulate the stereo matching problem as a Markov network consisting of three coupled Markov random fields (MRF's). These three MRF's model a smooth field for depth/disparity, a line process for depth discontinuity and a binary process for occlusion, respectively. After eliminating the line process and the binary process by introducing two robust functions, we obtain the maximum a posteriori (MAP) estimation in the Markov network by applying a Bayesian belief propagation (BP) algorithm. Furthermore, we extend our basic stereo model to incorporate other visual cues (e.g., image segmentation) that are not modeled in the three MRF's, and again obtain the MAP solution. Experimental results demonstrate that our method outperforms the state-of-art stereo algorithms for most test cases.

1,145 citations


"A Probabilistic Approach to Integra..." refers background in this paper

  • ...[21], the asymmetric message passing in BP guarantees that the information is propagated mainly from high-confidence cues to lowconfidence cues due to the smaller entropy of the messages in this direction....

    [...]

Frequently Asked Questions (1)
Q1. What are the contributions in "A probabilistic approach to integrating multiple cues in visual tracking" ?

This paper presents a novel probabilistic approach to integrating multiple cues in visual tracking. The authors perform tracking in different cues by interacting processes. The resulting Linked Hidden Markov Models naturally allow the use of particle filters and Belief Propagation in a unified framework. In particular, a target is tracked in each cue by a particle filter, and the particle filters in different cues interact via a message passing scheme. The general framework of their approach allows a customized combination of different cues in different situations, which is desirable from the implementation point of view. The authors demonstrate empirically that the ordering of the cues is nearly inconsequential, and that their approach is superior to other approaches such as Independent Integration and Hierarchical Integration in terms of flexibility and robustness.