# A Probabilistic Approach to Integrating Multiple Cues in Visual Tracking

## Summary (2 min read)

### 1 Introduction

- Various types of cues have been used to characterize different object properties, including color [1], texture [2], points [3], edges [4], motion [5] and contours [6].
- As no single cue remains reliable in all situations, the integration of multiple cues has proved successful at increasing the robustness of tracking algorithms.
- One important issue is how to model the dependence between different cues, which in turn determines the manner in which the cues are combined.
- The resulting Linked Hidden Markov Models 1 naturally allow the use of two powerful inference algorithms, particle filtering and Belief Propagation (BP).
- Experimental results are presented in Section 5.

### 4.1 Problem Formulation

- The authors first consider the inference in the chain model in Fig. 1(b).
- Due to the lack of analytic representations of the above formulations, Monte Carlo approximations are required and can be obtained by using importance sampling techniques.
- The authors address this by adapting Sequential Auxiliary Particle Belief Propagation [20].

### 4.2 Sequential Auxiliary Particle Belief Propagation

- Assume there are K terms in the product and each term consists of N particles.
- Sampling θk from this importance function is analogous to an auxiliary particle filter and is thus computationally efficient.
- These sampled particles are used to approximate the messages and beliefs in Eqs. 6–8.
- The authors algorithm treats each cue equally with no explicit preference.

### 4.3 Cues and Inter-Cue Potentials

- For this paper, the authors chose four simple and complementary cues including color, edges, motion and contours.
- The detected edges are then histogrammed into orientation bins weighted by their strengths [4].
- This procedure was implemented using integral histograms [23].
- Representing the same target in different cues differently allows a separate and efficient implementation of each tracking process.
- When different target representations are used in the two neighboring cues, i.e., when the contour cue is neighboring to the color, edge or motion cues, a similar potential function is defined by ignoring θt in xcontourt .

### 5 Results

- The authors tested the performance of their approach on sequences of various objects taken in both indoor and outdoor environments.
- The edge, color (color histogram) and motion cues were integrated in the given order for both sequences.
- For hierarchical integration, all shown cue orderings failed because misleading evidence was propagated downstream at some point.
- Moreover, no single cue was able to track the target by itself, except for the contour cue, which was inaccurate and brittle.
- It can be seen that both methods produced almost identical results, but their approach runs on average about 15 times faster than the full-product NBP.

### 6 Conclusions

- This paper presents a systematic approach to integrating multiple cues in visual tracking.
- The strength and beauty of the approach lies in its unprejudiced treatment of each individual cue, which permits efficient inference based on linked HMMs and a Sequential Auxiliary Particle Belief Propagation algorithm.
- The simultaneous cues are arranged in a simple chain topology.
- It doesn’t have to know what other cues are being used and how they are implemented.
- The authors experiments confirmed a robustness superior to two competing approaches.

Did you find this useful? Give us your feedback

##### Citations

1,234 citations

### Cites methods from "A Probabilistic Approach to Integra..."

...The method in [5] integrates multiple cues, edge, and color in a probabilistic framework while the method in [18] fuses multiple observation models with parallel and cascaded evaluation....

[...]

164 citations

### Cites background from "A Probabilistic Approach to Integra..."

...Exploiting these multiple sources of information can significantly improve tracking performance as a result of their complementary characteristics [2][14][7][18]....

[...]

91 citations

### Cites methods from "A Probabilistic Approach to Integra..."

...The inference methods range from Kalman filtering techniques, to those that use multiple cues [4,5,13,34,37,38,43,44] and fuse their results with methods like particle filtering [22], error analysis [44], and Markov chain Monte Carlo schemes [37]....

[...]

78 citations

75 citations

### Cites methods from "A Probabilistic Approach to Integra..."

...The methods that integrate trackers or features have been pr oposed using Condensation [1] or other Bayesian filters; they can be categorized into th ree kinds: a single tracker with multiple observations [6–8, 14], and multiple tracker s in parallel [9, 10, 15] or in cascade[11,12]....

[...]

...[11] proposed using Linked Hidden Markov Models whi ch enable the conjunction of particle filters with a belief propagation....

[...]

##### References

5,804 citations

### "A Probabilistic Approach to Integra..." refers background or methods in this paper

...Particle filters, also known as Condensation [6] in the computer vision community, have achieved great success in solving tracking problems....

[...]

...Various types of cues have been used to characterize different object properties, including color [1], texture [2], points [3], edges [4], motion [5] and contours [6]....

[...]

...Conventional particle filters maintain the target distributions over time based on a single observation model such as color [1] or contours [6]....

[...]

...Here, we borrow this term and extend it to multiple interacting processes. on the same independence assumption, Leichter et al. combined Condensationbased or explicit PDF-yielding trackers by fusing only the trackers’ output estimates [10]....

[...]

...In the contour cue, the Condensation algorithm [6] was implemented using a distance transformation....

[...]

^{1}

1,549 citations

### "A Probabilistic Approach to Integra..." refers background in this paper

...Various types of cues have been used to characterize different object properties, including color [1], texture [2], points [3], edges [4], motion [5] and contours [6]....

[...]

...Conventional particle filters maintain the target distributions over time based on a single observation model such as color [1] or contours [6]....

[...]

1,272 citations

1,145 citations

### "A Probabilistic Approach to Integra..." refers background in this paper

...[21], the asymmetric message passing in BP guarantees that the information is propagated mainly from high-confidence cues to lowconfidence cues due to the smaller entropy of the messages in this direction....

[...]