Segmentation Based Multi-cue Integration for Object Detection

TLDR: Experimental results show that the proposed multi-cue combination scheme significantly increases detection performance compared to any of its constituent cues alone, and provides an interesting evaluation tool to analyze the complementarity of local feature detectors and descriptors.

Abstract: This paper proposes a novel method for integrating multiple local cues, i.e. local region detectors as well as descriptors, in the context of object detection. Rather than to fuse the outputs of several distinct classifiers in a fixed setup, our approach implements a highly flexible combination scheme, where the contributions of all individual cues are flexibly recombined depending on their explanatory power for each new test image. The key idea behind our approach is to integrate the cues over an estimated top-down segmentation, which allows to quantify how much each of them contributed to the object hypothesis. By combining those contributions on a per-pixel level, our approach ensures that each cue only contributes to object regions for which it is confident and that potential correlations between cues are effectively factored out. Experimental results on several benchmark data sets show that the proposed multi-cue combination scheme significantly increases detection performance compared to any of its constituent cues alone. Moreover, it provides an interesting evaluation tool to analyze the complementarity of local feature detectors and descriptors.

Figures

Figure 3: Single-cue EER performances for all detector/descriptor combinations on the TUD motorbikes. The plots show the performance gradation when the clustering/matching threshold is varied. In all following experiments, we use only the bestperforming parameter setting for each cue.

Figure 4: Cue combination performances on the TUD motorbikes: (a) single-cue performance; (b) performance of the different combination strategies using all 5 descriptors with the same detector; (c) cue combination performance when t same descriptors are applied to different detectors.

Figure 5: Performance comparison on the TUD motorbikes (left), the more difficult VOC motorbiketest2 set (middle), and the pedestrian test set (right). The middle plot is rotated90◦ to make it consistent with the ones in [5]. Please note that while our detector is exclusively trained on side views, only 39% of the motorbikes in the VOC set are shown in side views, thus limiting the maximally achievable recall.

Figure 1: Visualization of the multi-cue integration stages: (a) initial detection, (b) topdown segmentation, (c)p(figure) maps obtained byaveragecombination, (d) closeup view of theargmaxvisualization (cf. eq.(9)) , (e) histogram of relative cue contributions.

Figure 6: Example multi-cue detections of our approach on difficult images from the VOC motorbikes and the pedestrian set (at the EER).

Figure 2: Some detections and the corresponding relative cue contributions.