A new approach toward target representation and localization, the central component in visual tracking of nonrigid objects, is proposed. The feature histogram-based target representations are regularized by spatial masking with an isotropic kernel. The masking induces spatially-smooth similarity functions suitable for gradient-based optimization, hence, the target localization problem can be formulated using the basin of attraction of the local maxima. We employ a metric derived from the Bhattacharyya coefficient as similarity measure, and use the mean shift procedure to perform the optimization. In the presented tracking examples, the new method successfully coped with camera motion, partial occlusions, clutter, and target scale variations. Integration with motion filters and data association techniques is also discussed. We describe only a few of the potential applications: exploitation of background information, Kalman tracking using motion models, and face tracking.

/pdf/kernel-based-object-tracking-2pj4gwhvr7.pdf

Kernel-based object tracking

While different optical flow techniques continue to appear, there has been a lack of quantitative evaluation of existing methods. For a common set of real and synthetic image sequences, we report the results of a number of regularly cited optical flow techniques, including instances of differential, matching, energy-based, and phase-based methods. Our comparisons are primarily empirical, and concentrate on the accuracy, reliability, and density of the velocity measurements; they show that performance can differ significantly among the techniques we implemented.

Performance of optical flow techniques

Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

We study an energy functional for computing optical flow that combines three assumptions: a brightness constancy assumption, a gradient constancy assumption, and a discontinuity-preserving spatio-temporal smoothness constraint. In order to allow for large displacements, linearisations in the two data terms are strictly avoided. We present a consistent numerical scheme based on two nested fixed point iterations. By proving that this scheme implements a coarse-to-fine warping strategy, we give a theoretical foundation for warping which has been used on a mainly experimental basis so far. Our evaluation demonstrates that the novel method gives significantly smaller angular errors than previous techniques for optical flow estimation. We show that it is fairly insensitive to parameter variations, and we demonstrate its excellent robustness under noise.

/pdf/high-accuracy-optical-flow-estimation-based-on-a-theory-for-2o1buld5hm.pdf

High Accuracy Optical Flow Estimation Based on a Theory for Warping

http://cvssp-data.eps.surrey.ac.uk/Personal/AdrianHilton/Publications_files/survey_techrep.pdf

A survey of advances in vision-based human motion capture and analysis

Smoothness constraints have been used to facilitate the estimation of displacement vector fields. Differing from HORN and SCHUNCK 81 who employ a general smoothness requirement, this contribution reports an analysis of an "oriented smoothness" requirement: a change in the displacement vector field is only constrained in the direction perpendicular to the characteristic gray value variation based on which the displacement vector is estimated. An iterative solution for the resulting system of nonlinear partial differential equations is developed. It is shown how this system of equations relates to the one derived by HORN and SCHUNCK 81.

/pdf/constraints-for-the-estimation-of-displacement-vector-fields-39shx8kfm4.pdf

Constraints for the estimation of displacement vector fields from image sequences

Moving vehicles are detected and tracked automatically in monocular image sequences from road traffic scenes recorded by a stationary camera. In order to exploit the a priori knowledge about shape and motion of vehicles in traffic scenes, a parameterized vehicle model is used for an intraframe matching process and a recursive estimator based on a motion model is used for motion estimation. An interpretation cycle supports the intraframe matching process with a state MAP-update step. Initial model hypotheses are generated using an image segmentation component which clusters coherently moving image features into candidate representations of images of a moving vehicle. The inclusion of an illumination model allows taking shadow edges of the vehicle into account during the matching process. Only such an elaborate combination of various techniques has enabled us to track vehicles under complex illumination conditions and over long (over 400 frames) monocular image sequences. Results on various real-world road traffic scenes are presented and open problems as well as future work are outlined.

Model-based object tracking in monocular image sequences of road traffic scenes

A mapping between one frame from an image sequence and the preceding or following frame can be represented as a displacement vector field. In most situations, the mere gray value variations do not provide sufficient information in order to estimate such a displacement vector field. Supplementary constraints are necessary, for example the postulate that a displacement vector field varies smoothly as a function of the image position. Taken as a general requirement, this creates difficulties at gray value transitions which correspond to occluding contours. Nagel therefore introduced the ``oriented smoothness'' requirement which restricts variations of the displacement vector field only in directions with small or no variation of gray values. This contribution reports results of an investigation about how such an ``oriented smoothness'' constraint may be formulated and evaluated.

An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences

Tracking Persons in Monocular Image Sequences

This contribution addresses the problem of pose estimation and tracking of vehicles in image sequences from traffic scenes recorded by a stationary camera. In a new algorithm, the vehicle pose is estimated by directly matching polyhedral vehicle models to image gradients without an edge segment extraction process. The new approach is significantly more robust than approaches that rely on feature extraction since the new approach exploits more information from the image data. We successfully tracked vehicles that were partially occluded by textured objects, e.g., foliage, where a previous approach based on edge segment extraction failed. Moreover, the new pose estimation approach is also used to determine the orientation and position of the road relative to the camera by matching an intersection model directly to image gradients. Results from various experiments with real world traffic scenes are presented.

Hans-Hellmut Nagel

Papers

Constraints for the estimation of displacement vector fields from image sequences

Model-based object tracking in monocular image sequences of road traffic scenes

An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences

Tracking Persons in Monocular Image Sequences

3D Pose Estimation by Directly Matching Polyhedral Models to Gray Value Gradients