The problem of tracking curves in dense visual clutter is challenging. Kalman filtering is inadequate because it is based on Gaussian densities which, being unimo dal, cannot represent simultaneous alternative hypotheses. The Condensation algorithm uses “factored sampling”, previously applied to the interpretation of static images, in which the probability distribution of possible interpretations is represented by a randomly generated set. Condensation uses learned dynamical models, together with visual observations, to propagate the random set over time. The result is highly robust tracking of agile motion. Notwithstanding the use of stochastic methods, the algorithm runs in near real-time.

/pdf/c-ondensation-conditional-density-propagation-forvisual-2zlkn4lnac.pdf

C ONDENSATION —Conditional Density Propagation forVisual Tracking

There are two major types of uncertainty one can model. Aleatoric uncertainty captures noise inherent in the observations. On the other hand, epistemic uncertainty accounts for uncertainty in the model -- uncertainty which can be explained away given enough data. Traditionally it has been difficult to model epistemic uncertainty in computer vision, but with new Bayesian deep learning tools this is now possible. We study the benefits of modeling epistemic vs. aleatoric uncertainty in Bayesian deep learning models for vision tasks. For this we present a Bayesian deep learning framework combining input-dependent aleatoric uncertainty together with epistemic uncertainty. We study models under the framework with per-pixel semantic segmentation and depth regression tasks. Further, our explicit uncertainty formulation leads to new loss functions for these tasks, which can be interpreted as learned attenuation. This makes the loss more robust to noisy data, also giving new state-of-the-art results on segmentation and depth regression benchmarks.

What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision

The Visual Analysis of Human Movement

A scheme is developed for classifying the types of motion perceived by a humanlike robot. It is assumed that the robot receives visual images of the scene using a perspective system model. Equations, theorems, concepts, clues, etc., relating the objects, their positions, and their motion to their images on the focal plane are presented. >

http://ieeexplore.ieee.org/iel2/815/3148/00100651.pdf

Robot vision

The problem of tracking curves in dense visual clutter is a challenging one. Trackers based on Kalman filters are of limited use; because they are based on Gaussian densities which are unimodal, they cannot represent simultaneous alternative hypotheses. Extensions to the Kalman filter to handle multiple data associations work satisfactorily in the simple case of point targets, but do not extend naturally to continuous curves. A new, stochastic algorithm is proposed here, the Condensation algorithm — Conditional Density Propagation over time. It uses ‘factored sampling’, a method previously applied to interpretation of static images, in which the distribution of possible interpretations is represented by a randomly generated set of representatives. The Condensation algorithm combines factored sampling with learned dynamical models to propagate an entire probability distribution for object position and shape, over time. The result is highly robust tracking of agile motion in clutter, markedly superior to what has previously been attainable from Kalman filtering. Notwithstanding the use of stochastic methods, the algorithm runs in near real-time.

https://link.springer.com/content/pdf/10.1007%2FBFb0015549.pdf

Contour Tracking by Stochastic Propagation of Conditional Density

There has been a great deal of research interest in contour tracking over the last five years. This article combines themes from tracking theory—elastic models and stochastic filtering—with the notion of affine invariance to synthesize a substantially new and demonstrably effective framework for contour tracking. A mechanism is developed for incorporating a shape template into a contour tracker via an affine invariant coupling. In that way the tracker becomes selective for shape and therefore able to ignore background clutter. Affine invariance ensures that the effect of varying viewpoint is accommodated. Use of a standard statistical filtering framework allows uncertainties to be treated systematically, which accommodates object flexibility and un-modeled distortions such as the deformation of a silhouette under motion. The statistical framework also facilitates a further development. In place of heuristically determined spatial scale for feature search, both spatial scale and temporal memory are controlled automatically and in a way that is responsive to the tracking process. Typically, the tracker operates initially in a coarse scale/short memory mode while it searches for a feature. Then spatial scale diminishes to allow more precise localization while memory (temporal scale) lengths to take advantage of motion coherence. All system parameters are determined by natural assumptions and desired tracking performance, leaving none to be fixed heuristically. Versions of the tracker have been implemented at video rate, both on SUN 4 and in parallel, using a network of 11 transputers. The theoretically established properties of automatic control of spatiotemporal scale and of affine invariance are demonstrated using the implemented tracker.

A framework for spatiotemporal control in the tracking of visual contours

Dynamic contours: real-time active splines

A framework for visual tracking of contours is presented, based on a synthesis of elastic models, stochastic filtering, and geometric invariance. Flexibly coupled curve templates implement soft prior assumptions about shape. Affine invariance, built into the flexible coupling, ensures that the affine deformations that arise naturally from image projection are favored. The stochastic basis of the framework is shown to result naturally in an automatic mechanism for the control of spatiotemporal scale. >

Affine-invariant contour tracking with automatic control of spatiotemporal scale

Snakes as originally proposed by Kass, Witkin and Terzopoulous are active contour models which minimise an expression of energy to locate image features. The original formulation involves hundreds of state variables and does not submit to a real-time implementation. We explore the use of a B-spline model of the feature to reduce the state space of the problem. Using a coarse to fine feature search and Lagrangian Dynamics we demonstrate a real-time, parallel implementation of B-spline snakes on a network of transputers.

/pdf/parallel-implementation-of-lagrangian-dynamics-for-real-time-nw5q5eoya8.pdf

Parallel Implementation of Lagrangian Dynamics for real-time snakes

In this paper we report progress towards a flexible, visually driven, object manipulation system. The aim is that a robot arm with a camera and gripper mounted on its tip should be able to transport objects across an obstacle-strewn environment. Our system is based on the analysis of moving image contours, which can provide direct estimates of the shape of curved surfaces. Recently we have elaborated on this basis in two respects. First we have developed real-time visual tracking methods using “dynamic contours” with Lagrangian Dynamics allowing direct generation of approximations to geodesic paths around obstacles. Secondly we have built a 2 1/2D system for incremental, active exploration of free-space.

/pdf/real-time-visual-tracking-for-surveillance-and-path-planning-3giixzrzxv.pdf

Rupert W. Curwen

Papers

A framework for spatiotemporal control in the tracking of visual contours

Dynamic contours: real-time active splines

Affine-invariant contour tracking with automatic control of spatiotemporal scale

Parallel Implementation of Lagrangian Dynamics for real-time snakes

Real-time Visual Tracking for Surveillance and Path Planning