http://bigwww.epfl.ch/publications/fortun1501.pdf

Optical flow modeling and computation

This paper presents a new Bayesian framework for motion segmentation /sub i/viding a frame from an image sequence into layers representing different moving objects - by tracking edges between frames. Edges are found using the Canny edge detector, and the expectation-maximization algorithm is then used to fit motion models to these edges and also to calculate the probabilities of the edges obeying each motion model. The edges are also used to segment the image into regions of similar color. The most likely labeling for these regions is then calculated by using the edge probabilities, in association with a Markov random field-style prior. The identification of the relative depth ordering of the different motion layers is also determined, as an integral part of the process. An efficient implementation of this framework is presented for segmenting two motions (foreground and background) using two frames. It is then demonstrated how, by tracking the edges into further frames, the probabilities may be accumulated to provide an even more accurate and robust estimate, and segment an entire sequence. Further extensions are then presented to address the segmentation of more than two motions. Here, a hierarchical method of initializing the expectation-maximization algorithm is described, and it is demonstrated that the minimum description length principle may be used to automatically select the best number of motion layers. The results from over 30 sequences (demonstrating both two and three motions) are presented and discussed.

/pdf/layered-motion-segmentation-and-depth-ordering-by-tracking-j63h6gs71s.pdf

Layered motion segmentation and depth ordering by tracking edges

This paper proposes a solution for the automatic detection and tracking of human motion in image sequences. Due to the complexity of the human body and its motion, automatic detection of 3D human motion remains an open, and important, problem. Existing approaches for automatic detection and tracking focus on 2D cues and typically exploit object appearance (color distribution, shape) or knowledge of a static background. In contrast, we exploit 2D optical flow information which provides rich descriptive cues, while being independent of object and background appearance. To represent the optical flow patterns of people from arbitrary viewpoints, we develop a novel representation of human motion using low-dimensional spatio-temporal models that are learned using motion capture data of human subjects. In addition to human motion (the foreground) we probabilistically model the motion of generic scenes (the background); these statistical models are defined as Gibbsian fields specified from the first-order derivatives of motion observations. Detection and tracking are posed in a principled Bayesian framework which involves the computation of a posterior probability distribution over the model parameters (i.e., the location and the type of the human motion) given a sequence of optical flow observations. Particle filtering is used to represent and predict this non-Gaussian posterior distribution over time. The model parameters of samples from this distribution are related to the pose parameters of a 3D articulated model (e.g. the approximate joint angles and movement direction). Thus the approach proves suitable for initializing more complex probabilistic models of human motion. As shown by experiments on real image sequences, our method is able to detect and track people under different viewpoints with complex backgrounds.

/pdf/automatic-detection-and-tracking-of-human-motion-with-a-view-2mlop3yfxn.pdf

Automatic Detection and Tracking of Human Motion with a View-Based Representation

Statistical background modeling for non-stationary camera

A new approach for motion characterization in image sequences is presented. It relies on the probabilistic modeling of temporal and scale co-occurrence distributions of local motion-related measurements directly computed over image sequences. Temporal multiscale Gibbs models allow us to handle both spatial and temporal aspects of image motion content within a unified statistical framework. Since this modeling mainly involves the scalar product between co-occurrence values and Gibbs potentials, we can formulate and address several fundamental issues: model estimation according to the ML criterion (hence, model training and learning) and motion classification. We have conducted motion recognition experiments over a large set of real image sequences comprising various motion types such as temporal texture samples, human motion examples, and rigid motion situations.

/pdf/motion-recognition-using-nonparametric-image-motion-models-6w2aksfl3k.pdf

Motion recognition using nonparametric image motion models estimated from temporal and multiscale co-occurrence statistics

We present a statistical method to detect regions whose apparent motion in the image is not conforming to the dominant motion of the background resulting from the camera movement. Alternatively, the same scheme can be used to track a particular region of interest of the scene. The apparent motion induced by the camera motion is represented by a 2D parametric motion model, and compensated for using the values of the motion model parameters estimated by a multiresolution robust statistical technique. Then, regions whose motion cannot be described by this global model estimated over the entire image, are extracted. The detection of these non conforming regions is achieved through a statistical regularization approach based on multiscale Markov random field (MRF) models. We have paid a particular attention to the definition of the energy function involved and to the observations taken into account. To gain robustness, information is integrated over time. This method has been validated by experiments carried out on many real image sequences.

P. Bouthemy

Papers

Separation of Moving Regions from Background in an Image Sequence Acquired with a Mobil Camera