Visual surveillance in dynamic scenes, especially for humans and vehicles, is currently one of the most active research topics in computer vision. It has a wide spectrum of promising applications, including access control in special areas, human identification at a distance, crowd flux statistics and congestion analysis, detection of anomalous behaviors, and interactive surveillance using multiple cameras, etc. In general, the processing framework of visual surveillance in dynamic scenes includes the following stages: modeling of environments, detection of motion, classification of moving objects, tracking, understanding and description of behaviors, human identification, and fusion of data from multiple cameras. We review recent developments and general strategies of all these stages. Finally, we analyze possible research directions, e.g., occlusion handling, a combination of twoand three-dimensional tracking, a combination of motion analysis and biometrics, anomaly detection and behavior prediction, content-based retrieval of surveillance videos, behavior understanding and natural language description, fusion of information from multiple sensors, and remote surveillance.

/pdf/a-survey-on-visual-surveillance-of-object-motion-and-302zsaqo09.pdf

A survey on visual surveillance of object motion and behaviors

The Visual Analysis of Human Movement

As an object moves through the field of view of a camera, the images of the object may change dramatically. This is not simply due to the translation of the object across the image plane; complications arise due to the fact that the object undergoes changes in pose relative to the viewing camera, in illumination relative to light sources, and may even become partially or fully occluded. We develop an efficient general framework for object tracking, which addresses each of these complications. We first develop a computationally efficient method for handling the geometric distortions produced by changes in pose. We then combine geometry and illumination into an algorithm that tracks large image regions using no more computation than would be required to track with no accommodation for illumination changes. Finally, we augment these methods with techniques from robust statistics and treat occluded regions on the object as statistical outliers. Experimental results are given to demonstrate the effectiveness of our methods.

/pdf/efficient-region-tracking-with-parametric-models-of-geometry-1knpo80koe.pdf

Efficient region tracking with parametric models of geometry and illumination

This paper presents a new variational framework for detecting and tracking multiple moving objects in image sequences. Motion detection is performed using a statistical framework for which the observed interframe difference density function is approximated using a mixture model. This model is composed of two components, namely, the static (background) and the mobile (moving objects) one. Both components are zero-mean and obey Laplacian or Gaussian law. This statistical framework is used to provide the motion detection boundaries. Additionally, the original frame is used to provide the moving object boundaries. Then, the detection and the tracking problem are addressed in a common framework that employs a geodesic active contour objective function. This function is minimized using a gradient descent method. A new approach named Hermes is proposed, which exploits aspects from the well-known front propagation algorithms and compares favorably to them. Very promising experimental results are provided using real video sequences.

Geodesic active contours and level sets for the detection and tracking of moving objects

Many applications require tracking of complex 3D objects. These include visual servoing of robotic arms on specific target objects, Augmented Reality systems that require real-time registration of the object to be augmented, and head tracking systems that sophisticated interfaces can use. Computer Vision offers solutions that are cheap, practical and non-invasive.This survey reviews the different techniques and approaches that have been developed by industry and research. First, important mathematical tools are introduced: Camera representation, robust estimation and uncertainty estimation. Then a comprehensive study is given of the numerous approaches developed by the Augmented Reality and Robotics communities, beginning with those that are based on point or planar fiducial marks and moving on to those that avoid the need to engineer the environment by relying on natural features such as edges, texture or interest. Recent advances that avoid manual initialization and failures due to fast motion are also presented. The survery concludes with the different possible choices that should be made when implementing a 3D tracking system and a discussion of the future of vision-based 3D tracking.Because it encompasses many computer vision techniques from low-level vision to 3D geometry and includes a comprehensive study of the massive literature on the subject, this survey should be the handbook of the student, the researcher, or the engineer who wants to implement a 3D tracking system.

/pdf/monocular-model-based-3d-tracking-of-rigid-objects-a-survey-1tb409lav4.pdf

Monocular Model-Based 3D Tracking of Rigid Objects: A Survey

This contribution addresses the problem of pose estimation and tracking of vehicles in image sequences from traffic scenes recorded by a stationary camera. In a new algorithm, the vehicle pose is estimated by directly matching polyhedral vehicle models to image gradients without an edge segment extraction process. The new approach is significantly more robust than approaches that rely on feature extraction since the new approach exploits more information from the image data. We successfully tracked vehicles that were partially occluded by textured objects, e.g., foliage, where a previous approach based on edge segment extraction failed. Moreover, the new pose estimation approach is also used to determine the orientation and position of the road relative to the camera by matching an intersection model directly to image gradients. Results from various experiments with real world traffic scenes are presented.

3D Pose Estimation by Directly Matching Polyhedral Models to Gray Value Gradients

This contribution addresses the problem of detection and tracking of moving vehicles in image sequences from traffic scenes recorded by a stationary camera. By replacing the low level vision system component for the estimation of displacement vectors by an optical flow estimation module we are able to detect all moving vehicles in our test image sequence. By replacing the edge detector and by doubling the sampling rate we improve the model-based object tracking system significantly compared to an earlier system. The trajectories of vehicles are characterized by motion verbs and verb phrases. Results from various experiments with real world traffic scenes are presented.

/pdf/association-of-motion-verbs-with-vehicle-movements-extracted-56q30mjyxv.pdf

Association of motion verbs with vehicle movements extracted from dense optical flow fields

Vehicles on downtown roads can be occluded by other vehicles or by stationary scene components such as traffic lights or road signs. After having recorded such a scene by a video camera, we noticed that the occlusion may disturb the detection and tracking of vehicles by previous versions of our computer vision approach. In this contribution we demonstrate how our image sequence analysis system can be improved by an explicit model-based recognition of 3D occlusion situations. Results obtained from real world image sequences recording gas station traffic as well as inner-city intersection traffic are presented.

/pdf/tracking-of-occluded-vehicles-in-traffic-scenes-28z9vpx59i.pdf

Tracking of Occluded Vehicles in Traffic Scenes

While estimating both components of optical flow based on the postulated validity of the Optical Flow Constraint Equation (OFCE), it has been tacitly assumed so far that the partial derivatives of the gray value distribution — which are required for this approach at the pixel positions involved — are independent from each other. [Nagel 94] has shown in a theoretical investigation how dropping this assumption affects the estimation procedure. The advantage of such a more rigorous approach consists in the possibility to replace heuristic tests for the local detection of discontinuities in optical flow fields by well known stochastic tests. First results from various experiments with this new approach are presented and discussed.

https://link.springer.com/content/pdf/10.1007%2FBFb0028363.pdf

Henner Kollnig

Papers

3D Pose Estimation by Directly Matching Polyhedral Models to Gray Value Gradients

Association of motion verbs with vehicle movements extracted from dense optical flow fields

Tracking of Occluded Vehicles in Traffic Scenes

Motion boundary detection in image sequences by local stochastic tests

Ermittlung von begrifflichen Beschreibungen von Geschehen in Straßenverkehrsszenen mit Hilfe unscharfer Mengen.