We discussed emerging multicamera tracking algorithms that find their roots in signal processing, wireless sensor networks, and computer vision. Based on how cameras share estimates and fuse information, we classified these trackers as distributed, decentralized, and centralized algorithms. We also highlighted the challenges to be addressed in the design of decentralized and distributed tracking algorithms. In particular, we showed how the constraints derived from the topology of the networks and the nature of the task have favored so far decentralized architectures with multiple local fusion centers. Because of the availability of fewer fusion centers compared to distributed algorithms, decentralized algorithms can share larger amounts of data (e.g., occupancy maps) and can back-project estimates among views and fusion centers to validate results. Distributed tracking uses algorithms that can operate with smaller amounts of data at any particular node and obtain state estimates through iterative fusion. Despite recent advances, there are important issues to be addressed to achieve efficient multitarget multicamera tracking. Current algorithms either assume the track-to-measurement association information to be available for the tracker or operate on a small (known) number of targets. Algorithms performing track-to-measurement association for a time-varying number of targets with higher accuracy usually incur much higher costs, whose reduction is an important open problem to be addressed in multicamera networks.

Distributed and Decentralized Multicamera Tracking

We employ hierarchical data association to track players in team sports. Player movements are often complex and highly correlated with both nearby and distant players. A single model would require many degrees of freedom to represent the full motion diversity and could be difficult to use in practice. Instead, we introduce a set of Game Context Features extracted from noisy detections to describe the current state of the match, such as how the players are spatially distributed. Our assumption is that players react to the current situation in only a finite number of ways. As a result, we are able to select an appropriate simplified affinity model for each player and time instant using a random decision forest based on current track and game context features. Our context-conditioned motion models implicitly incorporate complex inter-object correlations while remaining tractable. We demonstrate significant performance improvements over existing multi-target tracking algorithms on basketball and field hockey sequences several minutes in duration and containing 10 and 20 players respectively.

/pdf/tracking-sports-players-with-context-conditioned-motion-2hvql9mr00.pdf

Tracking Sports Players with Context-Conditioned Motion Models

3D face recognition using local binary patterns

Battery lifetime is critical for wireless video sensors. To enable battery-powered wireless video sensors, low-power design is required. In this paper, we consider applying multi-view summarization to wireless video sensors to remove redundant contents such that the compression and transmission power can be reduced. A low-complexity online multi-view video summarization scheme is proposed. Experiments show that the proposed summarization method successfully reduces the video content while keeping important events. A power analysis of the system also shows that a significant amount of energy can be saved.

On-Line Multi-View Video Summarization for Wireless Video Sensor Network

We utilize ideas from two growing but disparate ideas in computer vision-shape analysis using tools from differential geometry and feature selection using machine learning-to select and highlight salient geometrical facial features that contribute most in 3-D face recognition and gender classification. First, a large set of geometries curve features are extracted using level sets (circular curves) and streamlines (radial curves) of the Euclidean distance functions of the facial surface; together they approximate facial surfaces with arbitrarily high accuracy. Then, we use the well-known Adaboost algorithm for feature selection from this large set and derive a composite classifier that achieves high performance with a minimal set of features. This greatly reduced set, consisting of some level curves on the nose and some radial curves in the forehead and cheeks regions, provides a very compact signature of a 3-D face and a fast classification algorithm for face recognition and gender selection. It is also efficient in terms of data storage and transmission costs. Experimental results, carried out using the FRGCv2 dataset, yield a rank-1 face recognition rate of 98% and a gender classification rate of 86% rate.

/pdf/boosting-3-d-geometric-features-for-efficient-face-3mvugpnwrr.pdf

Boosting 3-D-Geometric Features for Efficient Face Recognition and Gender Classification

We present a content-aware multi-camera selection technique that uses object- and frame-level features. First objects are detected using a color-based change detector. Next trajectory information for each object is generated using multi-frame graph matching. Finally, multiple features including size and location are used to generate an object score. At frame-level, we consider total activity, event score, number of objects and cumulative object score. These features are used to generate score information using a multivariate Gaussian distribution. The algorithm. The best view is selected using a Dynamic Bayesian Network (DBN), which utilizes camera network information. DBN employs previous view information to select the current view thus increasing resilience to frequent switching. The performance of the proposed approach is demonstrated on three multi-camera setups with semi-overlapping fields of view: a basketball game, an indoor airport surveillance scenario and a synthetic outdoor pedestrian dataset. We compare the proposed view selection approach with a maximum score based camera selection criterion and demonstrate a significant decrease in camera flickering. The performance of the proposed approach is also validated through subjective testing.

/pdf/content-and-task-based-view-selection-from-multiple-video-3iytxphdqz.pdf

Content and task-based view selection from multiple video streams

We present a novel algorithm for automated video production based on content ranking. The proposed algorithm generates videos by performing camera selection while minimizing the number of inter-camera switch. We model the problem as a finite horizon Partially Observable Markov Decision Process over temporal windows and we use a multivariate Gaussian distribution to represent the content-quality score for each camera. The performance of the proposed approach is demonstrated on a multi-camera setup of fixed cameras with partially overlapping fields of view. Subjective experiments based on the Turing test confirmed the quality of the automatically produced videos. The proposed approach is also compared with recent methods based on Recursive Decision and on Dynamic Bayesian Networks and its results outperform both methods.

/pdf/multi-camera-scheduling-for-video-production-52mrjusg81.pdf

Multi-camera Scheduling for Video Production

We present a novel approach to 3D face recognition using compact face signatures based on automatically detected 3D landmarks. We represent the face geometry with inter-landmark distances within selected regions of interest to achieve robustness to expression variations. The inter-landmark distances are compressed through Principal Component Analysis and Linear Discriminant Analysis is then applied on the reduced features to maximize the separation between face classes. The classification of a probe face is based on a nearest mean classifier after transforming the probe onto the subspace. We analyze the performance of different landmark combinations (signatures) to determine a signature that is robust to expressions. The selected signature is then used to train a Point Distribution Model for the automatic localization of the landmarks, without any prior knowledge of scale, pose, orientation or texture. We evaluate the proposed approach on a challenging publicly available facial expression database (BU-3DFE) and achieve 96.5% recognition rate using the automatically localized signature. Moreover, because of its compactness the face signature can be stored on 2D barcodes and used for radio-frequency identification.

/pdf/compact-signatures-for-3d-face-recognition-under-varying-4gwn400wxx.pdf

Compact Signatures for 3D Face Recognition under Varying Expressions

We present a technique for estimating the location of the ball during a basketball game without using a detector. The technique is based on the analysis of the dynamics in the scene and allows us to overcome the challenges due to frequent occlusions of the ball and its similarity in appearance with the background. Based on the assumption that the ball is the point of focus of the game and that the motion flow of the players is dependent on its position during attack actions, the most probable candidates for the ball location are extracted from each frame. These candidates are then validated over time using a Kalman filter. Experimental results on a real basketball dataset show that the location of the ball can be estimated with an average accuracy of 82%.

Detector-less ball localization using context and motion flow analysis

We present a motion classification approach to detect movements of interest (abnormal motion) based on local feature modeling within spatio-temporal detectors. The modeling is performed using motion vectors and local detectors. The detectors are trained independently for learning abnormal motion based on labeled samples. Each detector is assigned an abnormality score, both in space and time, which is the basis of the final classification. The spatial relationship across detectors is used to discriminate simultaneous occurrences of abnormal motion. The performance of the proposed method is evaluated on 52 hours of the multi-camera surveillance dataset of the TRECVID [1] 2010 challenge.

Fahad Daniyal

Papers

Content and task-based view selection from multiple video streams

Multi-camera Scheduling for Video Production

Compact Signatures for 3D Face Recognition under Varying Expressions

Detector-less ball localization using context and motion flow analysis

Abnormal motion detection in crowded scenes using local spatio-temporal analysis