Robust Tracking in A Camera Network: A Multi-Objective Optimization Framework
TL;DR: These experiments prove that incorporation of the long-term models enable us to hold tracks of objects over extended periods of time, including situations where there are large ldquoblindrdquo areas.
Abstract: We address the problem of tracking multiple people in a network of nonoverlapping cameras. This introduces certain challenges that are unique to this particular application scenario, in addition to existing challenges in tracking like pose and illumination variations, occlusion, clutter and sensor noise. For this purpose, we propose a novel multi-objective optimization framework by combining short term feature correspondences across the cameras with long-term feature dependency models. The overall solution strategy involves adapting the similarities between features observed at different cameras based on the long-term models and finding the stochastically optimal path for each person. For modeling the long-term interdependence of the features over space and time, we propose a novel method based on discriminant analysis models. The entire process allows us to adaptively evolve the feature correspondences by observing the system performance over a time window, and correct for errors in the similarity estimations. We show results on data collected by two large camera networks. These experiments prove that incorporation of the long-term models enable us to hold tracks of objects over extended periods of time, including situations where there are large ldquoblindrdquo areas. The proposed approach is implemented by distributing the processing over the entire network.
Citations
More filters
TL;DR: This paper reviews the recent development of relevant technologies from the perspectives of computer vision and pattern recognition, and discusses how to face emerging challenges of intelligent multi-camera video surveillance.
695 citations
Cites background from "Robust Tracking in A Camera Network..."
...…integrated with spatio-temporal reasoning (Alexander and Lucchesi, xxxx; Huang and Russell, 1997; Pasula et al., 1999; Veenman et al., 2001; Javed et al., 2003; Shafique and Shah, 2003; Morariu and Camps, 2006; Jiang et al., 2007; Song and Roy-Chowdhury, 2008; Hamid et al., 2010; Kuo et al., 2010)....
[...]
TL;DR: Based on how cameras share estimates and fuse information, this article classified these trackers as distributed, decentralized, and centralized algorithms and highlighted the challenges to be addressed in the design of decentralized and distributed tracking algorithms.
Abstract: We discussed emerging multicamera tracking algorithms that find their roots in signal processing, wireless sensor networks, and computer vision. Based on how cameras share estimates and fuse information, we classified these trackers as distributed, decentralized, and centralized algorithms. We also highlighted the challenges to be addressed in the design of decentralized and distributed tracking algorithms. In particular, we showed how the constraints derived from the topology of the networks and the nature of the task have favored so far decentralized architectures with multiple local fusion centers. Because of the availability of fewer fusion centers compared to distributed algorithms, decentralized algorithms can share larger amounts of data (e.g., occupancy maps) and can back-project estimates among views and fusion centers to validate results. Distributed tracking uses algorithms that can operate with smaller amounts of data at any particular node and obtain state estimates through iterative fusion. Despite recent advances, there are important issues to be addressed to achieve efficient multitarget multicamera tracking. Current algorithms either assume the track-to-measurement association information to be available for the tracker or operate on a small (known) number of targets. Algorithms performing track-to-measurement association for a time-varying number of targets with higher accuracy usually incur much higher costs, whose reduction is an important open problem to be addressed in multicamera networks.
139 citations
05 Sep 2010
TL;DR: This work proposes a novel system for associating multi-target tracks across multiple non-overlapping cameras by an on-line learned discriminative appearance affinity model and presents an improved inter-camera track association framework to solve the "target handover" problem across cameras.
Abstract: We propose a novel system for associating multi-target tracks across multiple non-overlapping cameras by an on-line learned discriminative appearance affinity model. Collecting reliable training samples is a major challenge in on-line learning since supervised correspondence is not available at runtime. To alleviate the inevitable ambiguities in these samples, Multiple Instance Learning (MIL) is applied to learn an appearance affinity model which effectively combines three complementary image descriptors and their corresponding similarity measurements. Based on the spatial-temporal information and the proposed appearance affinity model, we present an improved inter-camera track association framework to solve the "target handover" problem across cameras. Our evaluations indicate that our method have higher discrimination between different targets than previous methods.
120 citations
Cites background from "Robust Tracking in A Camera Network..."
...Song and Roy-Chowdhury [18] proposed a multi-objective optimization framework by combining short-term feature correspondences across the cameras with long-term feature dependency models....
[...]
TL;DR: This article reviews and analyzes different desirable objectives to show whether they conflict with each other, support each other or they are design dependent, and presents a generic multi-objective optimization problem relating to wireless sensor network which consists of input variables, required output, objectives and constraints.
Abstract: Optimization problems relating to wireless sensor network planning, design, deployment and operation often give rise to multi-objective optimization formulations where multiple desirable objectives compete with each other and the decision maker has to select one of the tradeoff solutions. These multiple objectives may or may not conflict with each other. Keeping in view the nature of the application, the sensing scenario and input/output of the problem, the type of optimization problem changes. To address different nature of optimization problems relating to wireless sensor network design, deployment, operation, planing and placement, there exist a plethora of optimization solution types. We review and analyze different desirable objectives to show whether they conflict with each other, support each other or they are design dependent. We also present a generic multi-objective optimization problem relating to wireless sensor network which consists of input variables, required output, objectives and constraints. A list of constraints is also presented to give an overview of different constraints which are considered while formulating the optimization problems in wireless sensor networks. Keeping in view the multi facet coverage of this article relating to multi-objective optimization, this will open up new avenues of research in the area of multi-objective optimization relating to wireless sensor networks.
110 citations
Cites methods from "Robust Tracking in A Camera Network..."
...A stochastic algorithm has been used in [188] to address the problem of tracking multiple people in a network of video sensors....
[...]
TL;DR: An information-weighted, consensus-based, distributed multi-target tracking algorithm referred to as the Multi-target Information Consensus (MTIC) algorithm that is designed to address both the naivety and the data association problems and converges to the centralized minimum mean square error estimate.
Abstract: Distributed algorithms have recently gained immense popularity. With regards to computer vision applications, distributed multi-target tracking in a camera network is a fundamental problem. The goal is for all cameras to have accurate state estimates for all targets. Distributed estimation algorithms work by exchanging information between sensors that are communication neighbors. Vision-based distributed multi-target state estimation has at least two characteristics that distinguishes it from other applications. First, cameras are directional sensors and often neighboring sensors may not be sensing the same targets, i.e., they are naive with respect to that target. Second, in the presence of clutter and multiple targets, each camera must solve a data association problem. This paper presents an information-weighted, consensus-based, distributed multi-target tracking algorithm referred to as the Multi-target Information Consensus (MTIC) algorithm that is designed to address both the naivety and the data association problems. It converges to the centralized minimum mean square error estimate. The proposed MTIC algorithm and its extensions to non-linear camera models, termed as the Extended MTIC (EMTIC), are robust to false measurements and limited resources like power, bandwidth and the real-time operational requirements. Simulation and experimental analysis are provided to support the theoretical results.
87 citations
Cites background from "Robust Tracking in A Camera Network..."
...In [13], a multi-objective optimization based tracking framework was proposed which computes the optimal association of the measurements of each target over time....
[...]
References
More filters
Book•
18 May 2005
TL;DR: This paper presents a meta-modelling framework for estimating the modeled solutions for various types of optimization problems in the multicriteria setting.
Abstract: Multicriteria Optimization and Decision Analysis, 2017 ... Multicriteria Optimization | Request PDF Multicriteria Optimization Solving Multicriteria Optimization Problems with WebOptim ... Multicriteria Optimization | Matthias Ehrgott | Springer Multicriteria Optimization | Matthias Ehrgott | Springer Multicriteria VMAT optimization PubMed Central (PMC) Multicriteria Optimization | Guide books Multiple-criteria decision analysis Wikipedia Multicriteria optimization: Sitespecific class solutions ... Multicriteria Optimization | SpringerLink Multicriteria Optimization Harvard University Multicriteria Optimization Matthias Ehrgott Google Books Multicriteria optimization in humanitarian aid ScienceDirect Multicriteria Optimization and
2,422 citations
1,106 citations
13 Oct 2003
TL;DR: This work presents a novel approach for establishing object correspondence across non-overlapping cameras, which exploits the redundance in paths that people and cars tend to follow, e.g. roads, walk-ways or corridors, by using motion trends and appearance of objects, to establish correspondence.
Abstract: Conventional tracking approaches assume proximity in space, time and appearance of objects in successive observations. However, observations of objects are often widely separated in time and space when viewed from multiple non-overlapping cameras. To address this problem, we present a novel approach for establishing object correspondence across non-overlapping cameras. Our multicamera tracking algorithm exploits the redundance in paths that people and cars tend to follow, e.g. roads, walk-ways or corridors, by using motion trends and appearance of objects, to establish correspondence. Our system does not require any inter-camera calibration, instead the system learns the camera topology and path probabilities of objects using Parzen windows, during a training phase. Once the training is complete, correspondences are assigned using the maximum a posteriori (MAP) estimation framework. The learned parameters are updated with changing trajectory patterns. Experiments with real world videos are reported, which validate the proposed approach.
531 citations
"Robust Tracking in A Camera Network..." refers background or methods in this paper
...The first objective is to choose the tracks such that they maximize the similarity between the observed features in a local neighborhood....
[...]
...We now perform an edge weight adaptation scheme for minimizing until we reach a local minimum....
[...]
...Motion in an image is detected and the region is segmented to obtain the features of the person....
[...]
...As in [11], the bipartite graph is obtained by splitting each vertex into and , where the edge connected to represents the path coming into while the edge connected to represents the path going out of ....
[...]
...Some of the existing methods on tracking in a camera network include [10], [11], [15]....
[...]
TL;DR: In this paper, the optimization theory of Dubovitskii and Milyutin is extended to multiobjective optimization problems, producing new necessary conditions for local Pareto optima.
Abstract: In this study, the optimization theory of Dubovitskii and Milyutin is extended to multiobjective optimization problems, producing new necessary conditions for local Pareto optima. Cones of directions of decrease, cones of feasible directions and a cone of tangent directions, as well as, a new cone of directions of nonincrease play an important role here. The dual cones to the cones of direction of decrease and to the cones of directions of nonincrease are characterized for convex functionals without differentiability, with the aid of their subdifferential, making the optimality theorems applicable. The theory is applied to vector mathematical programming, giving a generalized Fritz John theorem, and other applications are mentioned. It turns out that, under suitable convexity and regularity assumptions, the necessary conditions for local Pareto optima are also necessary and sufficient for global Pareto optimum. With the aid of the theory presented here, a result is obtained for the, so-called, “scalarization” problem of multiobjective optimization.
400 citations