Background subtraction techniques: a review
10 Oct 2004-Vol. 4, pp 3099-3104
TL;DR: A review of the main methods and an original categorisation based on speed, memory requirements and accuracy can effectively guide the designer to select the most suitable method for a given application in a principled way.
Abstract: Background subtraction is a widely used approach for detecting moving objects from static cameras. Many different methods have been proposed over the recent years and both the novice and the expert can be confused about their benefits and limitations. In order to overcome this problem, this paper provides a review of the main methods and an original categorisation based on speed, memory requirements and accuracy. Such a review can effectively guide the designer to select the most suitable method for a given application in a principled way. Methods reviewed include parametric and non-parametric background density estimates and spatial correlation approaches.
Citations
More filters
TL;DR: Efficiency figures show that the proposed technique for motion detection outperforms recent and proven state-of-the-art methods in terms of both computation speed and detection rate.
Abstract: This paper presents a technique for motion detection that incorporates several innovative mechanisms. For example, our proposed technique stores, for each pixel, a set of values taken in the past at the same location or in the neighborhood. It then compares this set to the current pixel value in order to determine whether that pixel belongs to the background, and adapts the model by choosing randomly which values to substitute from the background model. This approach differs from those based upon the classical belief that the oldest values should be replaced first. Finally, when the pixel is found to be part of the background, its value is propagated into the background model of a neighboring pixel. We describe our method in full details (including pseudo-code and the parameter values used) and compare it to other background subtraction techniques. Efficiency figures show that our method outperforms recent and proven state-of-the-art methods in terms of both computation speed and detection rate. We also analyze the performance of a downscaled version of our algorithm to the absolute minimum of one comparison and one byte of memory per pixel. It appears that even such a simplified version of our algorithm performs better than mainstream techniques.
1,777 citations
Cites background from "Background subtraction techniques: ..."
...Since its introduction, the model has gained vastly in popularity amo ng the computer vision community [4], [7], [11], [42]–[44], an d it is still raising a lot of interest as authors continue to revi sit the method and propose enhanced algorithms [45]–[50]....
[...]
...According to [7], a background subtraction technique must adapt to gradual or fast illumination changes (changing tim e of day, clouds, etc), motion changes (camera oscillations), h igh frequency background objects (e....
[...]
TL;DR: This work proposes an approach based on self organization through artificial neural networks, widely applied in human image processing systems and more generally in cognitive science, that can handle scenes containing moving backgrounds, gradual illumination variations and camouflage, and achieves robust detection for different types of videos taken with stationary cameras.
Abstract: Detection of moving objects in video streams is the first relevant step of information extraction in many computer vision applications. Aside from the intrinsic usefulness of being able to segment video streams into moving and background components, detecting moving objects provides a focus of attention for recognition, classification, and activity analysis, making these later steps more efficient. We propose an approach based on self organization through artificial neural networks, widely applied in human image processing systems and more generally in cognitive science. The proposed approach can handle scenes containing moving backgrounds, gradual illumination variations and camouflage, has no bootstrapping limitations, can include into the background model shadows cast by moving objects, and achieves robust detection for different types of videos taken with stationary cameras. We compare our method with other modeling techniques and report experimental results, both in terms of detection accuracy and in terms of processing speed, for color video sequences that represent typical situations critical for video surveillance systems.
792 citations
TL;DR: A camera-based method for automatically quantifying the individual and social behaviors of fruit flies, Drosophila melanogaster, interacting in a planar arena finds that behavioral differences between individuals were consistent over time and were sufficient to accurately predict gender and genotype.
Abstract: We present a camera-based method for automatically quantifying the individual and social behaviors of fruit flies, Drosophila melanogaster, interacting in a planar arena Our system includes machine-vision algorithms that accurately track many individuals without swapping identities and classification algorithms that detect behaviors The data may be represented as an ethogram that plots the time course of behaviors exhibited by each fly or as a vector that concisely captures the statistical properties of all behaviors displayed in a given period We found that behavioral differences between individuals were consistent over time and were sufficient to accurately predict gender and genotype In addition, we found that the relative positions of flies during social interactions vary according to gender, genotype and social environment We expect that our software, which permits high-throughput screening, will complement existing molecular methods available in Drosophila, facilitating new investigations into the genetic and cellular basis of behavior
760 citations
Cites background from "Background subtraction techniques: ..."
...Classifying a pixel location as foreground (belonging to a fly) or background (not belonging to any fly) is referred to as background subtraction [5]....
[...]
TL;DR: This paper reviews the recent development of relevant technologies from the perspectives of computer vision and pattern recognition, and discusses how to face emerging challenges of intelligent multi-camera video surveillance.
Abstract: Intelligent multi-camera video surveillance is a multidisciplinary field related to computer vision, pattern recognition, signal processing, communication, embedded computing and image sensors. This paper reviews the recent development of relevant technologies from the perspectives of computer vision and pattern recognition. The covered topics include multi-camera calibration, computing the topology of camera networks, multi-camera tracking, object re-identification, multi-camera activity analysis and cooperative video surveillance both with active and static cameras. Detailed descriptions of their technical challenges and comparison of different solutions are provided. It emphasizes the connection and integration of different modules in various environments and application scenarios. According to the most recent works, some problems can be jointly solved in order to improve the efficiency and accuracy. With the fast development of surveillance systems, the scales and complexities of camera networks are increasing and the monitored environments are becoming more and more complicated and crowded. This paper discusses how to face these emerging challenges.
695 citations
Cites background from "Background subtraction techniques: ..."
...Background subtraction (Piccardi, 2004) is widely used for detecting moving objects in video surveillance with static cameras....
[...]
23 Jun 2014
TL;DR: The latest release of the changedetection.net dataset is presented, which includes 22 additional videos spanning 5 new categories that incorporate challenges encountered in many surveillance settings and highlights strengths and weaknesses of these methods and identifies remaining issues in change detection.
Abstract: Change detection is one of the most important lowlevel tasks in video analytics. In 2012, we introduced the changedetection.net (CDnet) benchmark, a video dataset devoted to the evalaution of change and motion detection approaches. Here, we present the latest release of the CDnet dataset, which includes 22 additional videos (70; 000 pixel-wise annotated frames) spanning 5 new categories that incorporate challenges encountered in many surveillance settings. We describe these categories in detail and provide an overview of the results of more than a dozen methods submitted to the IEEE Change DetectionWorkshop 2014. We highlight strengths and weaknesses of these methods and identify remaining issues in change detection.
680 citations
References
More filters
TL;DR: It is proved the convergence of a recursive mean shift procedure to the nearest stationary point of the underlying density function and, thus, its utility in detecting the modes of the density.
Abstract: A general non-parametric technique is proposed for the analysis of a complex multimodal feature space and to delineate arbitrarily shaped clusters in it. The basic computational module of the technique is an old pattern recognition procedure: the mean shift. For discrete data, we prove the convergence of a recursive mean shift procedure to the nearest stationary point of the underlying density function and, thus, its utility in detecting the modes of the density. The relation of the mean shift procedure to the Nadaraya-Watson estimator from kernel regression and the robust M-estimators; of location is also established. Algorithms for two low-level vision tasks discontinuity-preserving smoothing and image segmentation - are described as applications. In these algorithms, the only user-set parameter is the resolution of the analysis, and either gray-level or color images are accepted as input. Extensive experimental results illustrate their excellent performance.
11,727 citations
"Background subtraction techniques: ..." refers background or methods in this paper
...In order to address such issues, Elgammal et a/. in [ 7 ] have proposed to model the background distribution by a non-parametric model based on Kemel Density Estimation (KDE) on the buffer of the last n background values....
[...]
...The model proposed in [ 7 ] is actually more complex than what outlined so far....
[...]
...In [ 7 ], the background pdf is given as a sum of...
[...]
...In [ 7 ], instead, this same issue is addressed at the model level, by suggesting to evaluate P(xJ also in the models from neighbouring pixels and use the maximum value found in the comparison against T....
[...]
...Consequently, both the Mixture of Gaussians and KDE approaches can model well the background pdf in general cases. In addition, in [ 7 ] the...
[...]
23 Jun 1999
TL;DR: This paper discusses modeling each pixel as a mixture of Gaussians and using an on-line approximation to update the model, resulting in a stable, real-time outdoor tracker which reliably deals with lighting changes, repetitive motions from clutter, and long-term scene changes.
Abstract: A common method for real-time segmentation of moving regions in image sequences involves "background subtraction", or thresholding the error between an estimate of the image without moving objects and the current image. The numerous approaches to this problem differ in the type of background model used and the procedure used to update the model. This paper discusses modeling each pixel as a mixture of Gaussians and using an on-line approximation to update the model. The Gaussian, distributions of the adaptive mixture model are then evaluated to determine which are most likely to result from a background process. Each pixel is classified based on whether the Gaussian distribution which represents it most effectively is considered part of the background model. This results in a stable, real-time outdoor tracker which reliably deals with lighting changes, repetitive motions from clutter, and long-term scene changes. This system has been run almost continuously for 16 months, 24 hours a day, through rain and snow.
7,660 citations
"Background subtraction techniques: ..." refers background in this paper
...Mixture of K Gaussians (μi,σi,ωi) (Stauffer and Grimson, 1999)...
[...]
...Mixture of Gaussians Mixture of K Gaussians (µi,σi,ωi) (Stauffer and Grimson, 1999) In this way, the model copes also with multimodal background distributions; however: • the number of modes is arbitrarily pre-defined (usually from 3 to 5) • how to initialize the Gaussians?...
[...]
TL;DR: Pfinder is a real-time system for tracking people and interpreting their behavior that uses a multiclass statistical model of color and shape to obtain a 2D representation of head and hands in a wide range of viewing conditions.
Abstract: Pfinder is a real-time system for tracking people and interpreting their behavior. It runs at 10 Hz on a standard SGI Indy computer, and has performed reliably on thousands of people in many different physical locations. The system uses a multiclass statistical model of color and shape to obtain a 2D representation of head and hands in a wide range of viewing conditions. Pfinder has been successfully used in a wide range of applications including wireless interfaces, video databases, and low-bandwidth coding.
4,280 citations
"Background subtraction techniques: ..." refers background in this paper
...Running Gaussian average Pfinder (Wren, Azarbayejani, Darrell, Pentland, 1997): • fitting one Gaussian distribution (µ,σ) over the histogram: this gives the background PDF • background PDF update: running average: • In test | F - µ | > Th, Th can be chosen as kσ • It does not cope with multimodal…...
[...]
...Running Gaussian average Pfinder (Wren, Azarbayejani, Darrell, Pentland, 1997): • fitting one Gaussian distribution (µ,σ) over the histogram: this gives the background PDF • background PDF update: running average: • In test | F - µ | > Th, Th can be chosen as kσ • It does not cope with multimodal backgrounds ( ) tt1t µα1Fαµ −+=+ ( ) 2t2tt2 1t σα1)µF(ασ −+−=+...
[...]
TL;DR: This paper focuses on motion tracking and shows how one can use observed motion to learn patterns of activity in a site and create a hierarchical binary-tree classification of the representations within a sequence.
Abstract: Our goal is to develop a visual monitoring system that passively observes moving objects in a site and learns patterns of activity from those observations. For extended sites, the system will require multiple cameras. Thus, key elements of the system are motion tracking, camera coordination, activity classification, and event detection. In this paper, we focus on motion tracking and show how one can use observed motion to learn patterns of activity in a site. Motion segmentation is based on an adaptive background subtraction method that models each pixel as a mixture of Gaussians and uses an online approximation to update the model. The Gaussian distributions are then evaluated to determine which are most likely to result from a background process. This yields a stable, real-time outdoor tracker that reliably deals with lighting changes, repetitive motions from clutter, and long-term scene changes. While a tracking system is unaware of the identity of any object it tracks, the identity remains the same for the entire tracking sequence. Our system leverages this information by accumulating joint co-occurrences of the representations within a sequence. These joint co-occurrence statistics are then used to create a hierarchical binary-tree classification of the representations. This method is useful for classifying sequences, as well as individual instances of activities in a site.
3,631 citations
26 Jun 2000
TL;DR: A novel non-parametric background model that can handle situations where the background of the scene is cluttered and not completely static but contains small motions such as tree branches and bushes is presented.
Abstract: Background subtraction is a method typically used to segment moving regions in image sequences taken from a static camera by comparing each new frame to a model of the scene background. We present a novel non-parametric background model and a background subtraction approach. The model can handle situations where the background of the scene is cluttered and not completely static but contains small motions such as tree branches and bushes. The model estimates the probability of observing pixel intensity values based on a sample of intensity values for each pixel. The model adapts quickly to changes in the scene which enables very sensitive detection of moving targets. We also show how the model can use color information to suppress detection of shadows. The implementation of the model runs in real-time for both gray level and color imagery. Evaluation shows that this approach achieves very sensitive detection with very low false alarm rates.
2,432 citations
"Background subtraction techniques: ..." refers background in this paper
...…In many works, such history is: • just the previous n frames • a weighted average where recent frames have higher weight In essence, the background model is computed as a chronological average from the pixel’s history No spatial correlation is used between different (neighbouring) pixel locations...
[...]