A new video segmentation method of moving objects based on blob-level knowledge

doi:10.1016/J.PATREC.2007.10.007

Home
/
Papers
/
A new video segmentation method of moving objects based on blob-level knowledge

Journal Article•DOI•

A new video segmentation method of moving objects based on blob-level knowledge

Enrique J. Carmona¹, Javier Martínez-Cantos¹, José Mira¹•Institutions (1)

ETSI¹

01 Feb 2008-Pattern Recognition Letters (Elsevier Science Inc.)-Vol. 29, Iss: 3, pp 272-285

TL;DR: This work proposes a new approach to the background subtraction method which operates in the colour space and manages the colour information in the segmentation process to detect and eliminate noise.

read less

Abstract: Variants of the background subtraction method are broadly used for the detection of moving objects in video sequences in different applications. In this work we propose a new approach to the background subtraction method which operates in the colour space and manages the colour information in the segmentation process to detect and eliminate noise. This new method is combined with blob-level knowledge associated with different types of blobs that may appear in the foreground. The idea is to process each pixel differently according to the category to which it belongs: real moving objects, shadows, ghosts, reflections, fluctuation or background noise. Thus, the foreground resulting from processing each image frame is refined selectively, applying at each instant the appropriate operator according to the type of noise blob we wish to eliminate. The approach proposed is adaptive, because it allows both the background model and threshold model to be updated. On the one hand, the results obtained confirm the robustness of the method proposed in a wide range of different sequences and, on the other hand, these results underline the importance of handling three colour components in the segmentation process rather than just the one grey-level component.

...read moreread less

Citations

PDF

Open Access

More filters

Journal Article•DOI•

Fast Background Subtraction Based on a Multilayer Codebook Model for Moving Object Detection

[...]

Jing-Ming Guo¹, Chih-Hsien Hsia², Yun-Fu Liu¹, Min-Hsiung Shih¹, Cheng-Hsin Chang¹, Jing-Yu Wu¹ - Show less +2 more•Institutions (2)

National Taiwan University of Science and Technology¹, Chinese Culture University²

01 Oct 2013-IEEE Transactions on Circuits and Systems for Video Technology

TL;DR: The pixel-based classification is adopted for refining the results from the block-based background subtraction, which can further classify pixels as foreground, shadows, and highlights and can provide a high precision and efficient processing speed to meet the requirements of real-time moving object detection.

...read moreread less

Abstract: Moving object detection is an important and fundamental step for intelligent video surveillance systems because it provides a focus of attention for post-processing. A multilayer codebook-based background subtraction (MCBS) model is proposed for video sequences to detect moving objects. Combining the multilayer block-based strategy and the adaptive feature extraction from blocks of various sizes, the proposed method can remove most of the nonstationary (dynamic) background and significantly increase the processing efficiency. Moreover, the pixel-based classification is adopted for refining the results from the block-based background subtraction, which can further classify pixels as foreground, shadows, and highlights. As a result, the proposed scheme can provide a high precision and efficient processing speed to meet the requirements of real-time moving object detection.

...read moreread less

99 citations

Cites methods from "A new video segmentation method of ..."

...[16] is applied, in which a cone-shaped color model is proposed to improve the CB’s color model, which is more robust and is employed in this paper to classify the pixels....
[...]

Journal Article•DOI•

Hierarchical Method for Foreground Detection Using Codebook Model

[...]

Jing-Ming Guo¹, Yun-Fu Liu¹, Chih-Hsien Hsia¹, Min-Hsiung Shih¹, Chih-Sheng Hsu¹ - Show less +1 more•Institutions (1)

National Taiwan University of Science and Technology¹

01 Jun 2011-IEEE Transactions on Circuits and Systems for Video Technology

TL;DR: This paper presents a hierarchical scheme with block-based and pixel-based codebooks for foreground detection with superior performance to that of the former related approaches.

...read moreread less

Abstract: This paper presents a hierarchical scheme with block-based and pixel-based codebooks for foreground detection. The codebook is mainly used to compress information to achieve a high efficient processing speed. In the block-based stage, 12 intensity values are employed to represent a block. The algorithm extends the concept of the block truncation coding, and thus it can further improve the processing efficiency by enjoying its low complexity advantage. In detail, the block-based stage can remove most of the backgrounds without reducing the true positive rate, yet it has low precision. To overcome this problem, the pixel-based stage is adopted to enhance the precision, which also can reduce the false positive rate. Moreover, the short-term information is employed to improve background updating for adaptive environments. As documented in the experimental results, the proposed algorithm can provide superior performance to that of the former related approaches.

...read moreread less

92 citations

Cites background or methods from "A new video segmentation method of ..."

...Another RGB color model proposed by Carmona et al. [ 3 ] can solve the third problem of [2], yet it needs too many parameters for their color model....
[...]
...In Carmona et al.’s work [ 3 ], a color model was proposed, which classified a pixel into shadow, highlight, background, and foreground four states in RGB color space....
[...]
...Moreover, a color model from the former approach [ 3 ] which can distinguish shadow, highlight, background, and foreground is modified with fewer parameters for improving the efficiency....
[...]

Journal Article•DOI•

Robust real-time ship detection and tracking for visual surveillance of cage aquaculture

[...]

Wu-Chih Hu¹, Ching-Yu Yang¹, Deng-Yuan Huang²•Institutions (2)

National Penghu University of Science and Technology¹, Dayeh University²

01 Aug 2011-Journal of Visual Communication and Image Representation

TL;DR: A visual surveillance scheme for cage aquaculture that automatically detects and tracks ships (intruders) and a fast 4-connected component labeling method to greatly reduce the computational cost associated with the conventional method is presented.

...read moreread less

Abstract: This paper presents a visual surveillance scheme for cage aquaculture that automatically detects and tracks ships (intruders). For ship detection and tracking, we propose a robust foreground detection and background updating to effectively reduce the influence of sea waves. Furthermore, we propose a fast 4-connected component labeling method to greatly reduce the computational cost associated with the conventional method. Wave ripples are removed from regions with ships. An improved full search algorithm based on adaptive template block matching with a wave ripple removal is presented to quickly, accurately, and reliably track overlapping ships whose scales change. Experimental results demonstrate that the proposed schemes have outstanding performance in ship detection and tracking. The proposed visual surveillance system for cage aquaculture triggers an alarm if intruders are detected. The security of cage aquaculture can be increased. The proposed visual surveillance can thus greatly help the popularization of cage aquaculture for ocean farming.

...read moreread less

60 citations

Journal Article•

Review and evaluation of well-known methods for moving object detection and tracking in videos

[...]

Bahadir Karasulu¹•Institutions (1)

Ege University¹

26 Mar 2010-Journal of Aeronautics and Space Technologies

TL;DR: This paper provides a systematic review of object D&T algorithms and performance measures and assesses their effectiveness via metrics.

...read moreread less

Abstract: Moving object detection and tracking (D&T) are important initial steps in object recognition, context analysis and indexing processes for visual surveillance systems. It is a big challenge for researchers to make a decision on which D&T algorithm is more suitable for which situation and/or environment and to determine how accurately object D&T (real-time or non-real-time) is made. There is a variety of object D&T algorithms (i.e. methods) and publications on their performance comparison and evaluation via performance metrics. This paper provides a systematic review of these algorithms and performance measures and assesses their effectiveness via metrics.

...read moreread less

51 citations

Cites methods from "A new video segmentation method of ..."

...In the foreground detection step, pixels in the video frame, which are not explained enough by the background model [5], are defined as a binary candidate foreground mask [23]....
[...]
...One can find bunch of methods dedicated to generic-object D&T in video processing like Background Subtraction (BS) [5, 6], Mean-Shift (MS) and/or Continuously Adaptive Mean-Shift (CMS) [7-9], Optical Flow (OF) [10, 11], Active Contour Models (i....
[...]

Journal Article•DOI•

Combining shadow detection and simulation for estimation of vehicle size and position

[...]

Björn Johansson¹, Johan Wiklund¹, Per-Erik Forssén¹, Gösta H. Granlund¹•Institutions (1)

Linköping University¹

01 Jun 2009-Pattern Recognition Letters

TL;DR: It is shown in a number of experiments and examples how the combination shadow detection/simulation improves the estimation compared to just using detection or simulation, especially when the shadow detection or the simulation is inaccurate.

...read moreread less

Abstract: This paper presents a method that combines shadow detection and a 3D box model including shadow simulation, for estimation of size and position of vehicles. We define a similarity measure between a simulated image of a 3D box, including the box shadow, and a captured image that is classified into background/foreground/shadow. The similarity measure is used in an optimization procedure to find the optimal box state. It is shown in a number of experiments and examples how the combination shadow detection/simulation improves the estimation compared to just using detection or simulation, especially when the shadow detection or the simulation is inaccurate. We also describe a tracking system that utilizes the estimated 3D boxes, including highlight detection, spatial window instead of a time based window for predicting heading, and refined box size estimates by weighting accumulated estimates depending on view. Finally, we show example results.

...read moreread less

49 citations

Cites methods from "A new video segmentation method of ..."

...Another recent similar method is (Carmona et al., 2008) that classifies pixels into even more categories such as reflections (similar to our highlights), ghosts, and fluctuations....
[...]

1
2
3
4
…
5
6
7
8
9
10
11
12
13
14

Collapse

References

PDF

Open Access

More filters

Proceedings Article•DOI•

Adaptive background mixture models for real-time tracking

[...]

Chris Stauffer¹, W.E.L. Grimson¹•Institutions (1)

Massachusetts Institute of Technology¹

23 Jun 1999

TL;DR: This paper discusses modeling each pixel as a mixture of Gaussians and using an on-line approximation to update the model, resulting in a stable, real-time outdoor tracker which reliably deals with lighting changes, repetitive motions from clutter, and long-term scene changes.

...read moreread less

Abstract: A common method for real-time segmentation of moving regions in image sequences involves "background subtraction", or thresholding the error between an estimate of the image without moving objects and the current image. The numerous approaches to this problem differ in the type of background model used and the procedure used to update the model. This paper discusses modeling each pixel as a mixture of Gaussians and using an on-line approximation to update the model. The Gaussian, distributions of the adaptive mixture model are then evaluated to determine which are most likely to result from a background process. Each pixel is classified based on whether the Gaussian distribution which represents it most effectively is considered part of the background model. This results in a stable, real-time outdoor tracker which reliably deals with lighting changes, repetitive motions from clutter, and long-term scene changes. This system has been run almost continuously for 16 months, 24 hours a day, through rain and snow.

...read moreread less

7,660 citations

"A new video segmentation method of ..." refers methods in this paper

...The approach proposed is adaptive, because it allows both the background model and threshold model to be updated....
[...]
...There are different methods for detecting moving objects based, for example, on statistical methods (Horprasert et al., 1999; Lee, 2005; Stauffer and Grimson, 1999), fuzzy logic (Jadon et al., 2001), the subtraction of consecutive frames (Lipton et al., 1998), optical flow (Wang et al., 2003),…...
[...]

Journal Article•DOI•

Pfinder: real-time tracking of the human body

[...]

Christopher R. Wren¹, Ali Azarbayejani¹, Trevor Darrell¹, Alex Pentland¹•Institutions (1)

Massachusetts Institute of Technology¹

01 Jul 1997-IEEE Transactions on Pattern Analysis and Machine Intelligence

TL;DR: Pfinder is a real-time system for tracking people and interpreting their behavior that uses a multiclass statistical model of color and shape to obtain a 2D representation of head and hands in a wide range of viewing conditions.

...read moreread less

Abstract: Pfinder is a real-time system for tracking people and interpreting their behavior. It runs at 10 Hz on a standard SGI Indy computer, and has performed reliably on thousands of people in many different physical locations. The system uses a multiclass statistical model of color and shape to obtain a 2D representation of head and hands in a wide range of viewing conditions. Pfinder has been successfully used in a wide range of applications including wireless interfaces, video databases, and low-bandwidth coding.

...read moreread less

4,280 citations

"A new video segmentation method of ..." refers background or methods in this paper

...Keywords: Background subtraction; Reflection detection; Shadow detection; Ghost detection; Permanence memory; Blob-level knowledge...
[...]
...Nevertheless, one of the most frequently used approaches with a fixed camera is based on background subtraction method and its multiple variants (Wren et al., 1997; Haritaoglu et al., 2000; Stauffer and Grimson, 2000; McKenna et al., 2000; Kim and Kim, 2003; Cucchiara et al., 2003; Xu et al., 2005;…...
[...]

Journal Article•DOI•

Learning patterns of activity using real-time tracking

[...]

Chris Stauffer¹, W.E.L. Grimson¹•Institutions (1)

Massachusetts Institute of Technology¹

01 Aug 2000-IEEE Transactions on Pattern Analysis and Machine Intelligence

TL;DR: This paper focuses on motion tracking and shows how one can use observed motion to learn patterns of activity in a site and create a hierarchical binary-tree classification of the representations within a sequence.

...read moreread less

Abstract: Our goal is to develop a visual monitoring system that passively observes moving objects in a site and learns patterns of activity from those observations. For extended sites, the system will require multiple cameras. Thus, key elements of the system are motion tracking, camera coordination, activity classification, and event detection. In this paper, we focus on motion tracking and show how one can use observed motion to learn patterns of activity in a site. Motion segmentation is based on an adaptive background subtraction method that models each pixel as a mixture of Gaussians and uses an online approximation to update the model. The Gaussian distributions are then evaluated to determine which are most likely to result from a background process. This yields a stable, real-time outdoor tracker that reliably deals with lighting changes, repetitive motions from clutter, and long-term scene changes. While a tracking system is unaware of the identity of any object it tracks, the identity remains the same for the entire tracking sequence. Our system leverages this information by accumulating joint co-occurrences of the representations within a sequence. These joint co-occurrence statistics are then used to create a hierarchical binary-tree classification of the representations. This method is useful for classifying sequences, as well as individual instances of activities in a site.

...read moreread less

3,631 citations

"A new video segmentation method of ..." refers background or methods in this paper

...Keywords: Background subtraction; Reflection detection; Shadow detection; Ghost detection; Permanence memory; Blob-level knowledge...
[...]
...…used approaches with a fixed camera is based on background subtraction method and its multiple variants (Wren et al., 1997; Haritaoglu et al., 2000; Stauffer and Grimson, 2000; McKenna et al., 2000; Kim and Kim, 2003; Cucchiara et al., 2003; Xu et al., 2005; Leone and Distante, 2007), because of…...
[...]

Journal Article•DOI•

W/sup 4/: real-time surveillance of people and their activities

[...]

Ismail Haritaoglu¹, D. Harwood², Larry S. Davis²•Institutions (2)

IBM¹, University of Maryland, College Park²

01 Aug 2000-IEEE Transactions on Pattern Analysis and Machine Intelligence

TL;DR: W/sup 4/ employs a combination of shape analysis and tracking to locate people and their parts and to create models of people's appearance so that they can be tracked through interactions such as occlusions.

...read moreread less

Abstract: W/sup 4/ is a real time visual surveillance system for detecting and tracking multiple people and monitoring their activities in an outdoor environment. It operates on monocular gray-scale video imagery, or on video imagery from an infrared camera. W/sup 4/ employs a combination of shape analysis and tracking to locate people and their parts (head, hands, feet, torso) and to create models of people's appearance so that they can be tracked through interactions such as occlusions. It can determine whether a foreground region contains multiple people and can segment the region into its constituent people and track them. W/sup 4/ can also determine whether people are carrying objects, and can segment objects from their silhouettes, and construct appearance models for them so they can be identified in subsequent frames. W/sup 4/ can recognize events between people and objects, such as depositing an object, exchanging bags, or removing an object. It runs at 25 Hz for 320/spl times/240 resolution images on a 400 MHz dual-Pentium II PC.

...read moreread less

2,870 citations

"A new video segmentation method of ..." refers background or methods in this paper

...Keywords: Background subtraction; Reflection detection; Shadow detection; Ghost detection; Permanence memory; Blob-level knowledge...
[...]
...…of the most frequently used approaches with a fixed camera is based on background subtraction method and its multiple variants (Wren et al., 1997; Haritaoglu et al., 2000; Stauffer and Grimson, 2000; McKenna et al., 2000; Kim and Kim, 2003; Cucchiara et al., 2003; Xu et al., 2005; Leone and…...
[...]

Journal Article•DOI•

Detecting moving objects, ghosts, and shadows in video streams

[...]

Rita Cucchiara, Costantino Grana, Massimo Piccardi¹, Andrea Prati•Institutions (1)

University of Technology, Sydney¹

01 Oct 2003-IEEE Transactions on Pattern Analysis and Machine Intelligence

TL;DR: A general-purpose method is proposed that combines statistical assumptions with the object-level knowledge of moving objects, apparent objects (ghosts), and shadows acquired in the processing of the previous frames to improve object segmentation and background update.

...read moreread less

Abstract: Background subtraction methods are widely exploited for moving object detection in videos in many applications, such as traffic monitoring, human motion capture, and video surveillance. How to correctly and efficiently model and update the background model and how to deal with shadows are two of the most distinguishing and challenging aspects of such approaches. The article proposes a general-purpose method that combines statistical assumptions with the object-level knowledge of moving objects, apparent objects (ghosts), and shadows acquired in the processing of the previous frames. Pixels belonging to moving objects, ghosts, and shadows are processed differently in order to supply an object-based selective update. The proposed approach exploits color information for both background subtraction and shadow detection to improve object segmentation and background update. The approach proves fast, flexible, and precise in terms of both pixel accuracy and reactivity to background changes.

...read moreread less

1,521 citations

"A new video segmentation method of ..." refers background or methods in this paper

...Thus, for example, in Sakbot (Cucchiara et al., 2003) several conditions have to be checked....
[...]
...Other proposals, however, attack the problem globally, i.e., trying to differentiate and classify moving object blobs and different types of noise blobs (Cucchiara et al., 2003)....
[...]
...Other proposals exist (Cucchiara et al., 2003), but ours consists of a set of entities and relations (see Fig....
[...]
...Nevertheless, Sakbot has the advantage of not depending on the value of an initialisation constant, as indeed occurs in our strategy with the constant Cmax....
[...]
...The Sakbot system (Cucchiara et al., 2003) reports an average frame rate of 9.82 fps, for a sequence with the same dimension frames and with Pentium 4, 1.5 GHz....
[...]