Robust Video Surveillance for Fall Detection Based on Human Shape Deformation
01 May 2011-IEEE Transactions on Circuits and Systems for Video Technology (IEEE)-Vol. 21, Iss: 5, pp 611-622
TL;DR: A new method is proposed to detect falls by analyzing human shape deformation during a video sequence, which gives very good results (as low as 0% error with a multi-camera setup) compared with other common image processing methods.
Abstract: Faced with the growing population of seniors, developed countries need to establish new healthcare systems to ensure the safety of elderly people at home. Computer vision provides a promising solution to analyze personal behavior and detect certain unusual events such as falls. In this paper, a new method is proposed to detect falls by analyzing human shape deformation during a video sequence. A shape matching technique is used to track the person's silhouette along the video sequence. The shape deformation is then quantified from these silhouettes based on shape analysis methods. Finally, falls are detected from normal activities using a Gaussian mixture model. This paper has been conducted on a realistic data set of daily activities and simulated falls, and gives very good results (as low as 0% error with a multi-camera setup) compared with other common image processing methods.
Citations
More filters
TL;DR: A comprehensive survey of different systems for fall detection and their underlying algorithms is given, divided into three main categories: wearable device based, ambience device based and vision based.
777 citations
Cites methods from "Robust Video Surveillance for Fall ..."
...in [59] proposed a classification method for fall detection by analysing human shape deformation....
[...]
...[59]’’ A Gaussian Mixture Model (GMM) classifier is implemented to detect falls....
[...]
TL;DR: An extensive literature review of fall detection systems is presented, including comparisons among various kinds of studies, to serve as a reference for both clinicians and biomedical engineers planning or conducting field investigations.
Abstract: Since falls are a major public health problem among older people, the number of systems aimed at detecting them has increased dramatically over recent years. This work presents an extensive literature review of fall detection systems, including comparisons among various kinds of studies. It aims to serve as a reference for both clinicians and biomedical engineers planning or conducting field investigations. Challenges, issues and trends in fall detection have been identified after the reviewing work. The number of studies using context-aware techniques is still increasing but there is a new trend towards the integration of fall detection into smartphones as well as the use of machine learning methods in the detection algorithm. We have also identified challenges regarding performance under real-life conditions, usability, and user acceptance as well as issues related to power consumption, real-time operations, sensing limitations, privacy and record of real-life falls.
535 citations
Cites background or methods from "Robust Video Surveillance for Fall ..."
...systems [14,15,17,19-23], or from the information provided by the sensors’ observation [28,29]....
[...]
...Model [14], Rule-based Techniques [21], Multi-frame Gaussian Classifier [17], Bayesian Filtering [28], Nearest-neighbour Rule [23,29], Hidden Markov Models [19], Thresholding...
[...]
...For example, video-based systems only consider one or two specific sequences in controlled environments [14,15,17,19-24] and other studies with different types of sensors (pressure [25], infrared [18], etc....
[...]
...However, their operation is limited to those places where the sensors have been previously deployed [14]....
[...]
...Some studies confirm that a fall has occurred by performing inactivity detection in the postfall phase [14,18,24]....
[...]
TL;DR: A method for detecting falls in the homes of older adults using the Microsoft Kinect and a two-stage fall detection system is presented and is compared against five state-of-the-art fall detection algorithms and significantly better results are achieved.
Abstract: A method for detecting falls in the homes of older adults using the Microsoft Kinect and a two-stage fall detection system is presented. The first stage of the detection system characterizes a person's vertical state in individual depth image frames, and then segments on ground events from the vertical state time series obtained by tracking the person over time. The second stage uses an ensemble of decision trees to compute a confidence that a fall preceded on a ground event. Evaluation was conducted in the actual homes of older adults, using a combined nine years of continuous data collected in 13 apartments. The dataset includes 454 falls, 445 falls performed by trained stunt actors and nine naturally occurring resident falls. The extensive data collection allows for characterization of system performance under real-world conditions to a degree that has not been shown in other studies. Cross validation results are included for standing, sitting, and lying down positions, near (within 4 m) versus far fall locations, and occluded versus not occluded fallers. The method is compared against five state-of-the-art fall detection algorithms and significantly better results are achieved.
463 citations
Cites background from "Robust Video Surveillance for Fall ..."
...…of researchers have looked at the use of environmentally mounted sensors for fall detection, such as floor vibration sensors [9], [10], passive infrared sensors [11], acoustic sensors [10], [12], and video-based sensors, including traditional cameras [13]–[18] and depth imaging sensors [19]–[23]....
[...]
...Approaches have ranged from single cameras mounted on the wall [13], [14], to cameras mounted on the ceiling [15], [16], to multiple cameras placed around a room to allow 3-...
[...]
TL;DR: An automated fall detection approach that requires only a low-cost depth camera and a variable-length particle swarm optimization algorithm to optimize the number of hidden neurons, corresponding input weights, and biases of ELM is presented.
Abstract: Falls are one of the major causes leading to injury of elderly people. Using wearable devices for fall detection has a high cost and may cause inconvenience to the daily lives of the elderly. In this paper, we present an automated fall detection approach that requires only a low-cost depth camera. Our approach combines two computer vision techniques-shape-based fall characterization and a learning-based classifier to distinguish falls from other daily actions. Given a fall video clip, we extract curvature scale space (CSS) features of human silhouettes at each frame and represent the action by a bag of CSS words (BoCSS). Then, we utilize the extreme learning machine (ELM) classifier to identify the BoCSS representation of a fall from those of other actions. In order to eliminate the sensitivity of ELM to its hyperparameters, we present a variable-length particle swarm optimization algorithm to optimize the number of hidden neurons, corresponding input weights, and biases of ELM. Using a low-cost Kinect depth camera, we build an action dataset that consists of six types of actions (falling, bending, sitting, squatting, walking, and lying) from ten subjects. Experimenting with the dataset shows that our approach can achieve up to 91.15% sensitivity, 77.14% specificity, and 86.83% accuracy. On a public dataset, our approach performs comparably to state-of-the-art fall detection methods that need multiple cameras.
239 citations
Additional excerpts
...Note that u is length-normalized....
[...]
TL;DR: An enhanced fall detection system is proposed for elderly person monitoring that is based on smart sensors worn on the body and operating through consumer home networks that can achieve a high detection accuracy and a low false positive rate.
Abstract: Various fall-detection solutions have been previously proposed to create a reliable surveillance system for elderly people with high requirements on accuracy, sensitivity and specificity. In this paper, an enhanced fall detection system is proposed for elderly person monitoring that is based on smart sensors worn on the body and operating through consumer home networks. With treble thresholds, accidental falls can be detected in the home healthcare environment. By utilizing information gathered from an accelerometer, cardiotachometer and smart sensors, the impacts of falls can be logged and distinguished from normal daily activities. The proposed system has been deployed in a prototype system as detailed in this paper. From a test group of 30 healthy participants, it was found that the proposed fall detection system can achieve a high detection accuracy of 97.5%, while the sensitivity and specificity are 96.8% and 98.1% respectively. Therefore, this system can reliably be developed and deployed into a consumer product for use as an elderly person monitoring device with high accuracy and a low false positive rate.
234 citations
Cites methods from "Robust Video Surveillance for Fall ..."
...By utilizing information gathered from an accelerometer, cardiotachometer and smart sensors, the impacts of falls can be logged and distinguished from normal daily activities....
[...]
References
More filters
TL;DR: There is a natural uncertainty principle between detection and localization performance, which are the two main goals, and with this principle a single operator shape is derived which is optimal at any scale.
Abstract: This paper describes a computational approach to edge detection. The success of the approach depends on the definition of a comprehensive set of goals for the computation of edge points. These goals must be precise enough to delimit the desired behavior of the detector while making minimal assumptions about the form of the solution. We define detection and localization criteria for a class of edges, and present mathematical forms for these criteria as functionals on the operator impulse response. A third criterion is then added to ensure that the detector has only one response to a single edge. We use the criteria in numerical optimization to derive detectors for several common image features, including step edges. On specializing the analysis to step edges, we find that there is a natural uncertainty principle between detection and localization performance, which are the two main goals. With this principle we derive a single operator shape which is optimal at any scale. The optimal detector has a simple approximate implementation in which edges are marked at maxima in gradient magnitude of a Gaussian-smoothed image. We extend this simple detector using operators of several widths to cope with different signal-to-noise ratios in the image. We present a general method, called feature synthesis, for the fine-to-coarse integration of information from operators at different scales. Finally we show that step edge detector performance improves considerably as the operator point spread function is extended along the edge.
28,073 citations
"Robust Video Surveillance for Fall ..." refers methods in this paper
...In addition to the foreground silhouette contour, we chose to extract edge points inside the silhouette using a Canny edge detector [25] to provide additional shape information....
[...]
TL;DR: This paper has always been one of my favorite children, combining as it does elements of the duality of linear programming and combinatorial tools from graph theory, and it may be of some interest to tell the story of its origin this article.
Abstract: This paper has always been one of my favorite “children,” combining as it does elements of the duality of linear programming and combinatorial tools from graph theory. It may be of some interest to tell the story of its origin.
11,096 citations
TL;DR: This paper presents work on computing shape models that are computationally fast and invariant basic transformations like translation, scaling and rotation, and proposes shape detection using a feature called shape context, which is descriptive of the shape of the object.
Abstract: We present a novel approach to measuring similarity between shapes and exploit it for object recognition. In our framework, the measurement of similarity is preceded by: (1) solving for correspondences between points on the two shapes; (2) using the correspondences to estimate an aligning transform. In order to solve the correspondence problem, we attach a descriptor, the shape context, to each point. The shape context at a reference point captures the distribution of the remaining points relative to it, thus offering a globally discriminative characterization. Corresponding points on two similar shapes will have similar shape contexts, enabling us to solve for correspondences as an optimal assignment problem. Given the point correspondences, we estimate the transformation that best aligns the two shapes; regularized thin-plate splines provide a flexible class of transformation maps for this purpose. The dissimilarity between the two shapes is computed as a sum of matching errors between corresponding points, together with a term measuring the magnitude of the aligning transform. We treat recognition in a nearest-neighbor classification framework as the problem of finding the stored prototype shape that is maximally similar to that in the image. Results are presented for silhouettes, trademarks, handwritten digits, and the COIL data set.
6,693 citations
"Robust Video Surveillance for Fall ..." refers methods in this paper
...The moving edge points extracted from two consecutive images are then matched using shape context [20]....
[...]
...The log-polar histogram has 5 bins for log r and 12 bins for θ as proposed by the authors in [20]....
[...]
...The authors in [20] proposed to use the Hungarian algorithm [27] for bipartite matching....
[...]
...a) the mean matching cost obtained from the shape context matching which has been used for shape recognition [20]; b) the full Procrustes distance [21] which is a wellknown tool for shape analysis, and which has been widely used to compare shapes in biology and medicine....
[...]
...For this purpose, we use the shape context matching method [20] described in Section VII....
[...]
TL;DR: A survey of contemporary techniques for outlier detection is introduced and their respective motivations are identified and distinguish their advantages and disadvantages in a comparative review.
Abstract: Outlier detection has been used for centuries to detect and, where appropriate, remove anomalous observations from data. Outliers arise due to mechanical faults, changes in system behaviour, fraudulent behaviour, human error, instrument error or simply through natural deviations in populations. Their detection can identify system faults and fraud before they escalate with potentially catastrophic consequences. It can identify errors and remove their contaminating effect on the data set and as such to purify the data for processing. The original outlier detection methods were arbitrary but now, principled and systematic techniques are used, drawn from the full gamut of Computer Science and Statistics. In this paper, we introduce a survey of contemporary techniques for outlier detection. We identify their respective motivations and distinguish their advantages and disadvantages in a comparative review.
3,235 citations
Additional excerpts
...A survey of novelty detection methods can be found in the article [31]....
[...]
01 Jan 2010
TL;DR: This paper has always been one of my favorite “children,” combining as it does elements of the duality of linear programming and combinatorial tools from graph theory.
Abstract: This paper has always been one of my favorite “children,” combining as it does elements of the duality of linear programming and combinatorial tools from graph theory. It may be of some interest to tell the story of its origin.
3,108 citations
"Robust Video Surveillance for Fall ..." refers methods in this paper
...The results obtained using the Hungarian algorithm [27] for bipartite matching with 20% dummy points are also shown in Table II(b) for comparison with our method using only the best matching points....
[...]
...The authors in [20] proposed to use the Hungarian algorithm [27] for bipartite matching....
[...]