From action to activity

This review article surveys extensively the current progresses made toward video-based human activity recognition Three aspects for human activity recognition are addressed including core technology, human activity recognition systems, and applications from low-level to high-level representation In the core technology, three critical processing stages are thoroughly discussed mainly: human object segmentation, feature extraction and representation, activity detection and classification algorithms In the human activity recognition systems, three main types are mentioned, including single person activity recognition, multiple people interaction and crowd behavior, and abnormal activity recognition Finally the domains of applications are discussed in detail, specifically, on surveillance environments, entertainment environments and healthcare systems Our survey, which aims to provide a comprehensive state-of-the-art review of the field, also addresses several challenges associated with these systems and applications Moreover, in this survey, various applications are discussed in great detail, specifically, a survey on the applications in healthcare monitoring systems

https://www.mdpi.com/2073-431X/2/2/88/pdf

A Review on Video-Based Human Activity Recognition

Space-time representation of people based on 3D skeletal data

Different levels of an intelligent video surveillance system (IVVS) are studied in this review.Existing approaches for abnormal behavior recognition relative to each level of an IVVS are extensively reviewed.Challenging datasets for IVVS evaluation are presented.Limitations of the abnormal behavior recognition area are discussed. With the increasing number of surveillance cameras in both indoor and outdoor locations, there is a grown demand for an intelligent system that detects abnormal events. Although human action recognition is a highly reached topic in computer vision, abnormal behavior detection is lately attracting more research attention. Indeed, several systems are proposed in order to ensure human safety. In this paper, we are interested in the study of the two main steps composing a video surveillance system which are the behavior representation and the behavior modeling. Techniques related to feature extraction and description for behavior representation are reviewed. Classification methods and frameworks for behavior modeling are also provided. Moreover, available datasets and metrics for performance evaluation are presented. Finally, examples of existing video surveillance systems used in real world are described.

Abnormal behavior recognition for intelligent video surveillance systems

We study the problem of classifying actions of human subjects using depth movies generated by Kinect or other depth sensors. Representing human body as dynamical skeletons, we study the evolution of their (skeletons’) shapes as trajectories on Kendall’s shape manifold. The action data is typically corrupted by large variability in execution rates within and across subjects and, thus, causing major problems in statistical analyses. To address that issue, we adopt a recently-developed framework of Su et al. [1] , [2] to this problem domain. Here, the variable execution rates correspond to re-parameterizations of trajectories, and one uses a parameterization-invariant metric for aligning, comparing, averaging, and modeling trajectories. This is based on a combination of transported square-root vector fields (TSRVFs) of trajectories and the standard Euclidean norm, that allows computational efficiency. We develop a comprehensive suite of computational tools for this application domain: smoothing and denoising skeleton trajectories using median filtering, up- and down-sampling actions in time domain, simultaneous temporal-registration of multiple actions, and extracting invertible Euclidean representations of actions. Due to invertibility these Euclidean representations allow both discriminative and generative models for statistical analysis. For instance, they can be used in a SVM-based classification of original actions, as demonstrated here using MSR Action-3D, MSR Daily Activity and 3D Action Pairs datasets. Using only the skeletal information, we achieve state-of-the-art classification results on these datasets.

Action Recognition Using Rate-Invariant Analysis of Skeletal Shape Trajectories

We present an effective real-time approach forautomatically estimating 3D human body poses frommonocular video sequences. In this approach, human bodyis automatically detected from video sequence, then imagefeatures such as silhouette, edge and color are extractedand integrated to infer 3D human poses by iterativelyminimizing the cost function defined between 2D featuresderived from the projected 3D model and those extractedfrom video sequence. In addition, 2D locations of head,hands, and feet are tracked to facilitate 3D tracking. Whentracking failure happens, the approach can detect andrecover from failures quickly. Finally, the efficiency androbustness of the proposed approach is shown in two realapplications: human event detection and video gaming.

Real-Time 3D Human Pose Estimation from Monocular View with Applications to Event Detection and Video Gaming

This paper proposes a system to recognize quasi-periodic human actions from monocular video sequences. First, each input video frame is analyzed and estimated to generate the best 3D human model pose which consists of a set of 3D coordinates of specific human joints. Next, these 3D coordinates for each frame are converted into corresponding 3D geometric relational features (GRFs), which describe the geometric relations among body joints of a pose. Finally, we train a cyclic hidden Markov model (CHMM) for each action based on the vector quantized 3D GRFs, and the trained CHMMs are used to classify different quasi-periodic human actions. The experimental results indicate the effectiveness of the proposed system in terms of the view point invariance, the low-dimensional feature vectors, and the encouraging recognition rates.

Quasi-periodic action recognition from monocular videos via 3D human models and cyclic HMMs

This paper proposes a 3D human body pose reconstruction system based on videos captured from any perspective view of a monocular camera. The appearance, color and temporal information extracted from the video frames are effectively combined to accurately track 2D body features. This view invariant system overcomes the challenges of requiring the modeled human to be viewed from a pre-specified angular perspective so as to initialize the 3D body model configuration, as well as to continuously find the best match between the tracked 2D features with the 3D model based on the downhill simplex algorithm. The matching information of 3D poses are also fed back to assist 2D tracking, which eventually provides more reliable 3D tracking performance.

View-invariant 3D human body pose reconstruction using a monocular video camera

Human action recognition is used in areas such as surveillance, entertainment, and healthcare. This paper proposes a system to recognize both single and continuous human actions from monocular video sequences, based on 3D human modeling and cyclic hidden Markov models (CHMMs). First, for each frame in a monocular video sequence, the 3D coordinates of joints belonging to a human object, through actions of multiple cycles, are extracted using 3D human modeling techniques. The 3D coordinates are then converted into a set of geometrical relational features (GRFs) for dimensionality reduction and discrimination increase. For further dimensionality reduction, k-means clustering is applied to the GRFs to generate clustered feature vectors. These vectors are used to train CHMMs separately for different types of actions, based on the Baum-Welch re-estimation algorithm. For recognition of continuous actions that are concatenated from several distinct types of actions, a designed graphical model is used to systematically concatenate different separately trained CHMMs. The experimental results show the effective performance of our proposed system in both single and continuous action recognition problems.

http://www.koreascience.or.kr:80/article/JAKO201468162228621.pdf

Shian-Ru Ke

Papers

A Review on Video-Based Human Activity Recognition

Real-Time 3D Human Pose Estimation from Monocular View with Applications to Event Detection and Video Gaming

Quasi-periodic action recognition from monocular videos via 3D human models and cyclic HMMs

View-invariant 3D human body pose reconstruction using a monocular video camera

Human Action Recognition Based on 3D Human Modeling and Cyclic HMMs