http://cvssp-data.eps.surrey.ac.uk/Personal/AdrianHilton/Publications_files/survey_techrep.pdf

A survey of advances in vision-based human motion capture and analysis

Modeling human behaviors and activity patterns for recognition or detection of special event has attracted significant research interest in recent years. Diverse methods that are abound for building intelligent vision systems aimed at scene understanding and making correct semantic inference from the observed dynamics of moving targets. Most applications are in surveillance, video content retrieval, and human-computer interfaces. This paper presents not only an update extending previous related surveys, but also a focus on contextual abnormal human behavior detection especially in video surveillance applications. The main purpose of this survey is to extensively identify existing methods and characterize the literature in a manner that brings key challenges to attention.

Video-Based Abnormal Human Behavior Recognition—A Review

3D Human pose estimation

Visual processing of biological motion (BM) produced by living organisms is of immense value for successful daily-life activities and, in particular, for adaptive social behavior and nonverbal communication. Investigation of BM perception in neurodevelopmental disorders related to autism, preterm birth, and genetic conditions substantially contributes to our understanding of the neural mechanisms underpinning the extraordinary tuning to BM. The most prominent research outcome is that patients with daily-life deficits in social cognition are also compromised on visual body motion processing. This raises the question of whether performance on body motion perception tasks may serve a hallmark of social cognition. Overall, the findings highlight the role of structural and functional brain connectivity for proper functioning of the neural circuitry involved in BM processing and visual social cognition that share topographically and dynamically overlapping neural networks.

/pdf/biological-motion-processing-as-a-hallmark-of-social-25j5tnf8vl.pdf

Biological Motion Processing as a Hallmark of Social Cognition

https://mediatum.ub.tum.de/doc/1285816/file.pdf

Human skeleton tracking from depth data using geodesic distances and optical flow

We present a method to simultaneously estimate 3D body pose and action categories from monocular video sequences. Our approach learns a generative model of the relationship of body pose and image appearance using a sparse kernel regressor. Body poses are modelled on a low-dimensional manifold obtained by Locally Linear Embedding dimensionality reduction. In addition, we learn a prior model of likely body poses and a dynamical model in this pose manifold. Sparse kernel regressors capture the nonlinearities of this mapping efficiently. Within a Recursive Bayesian Sampling framework, the potentially multimodal posterior probability distributions can then be inferred. An activity-switching mechanism based on learned transfer functions allows for inference of the performed activity class, along with the estimation of body pose and 2D image location of the subject. Using a rough foreground segmentation, we compare Binary PCA and distance transforms to encode the appearance. As a postprocessing step, the globally optimal trajectory through the entire sequence is estimated, yielding a single pose estimate per frame that is consistent throughout the sequence. We evaluate the algorithm on challenging sequences with subjects that are alternating between running and walking movements. Our experiments show how the dynamical model helps to track through poorly segmented low-resolution image sequences where tracking otherwise fails, while at the same time reliably classifying the activity type.

/pdf/learning-generative-models-for-multi-activity-body-pose-36kzzr7a88.pdf

Learning Generative Models for Multi-Activity Body Pose Estimation

A vast literature exists on human biological motion perception in impoverished displays, e.g., point-light walkers. Less is known about the perception of impoverished biological motion displays in macaques. We trained 3 macaques in the discrimination of facing direction (left versus right) and forward versus backward walking using motion-capture-based locomotion displays (treadmill walking) in which the body features were represented by cylinder-like primitives. The displays did not contain translatory motion. Discriminating forward versus backward locomotion requires motion information while the facing-direction/view task can be solved using motion and/or form. All monkeys required lengthy training to learn the forward-backward task, while the view task was learned more quickly. Once acquired, the discriminations were specific to walking and stimulus format but generalized across actors. Although the view task could be solved using form cues, there was a small impact of motion. Performance in the forward-backward task was highly susceptible to degradations of spatiotemporal stimulus coherence and motion information. These results indicate that rhesus monkeys require extensive training in order to use the intrinsic motion cues related to forward versus backward locomotion and imply that extrapolation of observations concerning human perception of impoverished biological motion displays onto monkey perception needs to be made cautiously.

Discrimination of locomotion direction in impoverished displays of walkers by macaque monkeys

We present an approach for unusual event detection, based on a tree of trackers. At lower levels, the trackers are trained on broad classes of targets. At higher levels, they aim at more specific targets. For instance, at the root, a general blob tracker could operate which may track any object. The next level could already use information about human appearance to better track people. A further level could go after specific types of actions like walking, running, or sitting. Yet another level up, several walking trackers can be tuned to the gait of a particular person each. Thus, at each layer, one or more families of more specific trackers are available. As long as the target behaves according to expectations, a member of a higher up such family will be better tuned to the data than its parent tracker at a lower level. Typically, a better informed tracker performs more robustly. But in cases where unusual events occur and the normal assumptions about the world no longer hold, they loose their reliability. In such cases, a less informed tracker, not relying on what has now become false information, has a good chance of performing better. Such performance inversion signals an unusual event. Inversions between levels higher up represent deviations that are semantically more subtle than inversions lower down: for instance an unknown intruder entering a house rather than seeing a non-human target.

Tracker trees for unusual event detection

We present a method to simultaneously estimate 3d body pose and action categories from monocular video sequences. Our approach learns a low-dimensional embedding of the pose manifolds using Locally Linear Embedding (LLE), as well as the statistical relationship between body poses and their image appearance. In addition, the dynamics in these pose manifolds are modelled. Sparse kernel regressors capture the nonlinearities of these mappings efficiently. Body poses are inferred by a recursive Bayesian sampling algorithm with an activity-switching mechanism based on learned transfer functions. Using a rough foreground segmentation, we compare Binary PCA and distance transforms to encode the appearance. As a postprocessing step, the globally optimal trajectory through the entire sequence is estimated, yielding a single pose estimate per frame that is consistent throughout the sequence. We evaluate the algorithm on challenging sequences with subjects that are alternating between running and walking movements. Our experiments show how the dynamical model helps to track through poorly segmented low-resolution image sequences where tracking otherwise fails, while at the same time reliably classifying the activity type.

Multi-activity tracking in LLE body pose space

We consider the problem of monocular 3d body pose tracking from video sequences. This task is inherently ambiguous. We propose to learn a generative model of the relationship of body pose and image appearance using a sparse kernel regressor. Within a particle filtering framework, the potentially multimodal posterior probability distributions can then be inferred. The 2d bounding box location of the person in the image is estimated along with its body pose. Body poses are modelled on a low-dimensional manifold, obtained by LLE dimensionality reduction. In addition to the appearance model, we learn a prior model of likely body poses and a nonlinear dynamical model, making both pose and bounding box estimation more robust. The approach is evaluated on a number of challenging video sequences, showing the ability of the approach to deal with low-resolution images and noise.

Tobias Jaeggli

Papers

Learning Generative Models for Multi-Activity Body Pose Estimation

Discrimination of locomotion direction in impoverished displays of walkers by macaque monkeys

Tracker trees for unusual event detection

Multi-activity tracking in LLE body pose space

Learning generative models for monocular body pose estimation