Trajectory Learning for Activity Understanding: Unsupervised, Multilevel, and Long-Term Adaptive Approach
Brendan Morris,Mohan M. Trivedi +1 more
TLDR
A framework for live video analysis in which the behaviors of surveillance subjects are described using a vocabulary learned from recurrent motion patterns, for real-time characterization and prediction of future activities, as well as the detection of abnormalities.Abstract:
Society is rapidly accepting the use of video cameras in many new and varied locations, but effective methods to utilize and manage the massive resulting amounts of visual data are only slowly developing. This paper presents a framework for live video analysis in which the behaviors of surveillance subjects are described using a vocabulary learned from recurrent motion patterns, for real-time characterization and prediction of future activities, as well as the detection of abnormalities. The repetitive nature of object trajectories is utilized to automatically build activity models in a 3-stage hierarchical learning process. Interesting nodes are learned through Gaussian mixture modeling, connecting routes formed through trajectory clustering, and spatio-temporal dynamics of activities probabilistically encoded using hidden Markov models. Activity models are adapted to small temporal variations in an online fashion using maximum likelihood regression and new behaviors are discovered from a periodic retraining for long-term monitoring. Extensive evaluation on various data sets, typically missing from other work, demonstrates the efficacy and generality of the proposed framework for surveillance-based activity analysis.read more
Citations
More filters
Proceedings ArticleDOI
Social LSTM: Human Trajectory Prediction in Crowded Spaces
Alexandre Alahi,Kratarth Goel,Vignesh Ramanathan,Alexandre Robicquet,Li Fei-Fei,Silvio Savarese +5 more
TL;DR: This work proposes an LSTM model which can learn general human movement and predict their future trajectories and outperforms state-of-the-art methods on some of these datasets.
Proceedings ArticleDOI
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks
TL;DR: A recurrent sequence-to-sequence model observes motion histories and predicts future behavior, using a novel pooling mechanism to aggregate information across people, and outperforms prior work in terms of accuracy, variety, collision avoidance, and computational complexity.
Proceedings ArticleDOI
SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints
Amir Sadeghian,Vineet Kosaraju,Ali Sadeghian,Noriaki Hirose,Hamid Rezatofighi,Silvio Savarese +5 more
TL;DR: In this paper, an interpretable framework based on Generative Adversarial Network (GAN) is proposed for path prediction for multiple interacting agents in a scene, which leverages two sources of information, the path history of all the agents in the scene, and the scene context information, using images of the scene.
Journal ArticleDOI
Vessel Pattern Knowledge Discovery from AIS Data: A Framework for Anomaly Detection and Route Prediction
TL;DR: An unsupervised and incremental learning approach to the extraction of maritime movement patterns is presented here to convert from raw data to information supporting decisions, and is a basis for automatically detecting anomalies and projecting current trajectories and patterns into the future.
Posted Content
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks.
TL;DR: In this paper, a recurrent sequence-to-sequence model observes motion histories and predicts future behavior, using a novel pooling mechanism to aggregate information across people, and predicts socially plausible future by training adversarially against a recurrent discriminator, and encourage diverse predictions with a novel variety loss.
References
More filters
Journal ArticleDOI
Maximum likelihood from incomplete data via the EM algorithm
Proceedings Article
On Spectral Clustering: Analysis and an algorithm
TL;DR: A simple spectral clustering algorithm that can be implemented using a few lines of Matlab is presented, and tools from matrix perturbation theory are used to analyze the algorithm, and give conditions under which it can be expected to do well.
Book
Fundamentals of speech recognition
TL;DR: This book presents a meta-modelling framework for speech recognition that automates the very labor-intensive and therefore time-heavy and therefore expensive and expensive process of manually modeling speech.
Proceedings ArticleDOI
Actions as space-time shapes
TL;DR: The method is fast, does not require video alignment and is applicable in many scenarios where the background is known, and the robustness of the method is demonstrated to partial occlusions, non-rigid deformations, significant changes in scale and viewpoint, high irregularities in the performance of an action and low quality video.
Journal ArticleDOI
Unsupervised learning of finite mixture models
TL;DR: The novelty of the approach is that it does not use a model selection criterion to choose one among a set of preestimated candidate models; instead, it seamlessly integrate estimation and model selection in a single algorithm.