Recognising behaviours of multiple people with hierarchical probabilistic model and statistical data association
01 Jan 2006-pp 1239-1248
TL;DR: The main contributions of this paper lie in the integration of multiple HHMMs for recognising high-level behaviours of multiple people and the construction of the Rao-Blackwellised particle filters (RBPF) for approximate inference.
Abstract: Recognising behaviours of multiple people, especially high-level behaviours, is an important task in surveillance systems. When the reliable assignment of people to the set of observations is unavailable, this task becomes complicated. To solve this task, we present an approach, in which the hierarchical hidden Markov model (HHMM) is used for modeling the behaviour of each person and the joint probabilistic data association filters (JPDAF) is applied for data association. The main contributions of this paper lie in the integration of multiple HHMMs for recognising high-level behaviours of multiple people and the construction of the Rao-Blackwellised particle filters (RBPF) for approximate inference. Preliminary experimental results in a real environment show the robustness of our integrated method in behaviour recognition and its advantage over the use of Kalman filter in tracking people.
Citations
More filters
TL;DR: The emergence of `ambient-assisted living’ (AAL) tools for older adults based on ambient intelligence paradigm is summarized and the state-of-the-art AAL technologies, tools, and techniques are summarized.
Abstract: In recent years, we have witnessed a rapid surge in assisted living technologies due to a rapidly aging society. The aging population, the increasing cost of formal health care, the caregiver burden, and the importance that the individuals place on living independently, all motivate development of innovative-assisted living technologies for safe and independent aging. In this survey, we will summarize the emergence of `ambient-assisted living” (AAL) tools for older adults based on ambient intelligence paradigm. We will summarize the state-of-the-art AAL technologies, tools, and techniques, and we will look at current and future challenges.
1,000 citations
Cites background from "Recognising behaviours of multiple ..."
...hierarchal HMM for providing hierarchal definitions of activities [79], [80], hidden semi Markov model for modeling activity...
[...]
01 Nov 2012
TL;DR: A comprehensive survey to examine the development and current status of various aspects of sensor-based activity recognition, making a primary distinction in this paper between data-driven and knowledge-driven approaches.
Abstract: Research on sensor-based activity recognition has, recently, made significant progress and is attracting growing attention in a number of disciplines and application domains. However, there is a lack of high-level overview on this topic that can inform related communities of the research state of the art. In this paper, we present a comprehensive survey to examine the development and current status of various aspects of sensor-based activity recognition. We first discuss the general rationale and distinctions of vision-based and sensor-based activity recognition. Then, we review the major approaches and methods associated with sensor-based activity monitoring, modeling, and recognition from which strengths and weaknesses of those approaches are highlighted. We make a primary distinction in this paper between data-driven and knowledge-driven approaches, and use this distinction to structure our survey. We also discuss some promising directions for future research.
944 citations
Cites methods from "Recognising behaviours of multiple ..."
...These include the dynamically multilinked HMM model [137], the hierarchical HMM model [138], the coupled...
[...]
TL;DR: This paper develops a multi-modal, wearable sensor platform to collect sensor data for multiple users, and study two temporal probabilistic models-Coupled Hidden Markov Model (CHMM) and Factorial Conditional Random Field (FCRF) to model interacting processes in a sensor-based, multi-user scenario.
Abstract: The advances of wearable sensors and wireless networks offer many opportunities to recognize human activities from sensor readings in pervasive computing. Existing work so far focuses mainly on recognizing activities of a single user in a home environment. However, there are typically multiple inhabitants in a real home and they often perform activities together. In this paper, we investigate the problem of recognizing multi-user activities using wearable sensors in a home setting. We develop a multi-modal, wearable sensor platform to collect sensor data for multiple users, and study two temporal probabilistic models-Coupled Hidden Markov Model (CHMM) and Factorial Conditional Random Field (FCRF)-to model interacting processes in a sensor-based, multi-user scenario. We conduct a real-world trace collection done by two subjects over two weeks, and evaluate these two models through our experimental studies. Our experimental results show that we achieve an accuracy of 96.41% with CHMM and an accuracy of 87.93% with FCRF, respectively, for recognizing multi-user activities.
149 citations
TL;DR: An overview of existing approaches and current practices for activity recognition in multioccupant smart homes is provided, which presents the latest developments and highlights the open issues in this field.
Abstract: Human activity recognition in ambient intelligent environments like homes, offices, and classrooms has been the center of a lot of research for many years now. The aim is to recognize the sequence of actions by a specific person using sensor readings. Most of the research has been devoted to activity recognition of single occupants in the environment. However, living environments are usually inhabited by more than one person and possibly with pets. Hence, human activity recognition in the context of multioccupancy is more general, but also more challenging. The difficulty comes from mainly two aspects: resident identification, known as data association, and diversity of human activities. The present survey article provides an overview of existing approaches and current practices for activity recognition in multioccupant smart homes. It presents the latest developments and highlights the open issues in this field.
118 citations
Cites background or methods from "Recognising behaviours of multiple ..."
...A more mature study in this area has been conducted by the computer vision community using normal cameras [Nguyen et al. 2006; McCowan et al. 2005; Du et al. 2006, 2007; Natarajan and Nevatia 2007]....
[...]
...There are two main technologies used to recognize human activities in smart environments including homes: computer vision [Nguyen et al. 2006; McCowan et al. 2005; Du et al. 2006, 2007; Natarajan and Nevatia 2007] and pervasive sensing [Prossegger and Bouchachia 2014; Crandall and Cook 2008a,…...
[...]
...There are two main technologies used to recognize human activities in smart environments including homes: computer vision [Nguyen et al. 2006; McCowan et al. 2005; Du et al. 2006, 2007; Natarajan and Nevatia 2007] and pervasive sensing [Prossegger and Bouchachia 2014; Crandall and Cook 2008a, 2008b, 2010; Hsu et al....
[...]
TL;DR: This paper presents a novel Knowledge-driven approach for Concurrent Activity Recognition (KCAR), which exploits the Pyramid Match Kernel, with a strength in approximate matching on hierarchical concepts, to recognise activities of varying grained constraints from a potentially noisy sensor sequence.
Abstract: Recognising human activities from sensors embedded in an environment or worn on bodies is an important and challenging research topic in pervasive computing. Existing work on activity recognition is mainly concerned with identifying single user sequential activities from well-scripted or pre-segmented sequences of sensor events. However a real-world environment often contains multiple users, with each performing activities simultaneously, in their own way and with no explicit instructions to follow. Recognising multi-user concurrent activities is challenging, but essential for designing applications for real environments. This paper presents a novel Knowledge-driven approach for Concurrent Activity Recognition (KCAR). Within KCAR, we explore the semantics underlying each sensor event and use semantic dissimilarity to segment a continuous sensor sequence into fragments, each of which corresponds to one ongoing activity. We exploit the Pyramid Match Kernel, with a strength in approximate matching on hierarchical concepts, to recognise activities of varying grained constraints from a potentially noisy sensor sequence. We conduct an empirical evaluation on a large-scale real-world data set that is collected over one year and consists of 2.8 millions of sensor events. Our results demonstrate that KCAR achieves an average recognition accuracy of 91%.
91 citations
References
More filters
Book•
01 Jan 1988
4,098 citations
"Recognising behaviours of multiple ..." refers methods in this paper
...An efficient method to resolve this problem is the joint probabilistic data association filter (JPDAF) [2, 5]....
[...]
...3 Compare the HHMM-JPDAF with the Kalman filter We use the multiple Kalman filters and the JPDAF to track people in a similar manner as in [2]....
[...]
Journal Article•
3,014 citations
01 Jan 2001
TL;DR: In this article, Rao-Blackwellised particle filters (RBPFs) were proposed to increase the efficiency of particle filtering, using a technique known as Rao-blackwellisation.
Abstract: Particle filters (PFs) are powerful sampling-based inference/learning algorithms for dynamic Bayesian networks (DBNs). They allow us to treat, in a principled way, any type of probability distribution, nonlinearity and non-stationarity. They have appeared in several fields under such names as "condensation", "sequential Monte Carlo" and "survival of the fittest". In this paper, we show how we can exploit the structure of the DBN to increase the efficiency of particle filtering, using a technique known as Rao-Blackwellisation. Essentially, this samples some of the variables, and marginalizes out the rest exactly, using the Kalman filter, HMM filter, junction tree algorithm, or any other finite dimensional optimal filter. We show that Rao-Blackwellised particle filters (RBPFs) lead to more accurate estimates than standard PFs. We demonstrate RBPFs on two problems, namely non-stationary online regression with radial basis function networks and robot localization and map building. We also discuss other potential application areas and provide references to some finite dimensional optimal filters.
1,231 citations
30 Jun 2000
TL;DR: In this paper, Rao-Blackwellised particle filters (RBPFs) were proposed to increase the efficiency of particle filtering, using a technique known as Rao-blackwellisation.
Abstract: Particle filters (PFs) are powerful sampling-based inference/learning algorithms for dynamic Bayesian networks (DBNs). They allow us to treat, in a principled way, any type of probability distribution, nonlinearity and non-stationarity. They have appeared in several fields under such names as "condensation", "sequential Monte Carlo" and "survival of the fittest". In this paper, we show how we can exploit the structure of the DBN to increase the efficiency of particle filtering, using a technique known as Rao-Blackwellisation. Essentially, this samples some of the variables, and marginalizes out the rest exactly, using the Kalman filter, HMM filter, junction tree algorithm, or any other finite dimensional optimal filter. We show that Rao-Blackwellised particle filters (RBPFs) lead to more accurate estimates than standard PFs. We demonstrate RBPFs on two problems, namely non-stationary online regression with radial basis function networks and robot localization and map building. We also discuss other potential application areas and provide references to some finite dimensional optimal filters.
1,141 citations
TL;DR: This work introduces, analyzes and demonstrates a recursive hierarchical generalization of the widely used hidden Markov models, which is motivated by the complex multi-scale structure which appears in many natural sequences, particularly in language, handwriting and speech.
Abstract: We introduce, analyze and demonstrate a recursive hierarchical generalization of the widely used hidden Markov models, which we name Hierarchical Hidden Markov Models (HHMM) Our model is motivated by the complex multi-scale structure which appears in many natural sequences, particularly in language, handwriting and speech We seek a systematic unsupervised approach to the modeling of such structures By extending the standard Baum-Welch (forward-backward) algorithm, we derive an efficient procedure for estimating the model parameters from unlabeled data We then use the trained model for automatic hierarchical parsing of observation sequences We describe two applications of our model and its parameter estimation procedure In the first application we show how to construct hierarchical models of natural English text In these models different levels of the hierarchy correspond to structures on different length scales in the text In the second application we demonstrate how HHMMs can be used to automatically identify repeated strokes that represent combination of letters in cursive handwriting
1,050 citations
"Recognising behaviours of multiple ..." refers background or methods in this paper
...Hierarchical probabilistic models such as the stochastic context free grammar (SCFG) [9], the abstract hidden Markov model (AHMM) [4], and the hierarchical hidden Markov model (HHMM) [3, 7] have been used recently to model the high-level behaviour and deal with uncertainty....
[...]
...The hierarchical hidden Markov model (HHMM) [3, 7] is an extension of the hidden Markov model (HMM) to include a hierarchy of hidden states....
[...]
...We use the HHMM [3, 7] − an extension of the hidden Markov model − in our framework because there are efficient learning and inference algorithms in this hierarchical model....
[...]