A Scalable Hybrid Activity Recognition Approach for Intelligent Environments
31 Jul 2018-IEEE Access (Institute of Electrical and Electronics Engineers (IEEE))-Vol. 6, pp 41745-41759
TL;DR: This paper presents a novel activity recognition system, based on a combination of unsupervised learning techniques and knowledge-based activity models, which has been tested on three real data sets and obtained performance comparable to supervised learning techniques.
Abstract: Human activity recognition is a key technology for ICT-based (infomation and communication technologies) assistive applications. The most successful activity recognition systems for intelligent environments in terms of performance rely on supervised learning techniques. However, those techniques demand large labelled data sets for specific sensor deployments and monitored person. Such requirements make supervised learning techniques not to scale well to real world deployments, where different sensor infrastructures may be used to monitor different users. In this paper, we present a novel activity recognition system, based on a combination of unsupervised learning techniques and knowledge-based activity models. First, we use a domain-specific data mining algorithm previously developed by Cook et al. to extract the most frequent action sequences executed by a person. Second, we insert knowledge-based activity models in a novel matching algorithm with the aim of inferring what activities are being performed in a given action sequence. The approach results on a scalable activity recognition system, which has been tested on three real data sets. The obtained performance is comparable to supervised learning techniques.
Citations
More filters
TL;DR: The paradigm of Smart Cities arises as a response to the goal of creating the city of the future, where the well-being and rights of their citizens are guaranteed, industry and urban planning is assessed from an environmental and sustainable viewpoint.
Abstract: The introduction of the Information and Communication Technologies throughout the last decades has created a trend of providing daily objects with smartness, aiming to make human life more comforta...
154 citations
TL;DR: The experiments show that the employment of personalization models improves, on average, the accuracy of machine learning algorithms, thus confirming the soundness of the approach and paving the way for future investigations on this topic.
Abstract: Recently, a significant amount of literature concerning machine learning techniques has focused on automatic recognition of activities performed by people. The main reason for this considerable interest is the increasing availability of devices able to acquire signals which, if properly processed, can provide information about human activities of daily living (ADL). The recognition of human activities is generally performed by machine learning techniques that process signals from wearable sensors and/or cameras appropriately arranged in the environment. Whatever the type of sensor, activities performed by human beings have a strong subjective characteristic that is related to different factors, such as age, gender, weight, height, physical abilities, and lifestyle. Personalization models have been studied to take into account these subjective factors and it has been demonstrated that using these models, the accuracy of machine learning algorithms can be improved. In this work we focus on the recognition of human activities using signals acquired by the accelerometer embedded in a smartphone. The contributions of this research are mainly three. A first contribution is the definition of a clear validation model that takes into account the problem of personalization and which thus makes it possible to objectively evaluate the performances of machine learning algorithms. A second contribution is the evaluation, on three different public datasets, of a personalization model which considers two aspects: the similarity between people related to physical aspects (age, weight, and height) and similarity related to intrinsic characteristics of the signals produced by these people when performing activities. A third and last contribution is the development of a personalization model that considers both the physical and signal similarities. The experiments show that the employment of personalization models improves, on average, the accuracy, thus confirming the soundness of the approach and paving the way for future investigations on this topic.
85 citations
Cites background or methods from "A Scalable Hybrid Activity Recognit..."
...[21] G. Azkune and A. Almeida, ‘‘A scalable hybrid activity recognition approach for intelligent environments,’’ IEEE Access, vol. 6, pp. 41745–41759, 2018....
[...]
...Examples of hybrid approaches are those from Azkune and Almeida [21], Stevenson and Dobson [24], and Civitarese et al. [25]....
[...]
...Examples of hybrid approaches are those from Azkune and Almeida [21], Stevenson and Dobson [24], and Civitarese et al....
[...]
...The prior knowledge may include for example, the implicit relationships between activities, the related temporal and spatial context, and the entities involved (objects and people) [21]....
[...]
TL;DR: In this article , a detailed narration on the three pillars of human activity recognition (HAR) is presented covering the period from 2011 to 2021, and the review presents the recommendations for an improved HAR design, its reliability, and stability.
Abstract: Human activity recognition (HAR) has multifaceted applications due to its worldly usage of acquisition devices such as smartphones, video cameras, and its ability to capture human activity data. While electronic devices and their applications are steadily growing, the advances in Artificial intelligence (AI) have revolutionized the ability to extract deep hidden information for accurate detection and its interpretation. This yields a better understanding of rapidly growing acquisition devices, AI, and applications, the three pillars of HAR under one roof. There are many review articles published on the general characteristics of HAR, a few have compared all the HAR devices at the same time, and few have explored the impact of evolving AI architecture. In our proposed review, a detailed narration on the three pillars of HAR is presented covering the period from 2011 to 2021. Further, the review presents the recommendations for an improved HAR design, its reliability, and stability. Five major findings were: (1) HAR constitutes three major pillars such as devices, AI and applications; (2) HAR has dominated the healthcare industry; (3) Hybrid AI models are in their infancy stage and needs considerable work for providing the stable and reliable design. Further, these trained models need solid prediction, high accuracy, generalization, and finally, meeting the objectives of the applications without bias; (4) little work was observed in abnormality detection during actions; and (5) almost no work has been done in forecasting actions. We conclude that: (a) HAR industry will evolve in terms of the three pillars of electronic devices, applications and the type of AI. (b) AI will provide a powerful impetus to the HAR industry in future.
65 citations
TL;DR: A layered architecture of SH that combines ontology and MA technologies is designed to automatically acquire semantic knowledge, and support heterogeneity and interoperability services, and results are provided to show the feasibility, effectiveness, and robustness of this proposal.
Abstract: Smart home (SH) as an emerging paradigm for alleviating the overstretched healthcare resources, and enhancing the quality of life has received increasing attention. While the remarkable progress has been made for the development of SH, it still suffers from a number of issues (e.g., device heterogeneity, composite activities recognition, and providing appropriate services). To address these issues, this paper proposes a knowledge-based approach for multiagent (MA) collaboration. Specifically, a layered architecture of SH that combines ontology and MA technologies is designed to automatically acquire semantic knowledge, and support heterogeneity and interoperability services. In such architecture, a generic inference algorithm is presented based on unordered actions and temporal property of activity for inferring both continuous composite activity and personalized service in real time. Then a novel idea is introduced for agent to learn the knowledge of human activity (HA) autonomously and translate into itself knowledge, the purpose of which is to guide agent for performing services in a way that is compatible with HA. The proposed schemes have been implemented in an SH, and evaluated through extensive experiments. The results are provided to show the feasibility, effectiveness, and robustness of our proposal.
56 citations
Cites background from "A Scalable Hybrid Activity Recognit..."
...2) Different from other similar knowledge approaches stated in [5] and [17], the more complicated scenar-...
[...]
...Azkune and Almeida [17] proposed a hybrid approach to...
[...]
...the single-resident single-activity [5], single-resident multiactivity [17], and multiresident multiactivity scenarios....
[...]
...Azkune and Almeida [17] proposed a hybrid approach to infer the single-resident multiactivity scenarios....
[...]
TL;DR: A wide range of systems developed to improve energy efficiency through human behaviour change is analysed and the detected research gaps are listed to guide future research when aiming to raise the awareness of individuals through Information and Communication Technologies.
Abstract: Energy-efficiency related research has reached a growing interest in recent years due to the imminent scarcity of non-renewable resources in our environment and the impending impacts their usage have on our environment. Thus, facing the reduction of energy waste and management has become a pivotal issue in our society. To cope with energy inefficiency, the scientific research community has identified the promotion of people's behaviour change as a critical field to foster environmental sustainability. However, the body of literature shows a lack of systematic methods and processes to reach a common ground when designing technology for promoting sustainable behaviour change. Therefore, this paper contributes with a thorough review and analysis of state of the art. Firstly, theoretical works related to behaviour change are collected and studied to clarify their main concepts and theories. Secondly, the different technologies, processes, methods and techniques applied in the field are reviewed to find diverse strategies in the application of the previously explained theoretical domains. Moreover, a wide range of systems developed to improve energy efficiency through human behaviour change is analysed (from augmented objects to the Internet of Things, digital applications or websites). Finally, the detected research gaps are listed to guide future research when aiming to raise the awareness of individuals through Information and Communication Technologies.
20 citations
Cites background from "A Scalable Hybrid Activity Recognit..."
...[39] G. Azkune and A. Almeida, ‘‘A scalable hybrid activity recognition approach for intelligent environments,’’ IEEE Access, vol. 6, pp. 41745–41759, 2018....
[...]
...[29] A. Almeida and D. López-de-Ipiña, ‘‘Assessing ambiguity of context data in intelligent environments: Towards a more reliable context managing system,’’ Sensors, vol. 12, no. 4, pp. 4934–4951, 2012....
[...]
...Moreover, Almeida et al. [32] represent user actions using Word2Vec embeddings and then apply multi-scale convolutional neural networks (CNN) to predict the user behaviour based on the previously executed actions....
[...]
...Almeida & Azkune [21] extended and formalised this model, providing definitions of each level and distinguishing two types of behaviours, intra-activity behaviours and inter-activity behaviours (Figure 2)....
[...]
...[32] A. Almeida, G. Azkune, and A. Bilbao, ‘‘Embedding-level attention and multi-scale convolutional neural networks for behaviour modelling,’’ in Proc....
[...]
References
More filters
IBM1
TL;DR: Three algorithms are presented to solve the problem of mining sequential patterns over databases of customer transactions, and empirically evaluating their performance using synthetic data shows that two of them have comparable performance.
Abstract: We are given a large database of customer transactions, where each transaction consists of customer-id, transaction time, and the items bought in the transaction. We introduce the problem of mining sequential patterns over such databases. We present three algorithms to solve this problem, and empirically evaluate their performance using synthetic data. Two of the proposed algorithms, AprioriSome and AprioriAll, have comparable performance, albeit AprioriSome performs a little better when the minimum number of customers that must support a sequential pattern is low. Scale-up experiments show that both AprioriSome and AprioriAll scale linearly with the number of customer transactions. They also have excellent scale-up properties with respect to the number of transactions per customer and the number of items in a transaction. >
5,663 citations
Additional excerpts
...2 This approach builds on previous research on pattern discovery, including methods for mining frequent sequences [38], [39], mining frequent patterns using regular expressions [26], constraint-based mining [40], mining frequent temporal relationships [41], and frequentperiodic pattern mining [24]....
[...]
TL;DR: The goal of this article is to review the state-of-the-art tracking methods, classify them into different categories, and identify new trends to discuss the important issues related to tracking including the use of appropriate image features, selection of motion models, and detection of objects.
Abstract: The goal of this article is to review the state-of-the-art tracking methods, classify them into different categories, and identify new trends. Object tracking, in general, is a challenging problem. Difficulties in tracking objects can arise due to abrupt object motion, changing appearance patterns of both the object and the scene, nonrigid object structures, object-to-object and object-to-scene occlusions, and camera motion. Tracking is usually performed in the context of higher-level applications that require the location and/or shape of the object in every frame. Typically, assumptions are made to constrain the tracking problem in the context of a particular application. In this survey, we categorize the tracking methods on the basis of the object and motion representations used, provide detailed descriptions of representative methods in each category, and examine their pros and cons. Moreover, we discuss the important issues related to tracking including the use of appropriate image features, selection of motion models, and detection of objects.
5,318 citations
"A Scalable Hybrid Activity Recognit..." refers background in this paper
...When human AR is targeted in intelligent environments, sensor-based AR is the most used solution [2], since vision-based approaches tend to generate privacy concerns among the users [6]....
[...]
21 Apr 2004
TL;DR: This is the first work to investigate performance of recognition algorithms with multiple, wire-free accelerometers on 20 activities using datasets annotated by the subjects themselves, and suggests that multiple accelerometers aid in recognition.
Abstract: In this work, algorithms are developed and evaluated to de- tect physical activities from data acquired using five small biaxial ac- celerometers worn simultaneously on different parts of the body. Ac- celeration data was collected from 20 subjects without researcher su- pervision or observation. Subjects were asked to perform a sequence of everyday tasks but not told specifically where or how to do them. Mean, energy, frequency-domain entropy, and correlation of acceleration data was calculated and several classifiers using these features were tested. De- cision tree classifiers showed the best performance recognizing everyday activities with an overall accuracy rate of 84%. The results show that although some activities are recognized well with subject-independent training data, others appear to require subject-specific training data. The results suggest that multiple accelerometers aid in recognition because conjunctions in acceleration feature values can effectively discriminate many activities. With just two biaxial accelerometers - thigh and wrist - the recognition performance dropped only slightly. This is the first work to investigate performance of recognition algorithms with multiple, wire-free accelerometers on 20 activities using datasets annotated by the subjects themselves.
3,223 citations
"A Scalable Hybrid Activity Recognit..." refers methods in this paper
...The learning techniques used in the literature are broad, going from simple Naive Bayes classifiers [7]–[12] to Hidden Markov Models [13]–[15], Dynamic Bayesian Networks [16], [17], Support VectorMachines [18], online (or incremental) classifiers [19] and dictionaries of patterns [20]....
[...]
TL;DR: This survey reviews recent trends in video-based human capture and analysis, as well as discussing open problems for future research to achieve automatic visual analysis of human movement.
2,738 citations
"A Scalable Hybrid Activity Recognit..." refers methods in this paper
...The learning techniques used in the literature are broad, going from simple Naive Bayes classifiers [7]–[12] to Hidden Markov Models [13]–[15], Dynamic Bayesian Networks [16], [17], Support VectorMachines [18], online (or incremental) classifiers [19] and dictionaries of patterns [20]....
[...]
21 Apr 2004
TL;DR: Preliminary results on a small dataset show that it is possible to recognize activities of interest to medical professionals such as toileting, bathing, and grooming with detection accuracies ranging from 25% to 89% depending on the evaluation criteria used.
Abstract: In this work, a system for recognizing activities in the home setting using a set of small and simple state-change sensors is introduced. The sensors are designed to be “tape on and forget” devices that can be quickly and ubiquitously installed in home environments. The proposed sensing system presents an alternative to sensors that are sometimes perceived as invasive, such as cameras and microphones. Unlike prior work, the system has been deployed in multiple residential environments with non-researcher occupants. Preliminary results on a small dataset show that it is possible to recognize activities of interest to medical professionals such as toileting, bathing, and grooming with detection accuracies ranging from 25% to 89% depending on the evaluation criteria used.
1,386 citations
"A Scalable Hybrid Activity Recognit..." refers methods in this paper
...Details of the datasets used for evaluation extracted from [37] and [10]....
[...]
...[10] use Naive Bayes classifiers for AR, but they do not show the results in terms of precision, recall and F-Measure....
[...]