Online Human Activity Recognition using Low-Power Wearable Devices

doi:10.1145/3240765.3240833

Home
/
Papers
/
Online Human Activity Recognition using Low-Power Wearable Devices

Proceedings Article•DOI•

Online Human Activity Recognition using Low-Power Wearable Devices

Ganapati Bhat¹, Ranadeep Deb¹, Vatika Vardhan Chaurasia¹, Holly A. Shill, Umit Y. Ogras¹ - Show less +1 more•Institutions (1)

Arizona State University¹

26 Aug 2018-arXiv: Computer Vision and Pattern Recognition-

TL;DR: This paper presents the first HAR framework that can perform both online training and inference, and starts with a novel technique that generates features using the fast Fourier and discrete wavelet transforms of a textile-based stretch sensor and accelerometer data.

read less

Abstract: Human activity recognition~(HAR) has attracted significant research interest due to its applications in health monitoring and patient rehabilitation. Recent research on HAR focuses on using smartphones due to their widespread use. However, this leads to inconvenient use, limited choice of sensors and inefficient use of resources, since smartphones are not designed for HAR. This paper presents the first HAR framework that can perform both online training and inference. The proposed framework starts with a novel technique that generates features using the fast Fourier and discrete wavelet transforms of a textile-based stretch sensor and accelerometer. Using these features, we design an artificial neural network classifier which is trained online using the policy gradient algorithm. Experiments on a low power IoT device (TI-CC2650 MCU) with nine users show 97.7% accuracy in identifying six activities and their transitions with less than 12.5 mW power consumption.

...read moreread less

Citations

PDF

Open Access

More filters

Journal Article•DOI•

Device free human gesture recognition using Wi-Fi CSI: A survey

[...]

Hasmath Farhana Thariq Ahmed¹, Hafisoh Ahmad¹, C V Aravind¹•Institutions (1)

Taylors University¹

01 Jan 2020-Engineering Applications of Artificial Intelligence

TL;DR: Various signal pre-processing, feature extraction, selection, and classification techniques that are widely adopted for gesture recognition along with the environmental factors that influence the recognition accuracy are discussed.

...read moreread less

61 citations

Journal Article•DOI•

Application of artificial intelligence in wearable devices: Opportunities and challenges.

[...]

Darius Nahavandi¹, Roohallah Alizadehsani¹, Abbas Khosravi¹, U. Rajendra Acharya²•Institutions (2)

Deakin University¹, Ngee Ann Polytechnic²

01 Jan 2022-Computer Methods and Programs in Biomedicine

TL;DR: In this paper, the authors reviewed the recent applications of wearables that have leveraged AI to achieve their objectives, and the most important challenges regarding design and development of wearable devices and the computation burden of using AI methods are presented.

...read moreread less

46 citations

Journal Article•DOI•

w-HAR: An Activity Recognition Dataset and Framework Using Low-Power Wearable Devices

[...]

Ganapati Bhat¹, Nicholas Tran², Holly A. Shill, Umit Y. Ogras³•Institutions (3)

Washington State University¹, Arizona State University², University of Wisconsin-Madison³

18 Sep 2020-Sensors

TL;DR: Wearable HAR (w-HAR) is presented which contains labeled data of seven activities from 22 users and contains the integration of data from inertial and wearable stretch sensors, thus providing two modalities of activity information.

...read moreread less

Abstract: Human activity recognition (HAR) is growing in popularity due to its wide-ranging applications in patient rehabilitation and movement disorders. HAR approaches typically start with collecting sensor data for the activities under consideration and then develop algorithms using the dataset. As such, the success of algorithms for HAR depends on the availability and quality of datasets. Most of the existing work on HAR uses data from inertial sensors on wearable devices or smartphones to design HAR algorithms. However, inertial sensors exhibit high noise that makes it difficult to segment the data and classify the activities. Furthermore, existing approaches typically do not make their data available publicly, which makes it difficult or impossible to obtain comparisons of HAR approaches. To address these issues, we present wearable HAR (w-HAR) which contains labeled data of seven activities from 22 users. Our dataset’s unique aspect is the integration of data from inertial and wearable stretch sensors, thus providing two modalities of activity information. The wearable stretch sensor data allows us to create variable-length segment data and ensure that each segment contains a single activity. We also provide a HAR framework to use w-HAR to classify the activities. To this end, we first perform a design space exploration to choose a neural network architecture for activity classification. Then, we use two online learning algorithms to adapt the classifier to users whose data are not included at design time. Experiments on the w-HAR dataset show that our framework achieves 95% accuracy while the online learning algorithms improve the accuracy by as much as 40%.

...read moreread less

41 citations

Cites background or methods or result from "Online Human Activity Recognition u..."

...In comparison to [6], this paper makes the following contributions:...
[...]
...Therefore, we use the activity-based segmentation in [6] to generate the segments in w-HAR....
[...]
...We also note that this paper is an extended version of our work in [6]....
[...]
...Segmented Data: The segmented dataset uses the segmentation algorithm proposed in [6] and summarized in Section 3....
[...]
...Here, we provide a summary of the features provided in the dataset, while the detailed motivation for choosing these is presented in [6]....
[...]

Journal Article•DOI•

OpenHealth: Open-Source Platform for Wearable Health Monitoring

[...]

Ganapati Bhat¹, Ranadeep Deb¹, Umit Y. Ogras¹•Institutions (1)

Arizona State University¹

19 Mar 2019-IEEE Design & Test of Computers

TL;DR: An open-source platform for wearable health monitoring that can enable autonomous collection of clinically relevant data is presented and reference implementations of human activity and gesture recognition applications within this platform are provided.

...read moreread less

Abstract: Editor’s note: This article presents an open-source platform for wearable health monitoring. It aims to design a standard set of hardware/software and wearable devices that can enable autonomous collection of clinically relevant data. It provides reference implementations of human activity and gesture recognition applications within this platform. –Jana, Doppa, Washington State University

...read moreread less

34 citations

Cites background or methods from "Online Human Activity Recognition u..."

...This approach enables physically flexible or stretchable devices that can blend in with clothes, such as a knee sleeve [1]....
[...]
...platform, which contains the MPU-9250 motion sensor, along with a wearable stretch sensor [1]....
[...]

Journal Article•DOI•

Trends in human activity recognition with focus on machine learning and power requirements

[...]

Binh Nguyen¹, Yves Luduvico Coelho², Teodiano Bastos², Sridhar Krishnan¹•Institutions (2)

Ryerson University¹, Universidade Federal do Espírito Santo²

15 Sep 2021

TL;DR: To reach a marketable HAR device, state-of-the-art classifications and power consumption methods such as convolutional neural network (CNN), data compression and other emerging techniques are reviewed here.

...read moreread less

Abstract: The advancement and availability of technology can be employed to improve our daily lives. One example is Human Activity Recognition (HAR). HAR research has been mainly explored using imagery but is currently evolving to the use of sensors and has the ability to have a positive impact, including individual health monitoring and removing the barrier of healthcare. To reach a marketable HAR device, state-of-the-art classifications and power consumption methods such as convolutional neural network (CNN), data compression and other emerging techniques are reviewed here. The review of the current literature creates a foundation in HAR and addresses the lack of available HAR datasets, recommendation of classification and power reduction techniques, current drawbacks and their respective solutions, as well as future trends in HAR. The lack of publicly available datasets makes it difficult for new users to explore the field of HAR. This paper dedicates a section to publicly available datasets for users to access. Finally, a framework is suggested for HAR applications, which envelopes the current literature and emerging trends in HAR.

...read moreread less

34 citations

1
2
3
4
…
5
6
7

Collapse

References

PDF

Open Access

More filters

Book•

The Elements of Statistical Learning

[...]

Trevor Hastie, Robert Tibshirani, Jerome H. Friedman

01 Jan 2001

19,211 citations

"Online Human Activity Recognition u..." refers methods in this paper

...Random Forests and Decision Trees: Random forests [14] use an ensemble of tree-structured classiers, where each tree independently predicts the output class as a function of the feature vector....
[...]
...Finally, existing studies on HAR approaches employ commonly used classiers, such as k-NN [14], support vector machines [14], decision trees [30], and random forest [14], which are trained oine....
[...]
...Support Vector Machine (SVM): SVM [14] nds a hyperplane that can separate the feature vectors of two output classes....
[...]
...k-Nearest Neighbors (k-NN): k-Nearest Neighbors [14] is one of the most popular techniques used by many previous HAR studies....
[...]

Journal Article•DOI•

The Elements of Statistical Learning

[...]

Eric R. Ziegel

01 Aug 2003-Technometrics

TL;DR: Chapter 11 includes more case studies in other areas, ranging from manufacturing to marketing research, and a detailed comparison with other diagnostic tools, such as logistic regression and tree-based methods.

...read moreread less

Abstract: Chapter 11 includes more case studies in other areas, ranging from manufacturing to marketing research. Chapter 12 concludes the book with some commentary about the scienti c contributions of MTS. The Taguchi method for design of experiment has generated considerable controversy in the statistical community over the past few decades. The MTS/MTGS method seems to lead another source of discussions on the methodology it advocates (Montgomery 2003). As pointed out by Woodall et al. (2003), the MTS/MTGS methods are considered ad hoc in the sense that they have not been developed using any underlying statistical theory. Because the “normal” and “abnormal” groups form the basis of the theory, some sampling restrictions are fundamental to the applications. First, it is essential that the “normal” sample be uniform, unbiased, and/or complete so that a reliable measurement scale is obtained. Second, the selection of “abnormal” samples is crucial to the success of dimensionality reduction when OAs are used. For example, if each abnormal item is really unique in the medical example, then it is unclear how the statistical distance MD can be guaranteed to give a consistent diagnosis measure of severity on a continuous scale when the larger-the-better type S/N ratio is used. Multivariate diagnosis is not new to Technometrics readers and is now becoming increasingly more popular in statistical analysis and data mining for knowledge discovery. As a promising alternative that assumes no underlying data model, The Mahalanobis–Taguchi Strategy does not provide suf cient evidence of gains achieved by using the proposed method over existing tools. Readers may be very interested in a detailed comparison with other diagnostic tools, such as logistic regression and tree-based methods. Overall, although the idea of MTS/MTGS is intriguing, this book would be more valuable had it been written in a rigorous fashion as a technical reference. There is some lack of precision even in several mathematical notations. Perhaps a follow-up with additional theoretical justi cation and careful case studies would answer some of the lingering questions.

...read moreread less

11,507 citations

"Online Human Activity Recognition u..." refers methods in this paper

...If a separating hyperplane does not exist, SVM maps the data into higher dimensions until a separating hyperplane is found....
[...]
...k-Nearest Neighbors (k-NN): k-Nearest Neighbors [14] is one of the most popular techniques used by many previous HAR studies....
[...]
...Random Forests and Decision Trees: Random forests [14] use an ensemble of tree-structured classi ers, where each tree independently predicts the output class as a function of the feature vector....
[...]
...This adds additional processing and memory requirements on the system, making it unsuitable for implementation on a wearable system with limited memory. k-Nearest Neighbors (k-NN): k-Nearest Neighbors [14] is one of the most popular techniques used by many previous HAR studies. k-NN evaluates the output class by rst calculating k nearest neighbors in the training dataset....
[...]
...Due to this, SVM is not suitable for reinforcement learning with multiple classes [21], which is the case in our HAR framework....
[...]

Book•

Introduction to Reinforcement Learning

[...]

Richard S. Sutton, Andrew G. Barto

01 Mar 1998

TL;DR: In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning.

...read moreread less

Abstract: From the Publisher: In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.

...read moreread less

7,016 citations

"Online Human Activity Recognition u..." refers background or methods in this paper

...ncluding arti˙cial neural network, random forest, and k-nearest neighbor (kNN). Among these, we focus on arti˙cial neural network, since it enables online reinforcement learning using policy gradient [33] with low implementation cost. Finally, this work is the ˙rst to provide a detailed power consumption and performance break-down of sensing, processing and communication tasks. We implement the propos...
[...]
...e the weights denoted byθin Figure 5 to tune our optimized ANN to individual users. Since we use the value function as the objective, the gradient of J„θ”is proportional to the gradient of the policy [33]. Using this result, the update equation for θis given as: θt+1 θt +αrt rθπ„a tjh;θ ” π„at jh;θt” ; α: Learning rate (5) where θt and θt+1 are the current and updated weight matrices, respectively. ...
[...]
... the reward is negative (1). We de˙ne the sequence of segments for which a reward is given as an epoch. The set of epochs in a given training session is called an episode following the RL terminology [33]. Objective: The value function for a state is de˙ned as the total reward that can be earned starting from that state and following the givenpolicyuntiltheendofanepisode.Ourobjectiveistomaximize the t...
[...]

Book Chapter•DOI•

Activity recognition from user-annotated acceleration data

[...]

Ling Bao¹, Stephen S. Intille¹•Institutions (1)

Massachusetts Institute of Technology¹

21 Apr 2004

TL;DR: This is the first work to investigate performance of recognition algorithms with multiple, wire-free accelerometers on 20 activities using datasets annotated by the subjects themselves, and suggests that multiple accelerometers aid in recognition.

...read moreread less

Abstract: In this work, algorithms are developed and evaluated to de- tect physical activities from data acquired using five small biaxial ac- celerometers worn simultaneously on different parts of the body. Ac- celeration data was collected from 20 subjects without researcher su- pervision or observation. Subjects were asked to perform a sequence of everyday tasks but not told specifically where or how to do them. Mean, energy, frequency-domain entropy, and correlation of acceleration data was calculated and several classifiers using these features were tested. De- cision tree classifiers showed the best performance recognizing everyday activities with an overall accuracy rate of 84%. The results show that although some activities are recognized well with subject-independent training data, others appear to require subject-specific training data. The results suggest that multiple accelerometers aid in recognition because conjunctions in acceleration feature values can effectively discriminate many activities. With just two biaxial accelerometers - thigh and wrist - the recognition performance dropped only slightly. This is the first work to investigate performance of recognition algorithms with multiple, wire-free accelerometers on 20 activities using datasets annotated by the subjects themselves.

...read moreread less

3,223 citations

"Online Human Activity Recognition u..." refers background or methods in this paper

...Early work on HAR used wearable sensors to perform data collection while performing various activities [5]....
[...]
...Then, both classier design and inference are performed oine [5]....
[...]

Journal Article•DOI•

Activity recognition using cell phone accelerometers

[...]

Jennifer R. Kwapisz¹, Gary M. Weiss¹, Samuel A. Moore¹•Institutions (1)

Fordham University¹

31 Mar 2011-Sigkdd Explorations

TL;DR: This work describes and evaluates a system that uses phone-based accelerometers to perform activity recognition, a task which involves identifying the physical activity a user is performing, and has a wide range of applications, including automatic customization of the mobile device's behavior based upon a user's activity.

...read moreread less

Abstract: Mobile devices are becoming increasingly sophisticated and the latest generation of smart cell phones now incorporates many diverse and powerful sensors These sensors include GPS sensors, vision sensors (ie, cameras), audio sensors (ie, microphones), light sensors, temperature sensors, direction sensors (ie, magnetic compasses), and acceleration sensors (ie, accelerometers) The availability of these sensors in mass-marketed communication devices creates exciting new opportunities for data mining and data mining applications In this paper we describe and evaluate a system that uses phone-based accelerometers to perform activity recognition, a task which involves identifying the physical activity a user is performing To implement our system we collected labeled accelerometer data from twenty-nine users as they performed daily activities such as walking, jogging, climbing stairs, sitting, and standing, and then aggregated this time series data into examples that summarize the user activity over 10- second intervals We then used the resulting training data to induce a predictive model for activity recognition This work is significant because the activity recognition model permits us to gain useful knowledge about the habits of millions of users passively---just by having them carry cell phones in their pockets Our work has a wide range of applications, including automatic customization of the mobile device's behavior based upon a user's activity (eg, sending calls directly to voicemail if a user is jogging) and generating a daily/weekly activity profile to determine if a user (perhaps an obese child) is performing a healthy amount of exercise

...read moreread less

2,417 citations

"Online Human Activity Recognition u..." refers background or methods or result in this paper

...Most existing studies employ statistical features such as mean, median, minimum, maximum, and kurtosis to perform HAR [4, 20, 28]....
[...]
...For instance, the studies in [4, 20] use 10 second windows to perform activity recognition....
[...]
...HAR studies typically use a xed window length to infer the activity of a person [4, 20]....
[...]
...In contrast, prior studies are forced to divide the sensor data into xed windows [4, 20] or smoothen noisy accelerometer data over long durations [10] (detailed in Section 2)....
[...]
...For example, accelerometers in smartphones are used to recognize activities such as stand, sit, lay down, walking, and jogging [3, 16, 20]....
[...]