In the last decade, Human Activity Recognition (HAR) has become a vibrant research area, especially due to the spread of electronic devices such as smartphones, smartwatches and video cameras present in our daily lives. In addition, the advance of deep learning and other machine learning algorithms has allowed researchers to use HAR in various domains including sports, health and well-being applications. For example, HAR is considered as one of the most promising assistive technology tools to support elderly’s daily life by monitoring their cognitive and physical function through daily activities. This survey focuses on critical role of machine learning in developing HAR applications based on inertial sensors in conjunction with physiological and environmental sensors.

Human Activity Recognition Using Inertial, Physiological and Environmental Sensors: A Comprehensive Survey

Data Augmentation for Deep-Learning-Based Electroencephalography.

Deep learning for motor imagery EEG-based classification: A review

The brain–computer interface (BCI) is an emerging technology that has the potential to revolutionize the world, with numerous applications ranging from healthcare to human augmentation. Electroencephalogram (EEG) motor imagery (MI) is among the most common BCI paradigms that have been used extensively in smart healthcare applications such as post-stroke rehabilitation and mobile assistive robots. In recent years, the contribution of deep learning (DL) has had a phenomenal impact on MI-EEG-based BCI. In this work, we systematically review the DL-based research for MI-EEG classification from the past ten years. This article first explains the procedure for selecting the studies and then gives an overview of BCI, EEG, and MI systems. The DL-based techniques applied in MI classification are then analyzed and discussed from four main perspectives: preprocessing, input formulation, deep learning architecture, and performance evaluation. In the discussion section, three major questions about DL-based MI classification are addressed: (1) Is preprocessing required for DL-based techniques? (2) What input formulations are best for DL-based techniques? (3) What are the current trends in DL-based techniques? Moreover, this work summarizes MI-EEG-based applications, extensively explores public MI-EEG datasets, and gives an overall visualization of the performance attained for each dataset based on the reviewed articles. Finally, current challenges and future directions are discussed.

Deep learning techniques for classification of electroencephalogram (EEG) motor imagery (MI) signals: a review

The advent of dedicated Deep Learning (DL) accelerators and neuromorphic processors has brought on new opportunities for applying both Deep and Spiking Neural Network (SNN) algorithms to healthcare and biomedical applications at the edge. This can facilitate the advancement of medical Internet of Things (IoT) systems and Point of Care (PoC) devices. In this paper, we provide a tutorial describing how various technologies including emerging memristive devices, Field Programmable Gate Arrays (FPGAs), and Complementary Metal Oxide Semiconductor (CMOS) can be used to develop efficient DL accelerators to solve a wide variety of diagnostic, pattern recognition, and signal processing problems in healthcare. Furthermore, we explore how spiking neuromorphic processors can complement their DL counterparts for processing biomedical signals. The tutorial is augmented with case studies of the vast literature on neural network and neuromorphic hardware as applied to the healthcare domain. We benchmark various hardware platforms by performing a sensor fusion signal processing task combining electromyography (EMG) signals with computer vision. Comparisons are made between dedicated neuromorphic processors and embedded AI accelerators in terms of inference latency and energy. Finally, we provide our analysis of the field and share a perspective on the advantages, disadvantages, challenges, and opportunities that various accelerators and neuromorphic processors introduce to healthcare and biomedical domains.

Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications

Objective Electroencephalography (EEG) motor imagery classification has been widely used in healthcare applications such as mobile assistive robots and post-stroke rehabilitation. Recently, EEG motor imagery classification methods based on convolutional neural networks (CNNs) have been proposed and have achieved relatively high classification accuracy. However, these methods use single convolution scale in the CNN, while the best convolution scale differs from subject to subject. This limits the classification accuracy. Another issue is that the classification accuracy degrades when training data is limited. Approach To address these issues, we have proposed a hybrid-scale CNN architecture with a data augmentation method for EEG motor imagery classification. Main results Compared with several state-of-the-art methods, the proposed method achieves an average classification accuracy of 91.57% and 87.6% on two commonly used datasets, which outperforms several state-of-the-art EEG motor imagery classification methods. Significance The proposed method effectively addresses the issues of existing CNN-based EEG motor imagery classification methods and improves the classification accuracy.

https://iopscience.iop.org/article/10.1088/1741-2552/ab405f/pdf

HS-CNN: a CNN with hybrid convolution scale for EEG motor imagery classification.

Human activity recognition has been widely used in healthcare applications such as elderly monitoring, exercise supervision, and rehabilitation monitoring. Compared with other approaches, sensor-based wearable human activity recognition is less affected by environmental noise and therefore is promising in providing higher recognition accuracy. However, one of the major issues of existing wearable human activity recognition methods is that although the average recognition accuracy is acceptable, the recognition accuracy for some activities (e.g., ascending stairs and descending stairs) is low, mainly due to relatively less training data and complex behavior pattern for these activities. Another issue is that the recognition accuracy is low when the training data from the test subject are limited, which is a common case in real practice. In addition, the use of neural network leads to large computational complexity and thus high power consumption. To address these issues, we proposed a new human activity recognition method with two-stage end-to-end convolutional neural network and a data augmentation method. Compared with the state-of-the-art methods (including neural network based methods and other methods), the proposed methods achieve significantly improved recognition accuracy and reduced computational complexity.

TSE-CNN: A Two-Stage End-to-End CNN for Human Activity Recognition

Wearable intelligent ECG monitoring devices can perform automatic ECG diagnosis in real time and send out alert signal together with abnormal ECG signal for doctor's further analysis. This provides a means for the patient to identify their heart problem as early as possible and go to doctors for medical treatment. For such system the key requirements include high accuracy and low power consumption. However, the existing wearable intelligent ECG monitoring schemes suffer from high power consumption in both ECG diagnosis and transmission in order to achieve high accuracy. In this work, we have proposed an energy-efficient wearable intelligent ECG monitor scheme with two-stage end-to-end neural network and diagnosis-based adaptive compression. Compared to the state-of-the-art schemes, it significantly reduces the power consumption in ECG diagnosis and transmission while maintaining high accuracy.

Energy-Efficient Intelligent ECG Monitoring for Wearable Devices

This letter presents an energy-efficient reconfigurable AI-based object detection and tracking processor for smart drone/robot applications. Several techniques have been proposed to achieve high energy efficiency while supporting flexible object detection and tracking tasks with online object learning, including a reconfigurable object detection and tracking architecture with reconfigurable neural network (NN) engine, an online object learning architecture with shared NN inference and learning engine and automatic label generation engine, and a layer- and stride-aware NN computing technique. Compared with several state-of-the-art designs, the proposed design achieves better energy efficiency (2.13 mJ/frame), while supporting flexible object detection and tracking tasks with online object learning.

Jiahui Huang

Papers

HS-CNN: a CNN with hybrid convolution scale for EEG motor imagery classification.

TSE-CNN: A Two-Stage End-to-End CNN for Human Activity Recognition

Energy-Efficient Intelligent ECG Monitoring for Wearable Devices

An Energy-Efficient Reconfigurable AI-Based Object Detection and Tracking Processor Supporting Online Object Learning