A review and categorization of techniques on device-free human activity recognition

Wearable human machine interfaces (HMI) such as smart gloves have attracted considerable interest in recent years. The quality of the interactive experience with the real and virtual world using wearable HMI technologies depends on the intuitive two‐way haptic interfaces they offer and the real‐time touch‐based information they send and receive. Herein, various smart glove solutions and their application in interaction, rehabilitation, virtual (VR) and augmented reality (AR), and augmentative and alternative communication (AAC) tasks are reviewed. While the early variants of such systems were based on commercial touch sensors and displays integrated (e.g., stitched) on wearables, electronic skin (e‐skin)‐type technologies with multifunctional capabilities are being explored nowadays for rich user experience and comfort. In this regard, instead of using separate touch sensors and actuators, miniaturized integrated devices providing both touch sensing and vibrotactile actuation have also been reported recently. Such advances, the associated challenges, and the advantages they offer for users to enjoy the full characteristic benefits of VR/ARs for interaction, immersion, and imagination are discussed. Finally, the huge potential the smart‐glove‐type solutions hold for advances in various application areas such as robotics, health care, sensorial augmentation for nondisabled and tactile Internet is also discussed.

https://onlinelibrary.wiley.com/doi/pdf/10.1002/aisy.202100091

Smart Tactile Gloves for Haptic Interaction, Communication, and Rehabilitation

Low-cost consumer radar integrated circuits combined with recent advances in machine learning have opened up a range of new possibilities in smart sensing. In this paper, we use a miniature radar sensor to capture Doppler signatures of 14 different hand gestures and train a deep convolutional neural network (DCNN) to classify these captured gestures. We utilize two receiving antennas of a continuous-wave Doppler radar capable of producing the in-phase and quadrature components of the beat signals. We map these two beat signals into three input channels of a DCNN as two spectrograms and an angle of arrival matrix. The classification results of the proposed architecture show a gesture classification accuracy exceeding 95% and a very low confusion between different gestures. This is almost 10% improvement over the single-channel Doppler methods reported in the literature.

Hand-Gesture Recognition Using Two-Antenna Doppler Radar With Deep Convolutional Neural Networks

While decade-long research has clearly demonstrated the vast potential of radio frequency (RF) for many human sensing tasks, scaling this technology to large scenarios remained problematic with conventional approaches. Recently, researchers have successfully applied deep learning to take radio-based sensing to a new level. Many different types of deep learning models have been proposed to achieve high sensing accuracy over a large population and activity set, as well as in unseen environments. Deep learning has also enabled detection of novel human sensing phenomena that were previously not possible. In this survey, we provide a comprehensive review and taxonomy of recent research efforts on deep learning based RF sensing. We also identify and compare several publicly released labeled RF sensing datasets that can facilitate such deep learning research. Finally, we summarize the lessons learned and discuss the current limitations and future directions of deep learning based RF sensing.

Deep Learning for Radio-Based Human Sensing: Recent Advances and Future Directions

Channel state information (CSI)-based human activity recognition plays an essential role in various application domains, such as security, healthcare, and Internet of Things. Most existing CSI-based activity recognition approaches rely on manually designed features that are classified using traditional classification methods. Furthermore, the use of deep learning methods for CSI-based activity recognition is still at its infancy with most of the existing approaches focus on recognizing single-human activities. The current study explores the feasibility of utilizing deep learning methods to recognize human-to-human interactions (HHIs) using CSI signals. Particularly, we introduce an end-to-end deep learning framework that comprises three phases, which are the input, feature extraction, and recognition phases. The input phase converts the raw CSI signals into CSI images that comprise time, frequency, and spatial information. In the feature extraction phase, a novel convolutional neural network (CNN) is designed to automatically extract deep features from the CSI images. Finally, the extracted features are fed to the recognition phase to identify the class of the HHI associated with each CSI image. The performance of our proposed framework is assessed using a publicly available CSI dataset that was acquired from 40 different pairs of subjects while performing 13 HHIs. Our proposed framework achieved an average recognition accuracy of 86.3% across all HHIs. Moreover, the experiments indicate that our proposed framework enabled significant improvements over the results achieved using three state-of-the-art pre-trained CNNs as well as the results obtained using four different conventional classifiers that employs traditional handcrafted features.

C V Aravind

Papers

Device free human gesture recognition using Wi-Fi CSI: A survey