Recently, device-free human behavior recognition has become a hot research topic and has achieved significant progress in the field of ubiquitous computing. Among various implementation, behavior recognition based on WiFi CSI (channel state information) has drawn wide attention due to its major advantages. This paper investigates more than 100 latest CSI based behavior recognition applications within the last 6 years and presents a comprehensive survey from every aspect of human behavior recognition. Firstly, this paper reviews general behavior recognition applications using the WiFi signal and presents the basic concept of CSI and the fundamental principle of CSI-based behavior recognition. This paper analyzes the key components and core characteristics of the system architecture of human behavior recognition using CSI. Afterward, we divide the sensing procedures into many steps and summarize the typical studies from these steps, including base signal selection, signal preprocessing, and identification approaches. Next, based on the recognition technique, we classify the applications into three groups, including pattern-based, model-based, and deep learning-based approach. In every group, we categorize the state-of-the-art applications into three groups, including coarse-grained specific behavior recognition, fine-grained specific behavior recognition, and activity inference. It elaborates the typical behavior recognition applications from five aspects, including experimental equipment, experimental scenario, behavior, classifier, and system performance. Then, this paper presents comprehensive discussions about representative applications from the implementation view and outlines the major consideration when developing a recognition system. Finally, this article concludes by analyzing the open issues of CSI-based behavior recognition applications and pointing out future research directions.

/pdf/a-survey-on-human-behavior-recognition-using-channel-state-17xs5detky.pdf

A Survey on Human Behavior Recognition Using Channel State Information

While decade-long research has clearly demonstrated the vast potential of radio frequency (RF) for many human sensing tasks, scaling this technology to large scenarios remained problematic with conventional approaches. Recently, researchers have successfully applied deep learning to take radio-based sensing to a new level. Many different types of deep learning models have been proposed to achieve high sensing accuracy over a large population and activity set, as well as in unseen environments. Deep learning has also enabled detection of novel human sensing phenomena that were previously not possible. In this survey, we provide a comprehensive review and taxonomy of recent research efforts on deep learning based RF sensing. We also identify and compare several publicly released labeled RF sensing datasets that can facilitate such deep learning research. Finally, we summarize the lessons learned and discuss the current limitations and future directions of deep learning based RF sensing.

Deep Learning for Radio-Based Human Sensing: Recent Advances and Future Directions

Hand gesture recognition is an important way for human machine interaction, and it is widely used in many areas, such as health care, smart home, virtual reality as well as other areas. While many valuable efforts have been made, it still lacks efficient ways to capture fine grain hand gesture as well as track data of long distance dependency in complex gesture. In this paper, we firstly design a novel data glove with two arm rings and a specially integrated three-dimensional flex sensor to capture fine grain motion from full arm and all knuckles. Secondly, an improved deep feature fusion network is proposed to detect long distance dependency in complex hand gestures. In order to track detailed motion features, a convolutional neural network based feature fusion strategy is given to fuse data from multi-sensors by extracting both shallow and deep features. Moreover, a residual module is introduced to avoid over fitting and gradient vanishing during deepening the neural network. Thirdly, a long short term memory (LSTM) model with fused feature vector as input is introduced to classify complex hand motions into corresponding categories. Results of comprehensive experiments demonstrate that our work performs better than related algorithms, especially in America sign language (with the precise of 99.93%) and Chinese sign language (with the precise of 96.1%).

Hand Gesture Recognition Using Deep Feature Fusion Network Based on Wearable Sensors

Due to its nonintrusive character, WiFi channel state information (CSI)-based activity recognition has attracted tremendous attention in recent years. Since activity recognition performance heavily relies on activity segmentation results, a number of activity segmentation methods have been designed, and most of them focus on seeking optimal thresholds to segment activities. However, these threshold-based methods are strongly dependent on designers’ experience and might suffer from performance decline when applying to the scenario, including both fine-grained and coarse-grained activities. To address these challenges, we present DeepSeg, a deep learning-based activity segmentation framework for activity recognition using WiFi signals. In this framework, we transform segmentation tasks into classification problems and propose a CNN-based activity segmentation algorithm, which can reduce the dependence on experience and address the performance degradation problem. To further enhance the overall performance, we design a feedback mechanism, where the segmentation algorithm is refined based on the feedback computed using activity recognition results. The experiments demonstrate that DeepSeg acquires remarkable gains compared with state-of-the-art approaches.

DeepSeg: Deep-Learning-Based Activity Segmentation Framework for Activity Recognition Using WiFi

Different gestures have different action speeds, directions, and trajectories that can cause distinctive effects on the propagation of WiFi signals. In this paper, we present a new approach that uses deep transfer learning techniques to recognize gestures based on the channel state information (CSI) extracted from WiFi signals. Firstly, the CSI streams of gestures are captured and the gesture segments are extracted based on the CSI amplitude changes, and then the WiFi-based gesture recognition problem is innovatively converted to an image classification problem by expressing CSI streams as an image matrix. After that, two deep transfer learning methods are applied to recognize gestures using high-level features extracted by deep convolutional neural network (CNN) and fine-tuned CNN models. We evaluated our method using a collected dataset with 12 gestures in two environments, and the experimental results demonstrated that the proposed method outperformed other state-of-the-art WiFi-based gesture recognition methods.

Deep transfer learning for gesture recognition with WiFi signals

This paper presents a deep transfer learning based gesture recognition approach using Channel State Information (CSI). In our method, the amplitude of each CSI stream is reorganized as an image matrix. Then Wi-Fi based gesture recognition is cast into image classification framework. The shift to an image classification paradigm provides the advantage of using an existing Convolutional Neural Network (CNN) model pre-trained on a very large number of images. Furthermore, transfer learning model is employed to eliminate the need for a lengthy network training process and to extract more suitable features for the CSI matrix. Finally, we test our method on a dataset collected by ourselves, and the result shows that our method has a better performance than state-of-the-art in Wi-Fi based gesture recognition.

Wi-Fi Based Gesture Recognition Using Deep Transfer Learning

Because of the uniqueness of human gait, the WiFi signal reflected by a walking person can generate a distinctive variation in the received WiFi channel state information (CSI). In this paper, we present a new passive human identification method named HumanFi based on fine-grained gait patterns captured by commercial WiFi device and long short term memory network (LSTM). Firstly, CSI measurements are collected by a commercial WiFi device, and then a buffer and filtering mechanism-based gait detection algorithm is proposed to solve the effects of short-term anomalous fluctuation. After that, a recurrent neural network, LSTM, is used to identify different people by discriminating the temporal characteristics of automatically extracted human gait features. We evaluated the proposed HumanFi using a dataset with 1920 gait instances collected from 24 human subjects walking in two different scenes. Experimental results showed that HumanFi achieved more than 96% human identification accuracy, which demonstrated the good performance of HumanFi on non-intrusive human identification tasks.

HumanFi: WiFi-Based Human Identification Using Recurrent Neural Network

WiFi has recently established itself as a powerful medium for radio frequency (RF) sensing due to its low cost and convenience. Many tasks, such as gesture recognition, activity recognition, and fall detection, can be implemented by measuring and calculating how the propagation of WiFi signals is affected by human activities. However, current WiFi-based sensing solutions have limited scales as they are designed for only a few activities and need to collect data and create training models in the same domain because the model established in a deployment environment is usually not applicable to new objects in the target domain. This paper presents TransferSense, an environment independent and one-shot WiFi sensing method based on deep learning. Firstly, amplitude and phase information of channel state information (CSI) are combined to increase the number of features to solve the problem of insufficient features due to single-source information. Secondly, TransferSense converts RF sensing tasks to image classification tasks and fuses low-level and high-level semantic features extracted from a pre-trained convolutional neural network to achieve an end-to-end high-precision sensing for activity recognition. Finally, TransferSense applies a transfer learning method with a small number of labeled samples in the target domain to perform high-precision cross-domain sensing, which can reduce the data collection cost in the target domain. We verified the effectiveness of TransferSense using two representative WiFi sensing applications, gait identification and sign recognition. In a single deployment environment, TransferSense achieved more than 97% human gait identification accuracy for 44 users and more than 81% sign language recognition for 100 isolated sign language words. In the case of new object recognition in the cross-domain sensing, TransferSense achieved more than 77% human gait identification accuracy for 10 new users, more than 88% sign language recognition for 10 new isolated sign language words, and more than 81% gesture identification for 2 new gestures.

Xingxia Ming

Papers

Deep transfer learning for gesture recognition with WiFi signals

Wi-Fi Based Gesture Recognition Using Deep Transfer Learning

HumanFi: WiFi-Based Human Identification Using Recurrent Neural Network

TransferSense: towards environment independent and one-shot wifi sensing