The past years have witnessed the rapid conceptualization and development of wireless sensing based on Channel State Information (CSI) with commodity WiFi devices. Recent studies have demonstrated the vast potential of WiFi sensing in detection, recognition, and estimation applications. However, the widespread deployment of WiFi sensing systems still faces a significant challenge: how to ensure the sensing performance when exposing a pre-trained sensing system to new domains, such as new environments, different configurations, and unseen users, without data collection and system retraining. This survey provides a comprehensive review of recent research efforts on cross-domain WiFi Sensing. We first introduce the mathematical model of CSI and explore the impact of different domains on CSI. Then we present a general workflow of cross-domain WiFi sensing systems, which consists of signal processing and cross-domain sensing. Five cross-domain sensing algorithms, including domain-invariant feature extraction, virtual sample generation, transfer learning, few-shot learning and big data solution, are summarized to show how they achieve high sensing accuracy when encountering new domains. The advantages and limitations of each algorithm are also summarized and the performance comparison is made based on different applications. Finally, we discuss the remaining challenges to further promote the practical usability of cross-domain WiFi sensing systems.

Cross-Domain WiFi Sensing with Channel State Information: A Survey

Recently, device-free human behavior recognition has become a hot research topic and has achieved significant progress in the field of ubiquitous computing. Among various implementation, behavior recognition based on WiFi CSI (channel state information) has drawn wide attention due to its major advantages. This paper investigates more than 100 latest CSI based behavior recognition applications within the last 6 years and presents a comprehensive survey from every aspect of human behavior recognition. Firstly, this paper reviews general behavior recognition applications using the WiFi signal and presents the basic concept of CSI and the fundamental principle of CSI-based behavior recognition. This paper analyzes the key components and core characteristics of the system architecture of human behavior recognition using CSI. Afterward, we divide the sensing procedures into many steps and summarize the typical studies from these steps, including base signal selection, signal preprocessing, and identification approaches. Next, based on the recognition technique, we classify the applications into three groups, including pattern-based, model-based, and deep learning-based approach. In every group, we categorize the state-of-the-art applications into three groups, including coarse-grained specific behavior recognition, fine-grained specific behavior recognition, and activity inference. It elaborates the typical behavior recognition applications from five aspects, including experimental equipment, experimental scenario, behavior, classifier, and system performance. Then, this paper presents comprehensive discussions about representative applications from the implementation view and outlines the major consideration when developing a recognition system. Finally, this article concludes by analyzing the open issues of CSI-based behavior recognition applications and pointing out future research directions.

/pdf/a-survey-on-human-behavior-recognition-using-channel-state-17xs5detky.pdf

A Survey on Human Behavior Recognition Using Channel State Information

Currently, developments in wireless sensing technologies have shown that wireless signals can be employed to transmit information between wireless communication devices and are also able to realize passive target wireless sensing. Wireless sensing has diverse Internet-of-Things applications in indoor human detection, such as in device-free localization, activity recognition and fall detection, respiration detection, gait recognition, user identification, and so forth. Deep learning (DL), with the latest breakthroughs in machine learning (ML) and artificial intelligence (AI), seems to be a feasible technique for device-free wireless sensing (DFWS) and human detection in a more intelligent and autonomous manner. Although DL has attracted wide spread attention in computer vision (CV), AI games, speech recognition, automated vehicles, and other fields, its application in wireless sensing systems (WSSs) is relatively new, and little attention has been paid to it. Motivated by these developments, this article clarifies the motivation and mechanism of the DL-aided WSSs for human detection. First, we survey the most advanced architecture of DL that may be powerful for WSSs. We also review conventional ML and DL approaches to human detection based on red green blue (RGB)/depth camera and radar: one reason is to introduce the successful experience in these areas to the field of wireless sensing and another reason is that the possibility of combining and fusing information from the heterogeneous types of sensors is expected to improve the overall performance of practical human detection systems. We provide a comprehensive survey of the state-of-the-art research on wireless sensing for human detection with a focus on WSSs. Furthermore, a general structure of the DL-based WSS is introduced in detail for hitherto unexplored applications and future wireless sensing scenarios. We also discuss some open research issues in wireless sensing for human detection, including data acquisition for DL model training, calibration of signals from commercial devices, multimodal sensing, simultaneous user identification and activity recognition, multiuser human detection, and generalization ability of DL models, to indicate future research directions.

Device-Free Wireless Sensing for Human Detection: The Deep Learning Perspective

Recent years have witnessed the rapid development in the research topic of WiFi sensing that automatically senses human with commercial WiFi devices. Past work falls into two major categories, i.e., activity recognition and the indoor localization. The former work utilizes WiFi devices to recognize human daily activities such as smoking, walking, and dancing. The latter one, indoor localization, can be used for indoor navigation, location-based services, and through-wall surveillance. The key rationale behind WiFi sensing is that people behaviors can influence the WiFi signal propagation and introduce specific patterns into WiFi signals, called WiFi fingerprints, which can be further explored to identify human activities and locations. In this paper, we propose a novel deep learning framework for joint activity recognition and indoor localization task using WiFi channel state information (CSI) fingerprints. More precisely, we develop a system running standard IEEE 802.11n WiFi protocol and collect more than 1400 CSI fingerprints on 6 activities at 16 indoor locations. Then we propose a dual-task convolutional neural network with one-dimensional convolutional layers for the joint task of activity recognition and indoor localization. The experimental results and ablation study show that our approach achieves good performances in this joint WiFi sensing task. Data and code have been made publicly available at https://github.com/geekfeiw/apl.

Joint Activity Recognition and Indoor Localization With WiFi Fingerprints

Multi-user activity recognition: Challenges and opportunities

This paper presents MultiTrack, a commodity WiFi based human sensing system that can track multiple users and recognize activities of multiple users performing them simultaneously. Such a system can enable easy and large-scale deployment for multi-user tracking and sensing without the need for additional sensors through the use of existing WiFi devices (e.g., desktops, laptops and smart appliances). The basic idea is to identify and extract the signal reflection corresponding to each individual user with the help of multiple WiFi links and all the available WiFi channels at 5GHz. Given the extracted signal reflection of each user, MultiTrack examines the path of the reflected signals at multiple links to simultaneously track multiple users. It further reconstructs the signal profile of each user as if only a single user has performed activity in the environment to facilitate multi-user activity recognition. We evaluate MultiTrack in different multipath environments with up to 4 users for multi-user tracking and up to 3 users for activity recognition. Experimental results show that our system can achieve decimeter localization accuracy and over 92% activity recognition accuracy under multi-user scenarios.

MultiTrack: Multi-User Tracking and Activity Recognition Using Commodity WiFi

Biometric-based authentication is gaining increasing attention for wearables and mobile applications. Meanwhile, the growing adoption of sensors in wearables also provides opportunities to capture novel wearable biometrics. In this work, we propose EarDynamic, an ear canal deformation based user authentication using in-ear wearables. EarDynamic provides continuous and passive user authentication and is transparent to users. It leverages ear canal deformation that combines the unique static geometry and dynamic motions of the ear canal when the user is speaking for authentication. It utilizes an acoustic sensing approach to capture the ear canal deformation with the built-in microphone and speaker of the in-ear wearable. Specifically, it first emits well-designed inaudible beep signals and records the reflected signals from the ear canal. It then analyzes the reflected signals and extracts fine-grained acoustic features that correspond to the ear canal deformation for user authentication. Our extensive experimental evaluation shows that EarDynamic can achieve a recall of 97.38% and an F1 score of 96.84%. Results also show that our system works well under different noisy environments with various daily activities.

EarDynamic: An Ear Canal Deformation Based Continuous User Authentication Using In-Ear Wearables

Walking while using a smartphone is becoming a major pedestrian safety concern as people may unknowingly bump into various obstacles that could lead to severe injuries. In this paper, we propose ObstacleWatch, an acoustic-based obstacle collision detection system to improve the safety of pedestrians who are engaged in smartphone usage while walking. ObstacleWatch leverages the advanced audio hardware of the smartphone to sense the surrounding obstacles and infers fine-grained information about the frontal obstacle for collision detection. In particular, our system emits well-designed inaudible beep signals from the smartphone built-in speaker and listens to the reflections with the stereo recording of the smartphone. By analyzing the reflected signals received at two microphones, ObstacleWatch is able to extract fine-grained information of the frontal obstacle including the distance, angle and size for detecting the possible collisions and to alert users. Our experimental evaluation under two real-world environments with different types of phones and obstacles shows that ObstacleWatch achieves over 92% accuracy in predicting obstacle collisions with distance estimation errors at about 2 cm. Results also show that ObstacleWatch is robust to different sizes of objects and is compatible to different phone models with low energy consumption.

ObstacleWatch: Acoustic-based Obstacle Collision Detection for Pedestrian Using Smartphone

The popularity of Internet-of-Things (IoT) has provided us with unprecedented opportunities to enable a variety of emerging services in a smart home environment. Among those services, sensing the liquid level in a container is critical to building many smart home and mobile healthcare applications that improve the quality of life. This paper presents LiquidSense, a liquid level sensing system that is low-cost, high accuracy, widely applicable to different daily liquids and containers, and can be easily integrated with existing smart home networks. LiquidSense uses existing home WiFi network and a low-cost transducer that attached to the container to sense the resonance of the container for liquid level detection. In particular, our system mounts a low-cost transducer on the surface of the container and emits a well-designed chirp signal to make the container resonant, which introduces subtle changes to the home WiFi signals. By analyzing the subtle phase changes of the WiFi signals, LiquidSense extracts the resonance frequency as a feature for liquid level detection. Our system constructs prediction models for both continuous and discrete predictions using curve fitting and SVM respectively. We evaluate LiquidSense in home environments with containers of three different materials and six types of liquids. Results show that LiquidSense achieves an overall accuracy of 97% for continuous prediction and an overall F-score of 0.968 for discrete predication. Results also show that our system has a large coverage in a home environment and works well under non-line-of-sight (NLOS) scenarios.

Liquid Level Sensing Using Commodity WiFi in a Smart Home Environment

The voice user interface (VUI) has been progressively used to authenticate users to numerous devices and applications. Such massive adoption of VUIs in IoT environments like individual homes and businesses arises extensive privacy and security concerns. Latest VUIs adopting traditional voice authentication methods are vulnerable to spoofing attacks, where a malicious party spoofs the VUIs with pre-recorded or synthesized voice commands of the genuine user. In this paper, we design VibLive, a continuous liveness detection system for secure VUIs in IoT environments. The underlying principle of VibLive is to catch the dissimilarities between bone-conducted vibrations and air-conducted voices when human speaks for liveness detection. VibLive is a text-independent system that verifies live users and detects spoofing attacks without requiring users to enroll specific passphrases. Moreover, VibLive is practical and transparent as it requires neither additional operations nor extra hardwares, other than a loudspeaker and a microphone that are commonly equipped on VUIs. Our evaluation with 25 participants under different IoT intended experiment settings shows that VibLive is highly effective with over 97% detection accuracy. Results also show that VibLive is robust to various use scenarios.

https://dl.acm.org/doi/pdf/10.1145/3427228.3427281

Zi Wang

Papers

MultiTrack: Multi-User Tracking and Activity Recognition Using Commodity WiFi

EarDynamic: An Ear Canal Deformation Based Continuous User Authentication Using In-Ear Wearables

ObstacleWatch: Acoustic-based Obstacle Collision Detection for Pedestrian Using Smartphone

Liquid Level Sensing Using Commodity WiFi in a Smart Home Environment

VibLive: A Continuous Liveness Detection for Secure Voice User Interface in IoT Environment