With the high demand for wireless data traffic, WiFi networks have experienced very rapid growth, because they provide high throughput and are easy to deploy. Recently, Channel State Information (CSI) measured by WiFi networks is widely used for different sensing purposes. To get a better understanding of existing WiFi sensing technologies and future WiFi sensing trends, this survey gives a comprehensive review of the signal processing techniques, algorithms, applications, and performance results of WiFi sensing with CSI. Different WiFi sensing algorithms and signal processing techniques have their own advantages and limitations and are suitable for different WiFi sensing applications. The survey groups CSI-based WiFi sensing applications into three categories, detection, recognition, and estimation, depending on whether the outputs are binary/multi-class classifications or numerical values. With the development and deployment of new WiFi technologies, there will be more WiFi sensing opportunities wherein the targets may go beyond from humans to environments, animals, and objects. The survey highlights three challenges for WiFi sensing: robustness and generalization, privacy and security, and coexistence of WiFi sensing and networking. Finally, the survey presents three future WiFi sensing trends, i.e., integrating cross-layer network information, multi-device cooperation, and fusion of different sensors, for enhancing existing WiFi sensing capabilities and enabling new WiFi sensing opportunities.

WiFi Sensing with Channel State Information: A Survey

Driven by a wide range of real-world applications, significant efforts have recently been made to explore device-free human activity recognition techniques that utilize the information collected by various wireless infrastructures to infer human activities without the need for the monitored subject to carry a dedicated device. Existing device free human activity recognition approaches and systems, though yielding reasonably good performance in certain cases, are faced with a major challenge. The wireless signals arriving at the receiving devices usually carry substantial information that is specific to the environment where the activities are recorded and the human subject who conducts the activities. Due to this reason, an activity recognition model that is trained on a specific subject in a specific environment typically does not work well when being applied to predict another subject's activities that are recorded in a different environment. To address this challenge, in this paper, we propose EI, a deep-learning based device free activity recognition framework that can remove the environment and subject specific information contained in the activity data and extract environment/subject-independent features shared by the data collected on different subjects under different environments. We conduct extensive experiments on four different device free activity recognition testbeds: WiFi, ultrasound, 60 GHz mmWave, and visible light. The experimental results demonstrate the superior effectiveness and generalizability of the proposed EI framework.

https://dl.acm.org/doi/pdf/10.1145/3241539.3241548

Towards Environment Independent Device Free Human Activity Recognition

We propose SignFi to recognize sign language gestures using WiFi. SignFi uses Channel State Information (CSI) measured by WiFi packets as the input and a Convolutional Neural Network (CNN) as the classification algorithm. Existing WiFi-based sign gesture recognition technologies are tested on no more than 25 gestures that only involve hand and/or finger gestures. SignFi is able to recognize 276 sign gestures, which involve the head, arm, hand, and finger gestures, with high accuracy. SignFi collects CSI measurements to capture wireless signal characteristics of sign gestures. Raw CSI measurements are pre-processed to remove noises and recover CSI changes over sub-carriers and sampling time. Pre-processed CSI measurements are fed to a 9-layer CNN for sign gesture classification. We collect CSI traces and evaluate SignFi in the lab and home environments. There are 8,280 gesture instances, 5,520 from the lab and 2,760 from the home, for 276 sign gestures in total. For 5-fold cross validation using CSI traces of one user, the average recognition accuracy of SignFi is 98.01%, 98.91%, and 94.81% for the lab, home, and lab+home environment, respectively. We also run tests using CSI traces from 5 different users in the lab environment. The average recognition accuracy of SignFi is 86.66% for 7,500 instances of 150 sign gestures performed by 5 different users.

https://dl.acm.org/doi/pdf/10.1145/3191755

SignFi: Sign Language Recognition Using WiFi

In-air interaction acts as a key enabler for ambient intelligence and augmented reality. As an increasing popular example, exergames, and the alike gesture recognition applications, have attracted extensive research in designing accurate, pervasive and low-cost user interfaces. Recent advances in wireless sensing show promise for a ubiquitous gesture-based interaction interface with Wi-Fi. In this work, we extract complete information of motion-induced Doppler shifts with only commodity Wi-Fi. The key insight is to harness antenna diversity to carefully eliminate random phase shifts while retaining relevant Doppler shifts. We further correlate Doppler shifts with motion directions, and propose a light-weight pipeline to detect, segment, and recognize motions without training. On this basis, we present WiDance, a Wi-Fi-based user interface, which we utilize to design and prototype a contactless dance-pad exergame. Experimental results in typical indoor environment demonstrate a superior performance with an accuracy of 92%, remarkably outperforming prior approaches.

Inferring Motion Direction using Commodity Wi-Fi for Interactive Exergames

We present CrossSense, a novel system for scaling up WiFi sensing to new environments and larger problems. To reduce the cost of sensing model training data collection, CrossSense employs machine learning to train, off-line, a roaming model that generates from one set of measurements synthetic training samples for each target environment. To scale up to a larger problem size, CrossSense adopts a mixture-of-experts approach where multiple specialized sensing models, or experts, are used to capture the mapping from diverse WiFi inputs to the desired outputs. The experts are trained offline and at runtime the appropriate expert for a given input is automatically chosen. We evaluate CrossSense by applying it to two representative WiFi sensing applications, gait identification and gesture recognition, in controlled single-link environments. We show that CrossSense boosts the accuracy of state-of-the-art WiFi sensing techniques from 20% to over 80% and 90% for gait identification and gesture recognition respectively, delivering consistently good performance - particularly when the problem size is significantly greater than that current approaches can effectively handle.

https://dl.acm.org/doi/pdf/10.1145/3241539.3241570

CrossSense: Towards Cross-Site and Large-Scale WiFi Sensing

Gesture recognition has become increasingly important in human-computer interaction (HCI) and can support a broad array of emerging applications, such as smart home, virtual reality, and mobile gaming. Traditional approaches usually rely on dedicated sensors that are worn by the user or cameras that require line of sight. In this paper, we present fine-grained finger gesture recognition by using a single commodity WiFi device without requiring user to wear any sensors. Our low-cost system, WiFinger, takes advantages of the fine-grained Channel State Information (CSI) available from commodity WiFi devices and the prevalence of WiFi network infrastructures. It senses and identifies subtle movements of finger gestures by examining the unique patterns exhibited in the detailed CSI. In WiFigner, we devise environmental noise removal mechanism to mitigate the effect of signal dynamic due to the environment changes. Moreover, we propose to capture the intrinsic gesture behavior to deal with individual diversity and gesture inconsistency. Our experimental evaluation in both home and office environments demonstrates that our system can achieve over 93% recognition accuracy and is robust to both environment changes and individual diversity. Results also show that our system can work with WiFi beacon signals and provides accurate gesture recognition under NLOS scenarios.

WiFinger: leveraging commodity WiFi for fine-grained finger gesture recognition

Voice biometrics is drawing increasing attention as it is a promising alternative to legacy passwords for mobile authentication. Recently, a growing body of work shows that voice biometrics is vulnerable to spoofing through replay attacks, where an adversary tries to spoof voice authentication systems by using a pre-recorded voice sample collected from a genuine user. In this work, we propose VoiceGesture, a liveness detection system for replay attack detection on smartphones. It detects a live user by leveraging both the unique articulatory gesture of the user when speaking a passphrase and the mobile audio hardware advances. Specifically, our system re-uses the smartphone as a Doppler radar, which transmits a high frequency acoustic sound from the built-in speaker and listens to the reflections at the microphone when a user speaks a passphrase. The signal reflections due to user's articulatory gesture result in Doppler shifts, which are then analyzed for live user detection. VoiceGesture is practical as it requires neither cumbersome operations nor additional hardware but a speaker and a microphone that are commonly available on smartphones. Our experimental evaluation with 21 participants and different types of phones shows that it achieves over 99% detection accuracy at around 1% Equal Error Rate (EER). Results also show that it is robust to different phone placements and is able to work with different sampling frequencies.

Hearing Your Voice is Not Enough: An Articulatory Gesture Based Liveness Detection for Voice Authentication

Voice authentication is drawing increasing attention and becomes an attractive alternative to passwords for mobile authentication. Recent advances in mobile technology further accelerate the adoption of voice biometrics in an array of diverse mobile applications. However, recent studies show that voice authentication is vulnerable to replay attacks, where an adversary can spoof a voice authentication system using a pre-recorded voice sample collected from the victim. In this paper, we propose VoiceLive, a practical liveness detection system for voice authentication on smartphones. VoiceLive detects a live user by leveraging the user's unique vocal system and the stereo recording of smartphones. In particular, with the phone closely placed to a user's mouth, it captures time-difference-of-arrival (TDoA) changes in a sequence of phoneme sounds to the two microphones of the phone, and uses such unique TDoA dynamic which doesn't exist under replay attacks for liveness detection. VoiceLive is practical as it doesn't require additional hardware but two-channel stereo recording that is supported by virtually all smartphones. Our experimental evaluation with 12 participants and different types of phones shows that VoiceLive achieves over 99% detection accuracy at around 1% Equal Error Rate (EER). Results also show that VoiceLive is robust to different phone placements and is compatible to different sampling rates and phone models.

VoiceLive: A Phoneme Localization based Liveness Detection for Voice Authentication on Smartphones

This paper presents MultiTrack, a commodity WiFi based human sensing system that can track multiple users and recognize activities of multiple users performing them simultaneously. Such a system can enable easy and large-scale deployment for multi-user tracking and sensing without the need for additional sensors through the use of existing WiFi devices (e.g., desktops, laptops and smart appliances). The basic idea is to identify and extract the signal reflection corresponding to each individual user with the help of multiple WiFi links and all the available WiFi channels at 5GHz. Given the extracted signal reflection of each user, MultiTrack examines the path of the reflected signals at multiple links to simultaneously track multiple users. It further reconstructs the signal profile of each user as if only a single user has performed activity in the environment to facilitate multi-user activity recognition. We evaluate MultiTrack in different multipath environments with up to 4 users for multi-user tracking and up to 3 users for activity recognition. Experimental results show that our system can achieve decimeter localization accuracy and over 92% activity recognition accuracy under multi-user scenarios.

MultiTrack: Multi-User Tracking and Activity Recognition Using Commodity WiFi

This paper explores the potential for wearable devices to identify driving activities and unsafe driving, without relying on information or sensors in the vehicle. In particular, we study how wrist-mounted inertial sensors such as those in smart watches and fitness trackers, can track steering wheel usage and inputs. Identifying steering wheel usage helps mobile device detect driving and reduce distractions. Tracking steering wheel turning angles can improve vehicle motion tracking by mobile devices and help identify unsafe driving. The approach relies on motion features that allow distinguishing steering from other confounding hand movements. Once steering wheel usage is detected, it also use wrist rotation measurements to infer steering wheel turning angles. Our preliminary experiments show that the technique is 98.9% accurate in detecting driving and can estimate turning angles with average error within two degrees.

/pdf/toward-detection-of-unsafe-driving-with-wearables-3v7bgorsgi.pdf

Sheng Tan

Papers

WiFinger: leveraging commodity WiFi for fine-grained finger gesture recognition

Hearing Your Voice is Not Enough: An Articulatory Gesture Based Liveness Detection for Voice Authentication

VoiceLive: A Phoneme Localization based Liveness Detection for Voice Authentication on Smartphones

MultiTrack: Multi-User Tracking and Activity Recognition Using Commodity WiFi

Toward Detection of Unsafe Driving with Wearables