Recently, device-free human behavior recognition has become a hot research topic and has achieved significant progress in the field of ubiquitous computing. Among various implementation, behavior recognition based on WiFi CSI (channel state information) has drawn wide attention due to its major advantages. This paper investigates more than 100 latest CSI based behavior recognition applications within the last 6 years and presents a comprehensive survey from every aspect of human behavior recognition. Firstly, this paper reviews general behavior recognition applications using the WiFi signal and presents the basic concept of CSI and the fundamental principle of CSI-based behavior recognition. This paper analyzes the key components and core characteristics of the system architecture of human behavior recognition using CSI. Afterward, we divide the sensing procedures into many steps and summarize the typical studies from these steps, including base signal selection, signal preprocessing, and identification approaches. Next, based on the recognition technique, we classify the applications into three groups, including pattern-based, model-based, and deep learning-based approach. In every group, we categorize the state-of-the-art applications into three groups, including coarse-grained specific behavior recognition, fine-grained specific behavior recognition, and activity inference. It elaborates the typical behavior recognition applications from five aspects, including experimental equipment, experimental scenario, behavior, classifier, and system performance. Then, this paper presents comprehensive discussions about representative applications from the implementation view and outlines the major consideration when developing a recognition system. Finally, this article concludes by analyzing the open issues of CSI-based behavior recognition applications and pointing out future research directions.

/pdf/a-survey-on-human-behavior-recognition-using-channel-state-17xs5detky.pdf

A Survey on Human Behavior Recognition Using Channel State Information

In recent years, wireless sensing has been exploited as a promising research direction for contactless human activity recognition. However, one major issue hindering the real deployment of these systems is that the signal variation patterns induced by the human activities with different devices and environmental settings are neither stable nor consistent, resulting in unstable system performance. The existing machine learning based methods usually take the "black box" approach and fails to achieve consistent performance. In this paper, we argue that a deep understanding of radio signal propagation in wireless sensing is needed, and it may be possible to develop a deterministic sensing model to make the signal variation patterns predictable. With this intuition, in this paper we investigate: 1) how wireless signals are affected by human activities taking transceiver location and environment settings into consideration; 2) a new deterministic sensing approach to model the received signal variation patterns for different human activities; 3) a proof-of-concept prototype to demonstrate our approach and a case study to detect diverse activities. In particular, we propose a diffraction-based sensing model to quantitatively determine the signal change with respect to a target's motions, which eventually links signal variation patterns with motions, and hence can be used to recognize human activities. Through our case study, we demonstrate that the diffraction-based sensing model is effective and robust in recognizing exercises and daily activities. In addition, we demonstrate that the proposed model improves the recognition accuracy of existing machine learning systems by above 10%.

Towards a Diffraction-based Sensing Approach on Human Activity Recognition

Human activity recognition is a key to a lot of applications such as healthcare and smart home. In this study, we provide a comprehensive survey on recent advances and challenges in human activity recognition (HAR) with deep learning. Although there are many surveys on HAR, they focused mainly on the taxonomy of HAR and reviewed the state-of-the-art HAR systems implemented with conventional machine learning methods. Recently, several works have also been done on reviewing studies that use deep models for HAR, whereas these works cover few deep models and their variants. There is still a need for a comprehensive and in-depth survey on HAR with recently developed deep learning methods.

A Survey on Deep Learning for Human Activity Recognition

While decade-long research has clearly demonstrated the vast potential of radio frequency (RF) for many human sensing tasks, scaling this technology to large scenarios remained problematic with conventional approaches. Recently, researchers have successfully applied deep learning to take radio-based sensing to a new level. Many different types of deep learning models have been proposed to achieve high sensing accuracy over a large population and activity set, as well as in unseen environments. Deep learning has also enabled detection of novel human sensing phenomena that were previously not possible. In this survey, we provide a comprehensive review and taxonomy of recent research efforts on deep learning based RF sensing. We also identify and compare several publicly released labeled RF sensing datasets that can facilitate such deep learning research. Finally, we summarize the lessons learned and discuss the current limitations and future directions of deep learning based RF sensing.

Deep Learning for Radio-Based Human Sensing: Recent Advances and Future Directions

Wireless sensing has become a hot research topic recently, enabling a large range of applications. However, due to the intrinsic nature of employing weak target-reflection signal for sensing, the sensing range is limited. Another issue is the strong interference from surroundings and therefore a lot of wireless sensing systems assume there is no interferer in the environment. One recent work explored the possibility of employing LoRa signal for long range sensing which is a favorable step in addressing the first issue. However, the interference issue becomes even more severe with LoRa due to its larger sensing range. In this paper, we propose Sen-fence - a LoRa-based sensing system - to significantly increase the sensing range and at the same time mitigate the interference. With careful signal processing, Sen-fence is able to maximize the movement-induced signal variation in software to increase the sensing range. To address the interference issue, we propose the concept of "virtual fence" to constrain sensing only within the area of interest. The location and size of virtual fence can be flexibly controlled in software to meet the requirements of different applications. Sen-fence is able to (i) achieve a 50 m sensing range for fine-grained human respiration, which is twice the state-of-the-art; and (ii) efficiently mitigate the interference to make LoRa sensing work in practice.

Combating interference for long range LoRa sensing

Wireless signals have been extensively utilized for contactless sensing in the past few years. Due to the intrinsic nature of employing the weak target-reflected signal for sensing, the sensing range is limited. For instance, WiFi and RFID can achieve 3-6 meter sensing range while acoustic-based sensing is limited to less than one meter. In this work, we identify exciting sensing opportunities with LoRa, which is the new long-range communication technology designed for IoT communication. We explore the sensing capability of LoRa, both theoretically and experimentally. We develop the sensing model to characterize the relationship between target movement and signal variation, and propose novel techniques to increase LoRa sensing range to over 25 meters for human respiration sensing. We further build a prototype system which is capable of sensing both coarse-grained and fine-grained human activities. Experimental results show that (1) human respiration can still be sensed when the target is 25 meters away from the LoRa devices, and 15 meters away with a wall in between; and (2) human walking (both displacement and direction) can be tracked accurately even when the target is 30 meters away from the LoRa transceiver pair.

Exploring LoRa for Long-range Through-wall Sensing

Despite extensive research effort in contact-free sensing using RF signals in the last few years, there still exist significant barriers preventing their wide adoptions. One key issue is the inability to sense multiple targets due to the intrinsic nature of relying on reflection signals for sensing: the reflections from multiple targets get mixed at the receiver and it is extremely difficult to separate these signals to sense each individual. This problem becomes even more severe in long-range LoRa sensing because the sensing range is much larger compared to WiFi and acoustic based sensing. In this work, we address the challenging multi-target sensing issue, moving LoRa sensing one big step towards practical adoption. The key idea is to effectively utilize multiple antennas at the LoRa gateway to enable spatial beamforming to support multi-target sensing. While traditional beamforming methods adopted in WiFi and Radar systems rely on accurate channel information or transmitter-receiver synchronization, these requirements can not be satisfied in LoRa systems: the transmitter and receiver are not synchronized and no channel state information can be obtained from the cheap LoRa nodes. Another interesting observation is that while beamforming helps to increase signal strength, the phase/amplitude information which is critical for sensing can get corrupted during the beamforming process, eventually compromising the sensing capability. In this paper, we propose novel signal processing methods to address the issues above to enable long-range multi-target reparation sensing with LoRa. Extensive experiments show that our system can monitor the respiration rates of five human targets simultaneously at an average accuracy of 98.1%.

/pdf/unlocking-the-beamforming-potential-of-lora-for-long-range-sh2o21f8z8.pdf

Unlocking the Beamforming Potential of LoRa for Long-range Multi-target Respiration Sensing

Recent work has revealed the sensing theory of human respiration outside the First Fresnel Zone (FFZ) using commodity Wi-Fi devices. However, there is still no theoretical model to guide human respiration detection when the subject locates in the FFZ. In our work [10], we propose a diffraction-based sensing model to investigate how to effectively sense human respiration in FFZ. We present this demo system to show human respiration sensing performance varies based on different human locations and postures. By deploying the respiration detection system using COTS Wi-Fi devices, we can observe that the respiration sensing results match the theoretical model well.

A Fresnel Diffraction Model Based Human Respiration Detection System Using COTS Wi-Fi Devices

Soil moisture sensing is one of the most important components in smart agriculture. It plays a critical role in increasing crop yields and reducing water waste. However, existing commercial soil moisture sensors are either expensive or inaccurate, limiting their real-world deployment. In this paper, we utilize wide-area LoRa signals to sense soil moisture without a need of dedicated soil moisture sensors. Different from traditional usage of LoRa in smart agriculture which is only for sensor data transmission, we leverage LoRa signal itself as a powerful sensing tool. The key insight is that the dielectric permittivity of soil which is closely related to soil moisture can be obtained from phase readings of LoRa signals. Therefore, antennas of a LoRa node can be placed in the soil to capture signal phase readings for soil moisture measurements. Though promising, it is non-trivial to extract accurate phase information due to unsynchronization of LoRa transmitter and receiver. In this work, we propose to include a low-cost switch to equip the LoRa node with two antennas to address the issue. We develop a delicate chirp ratio approach to cancel out the phase offset caused by transceiver unsynchronization to extract accurate phase information. The proposed system design has multiple unique advantages including high accuracy, robustness against motion interference and large sensing range for large-scale deployment in smart agriculture. Experiments with commodity LoRa nodes show that our system can accurately estimate soil moisture at an average error of 3.1%, achieving a performance comparable to high-end commodity soil moisture sensors. Field studies show that the proposed system can accurately sense soil moisture even when the LoRa gateway is 100 m away from the LoRa node, enabling wide-area soil moisture sensing for the first time.

Sensor-free Soil Moisture Sensing Using LoRa Signals

While GPS-based outdoor localization has become a norm, very few indoor localization systems have been deployed and used. In this paper, we share our 5-year experience on the design, development and evaluation of a large-scale WiFi indoor localization system. We address practical challenges encountered to bridge the gap between indoor localization research in the laboratory and system deployment in the wild. The system is currently used in 1469 shopping malls, 393 office buildings and 35 hospitals across 35 cities to provide location service to millions of users on a daily basis. We hope the shared experience can benefit the design of real-world indoor localization systems and the practical problems identified can change the focus of indoor localization research. We released our dataset that contains fingerprints collected from 1469 shopping malls and one office building.

/pdf/experience-pushing-indoor-localization-from-laboratory-to-6mz5w566.pdf

Zhaoxin Chang

Papers

Exploring LoRa for Long-range Through-wall Sensing

Unlocking the Beamforming Potential of LoRa for Long-range Multi-target Respiration Sensing

A Fresnel Diffraction Model Based Human Respiration Detection System Using COTS Wi-Fi Devices

Sensor-free Soil Moisture Sensing Using LoRa Signals

Experience: pushing indoor localization from laboratory to the wild