Vital signs are crucial indicators for human health, and researchers are studying contact-free alternatives to existing wearable vital signs sensors. Unfortunately, most of these designs demand a subject human body to be relatively static, rendering them very inconvenient to adopt in practice where body movements occur frequently. In particular, radio-frequency (RF) based contact-free sensing can be severely affected by body movements that overwhelm vital signs. To this end, we introduce MoVi-Fi as a motion-robust vital signs monitoring system, capable of recovering fine-grained vital signs waveform in a contact-free manner. Being a pure software system, MoVi-Fi can be built on top of virtually any commercial-grade radars. What inspires our design is that RF reflections caused by vital signs, albeit weak, do not totally disappear but are composited with other motion-incurred reflections in a nonlinear manner. As nonlinear blind source separation is inherently hard, MoVi-Fi innovatively employs deep contrastive learning to tackle the problem; this self-supervised method requires no ground truth in training, and it exploits contrastive signal features to distinguish vital signs from body movements. Our experiments with 12 subjects and 80hour data demonstrate that MoVi-Fi accurately recovers vital signs waveform under severe body movements.

https://dl.acm.org/doi/pdf/10.1145/3447993.3483251

MoVi-Fi: motion-robust vital signs waveform recovery via deep interpreted RF sensing

Radio-Frequency (RF) based device-free Human Activity Recognition (HAR) rises as a promising solution for many applications. However, device-free (or contactless) sensing is often more sensitive to environment changes than device-based (or wearable) sensing. Also, RF datasets strictly require on-line labeling during collection, starkly different from image and text data collections where human interpretations can be leveraged to perform off-line labeling. Therefore, existing solutions to RF-HAR entail a laborious data collection process for adapting to new environments. To this end, we propose RF-Net as a meta-learning based approach to one-shot RF-HAR; it reduces the labeling efforts for environment adaptation to the minimum level. In particular, we first examine three representative RF sensing techniques and two major meta-learning approaches. The results motivate us to innovate in two designs: i) a dual-path base HAR network, where both time and frequency domains are dedicated to learning powerful RF features including spatial and attention-based temporal ones, and ii) a metric-based meta-learning framework to enhance the fast adaption capability of the base network, including an RF-specific metric module along with a residual classification module. We conduct extensive experiments based on all three RF sensing techniques in multiple real-world indoor environments; all results strongly demonstrate the efficacy of RF-Net compared with state-of-the-art baselines.

RF-net: a unified meta-learning framework for RF-enabled one-shot human activity recognition

An ability to detect, classify, and locate complex acoustic events can be a powerful tool to help smart systems build context-awareness, e.g., to make rich inferences about human behaviors in physical spaces. Conventional methods to measure acoustic signals employ microphones as sensors. As signals from multiple acoustic sources are blended during propagation to a sensor, such methods impose a dual challenge of separating the signal for an acoustic event from background noise and from other acoustic events of interest. Recent research has proposed using radio-frequency (RF) signals, e.g., Wi-Fi and millimeter-wave (mmWave), to sense sound directly from source vibrations. Whereas these works allow separating an acoustic event from background noise, they cannot monitor multiple sound sources simultaneously. In this paper, we present UWHear, a system that simultaneously recovers and separates sounds from multiple sources. Unlike previous works using continuous-wave RF, UWHear employs Impulse Radio Ultra-Wideband (IR-UWB) technology, in order to construct an enhanced audio sensing system tackling the above challenges. Further, IR-UWB radios can penetrate light building materials, which enables UWHear to operate in some non-line-of-sight (NLOS) conditions. In addition to providing a theoretical guarantee for audio recovery using RF pulses, we also implement an audio sensing prototype exploiting a commercial-off-the-shelf IR-UWB radar. Our experiments show that UWHear can effectively separate the content of two speakers that are placed only 25cm apart. UWHear can also capture and separate multiple sounds and vibrations of household appliances while being immune to non-target noise coming from other directions.

https://dl.acm.org/doi/pdf/10.1145/3384419.3430772

UWHear: through-wall extraction and separation of audio vibrations using wireless signals

Deep learning has been increasingly applied to improve human activity recognition (HAR) accuracy and reduce the human efforts of handcrafted feature extractions. Federated Learning (FL) is an emerging learning paradigm that enables the collaborative learning of a global model without exposing users' raw data. However, existing FL approaches yield unsatisfactory HAR performance as they fail to dynamically aggregate models according to the statistical diversity of users' data. In this paper, we propose FedDL, a novel federated learning system for HAR that can capture the underlying user relationships and apply them to learn personalized models for different users dynamically. Specifically, we design a dynamic layer sharing scheme that learns the similarity among users' model weights to form the sharing structure and merges models accordingly in an iterative, bottom-up layer-wise manner. FedDL merges local models based on the dynamic sharing scheme, significantly speeding up the convergence while maintaining high accuracy. We have implemented FedDL and evaluated using a new data set we collected using LiDAR and four public real-world datasets involving 178 users in total. The results show that FedDL outperforms several state-of-the-art FL paradigms in terms of model accuracy (by more than 15%), converging rate (by more than 70%), and communication overhead (about 30% reduction). Moreover, the testing results on the datasets of different scales show that FedDL has high scalability and hence can be deployed for large-scale real-world applications.

https://dl.acm.org/doi/pdf/10.1145/3485730.3485946

FedDL: Federated Learning via Dynamic Layer Sharing for Human Activity Recognition

Crucial for healthcare and biomedical applications, respiration monitoring often employs wearable sensors in practice, causing inconvenience due to their direct contact with human bodies. Therefore, researchers have been constantly searching for contact-free alternatives. Nonetheless, existing contact-free designs mostly require human subjects to remain static, largely confining their adoptions in everyday environments where body movements are inevitable. Fortunately, radio-frequency (RF) enabled contact-free sensing, though suffering motion interference inseparable by conventional filtering, may offer a potential to distill respiratory waveform with the help of deep learning. To realize this potential, we introduce MoRe-Fi to conduct fine-grained respiration monitoring under body movements. MoRe-Fi leverages an IR-UWB radar to achieve contact-free sensing, and it fully exploits the complex radar signal for data augmentation. The core of MoRe-Fi is a novel variational encoder-decoder network; it aims to single out the respiratory waveforms that are modulated by body movements in a non-linear manner. Our experiments with 12 subjects and 66-hour data demonstrate that MoRe-Fi accurately recovers respiratory waveform despite the interference caused by body movements. We also discuss potential applications of MoRe-Fi for pulmonary disease diagnoses.

MoRe-Fi: Motion-robust and Fine-grained Respiration Monitoring via Deep-Learning UWB Radar

Given the significant amount of time people spend in vehicles, health issues under driving condition have become a major concern. Such issues may vary from fatigue, asthma, stroke, to even heart attack, yet they can be adequately indicated by vital signs and abnormal activities. Therefore, in-vehicle vital sign monitoring can help us predict and hence prevent these issues. Whereas existing sensor-based (including camera) methods could be used to detect these indicators, privacy concern and system complexity both call for a convenient yet effective and robust alternative. This paper aims to develop V2iFi, an intelligent system performing monitoring tasks using a COTS impulse radio mounted on the windshield. V2iFi is capable of reliably detecting driver's vital signs under driving condition and with the presence of passengers, thus allowing for potentially inferring corresponding health issues. Compared with prior work based on Wi-Fi CSI, V2iFi is able to distinguish reflected signals from multiple users, and hence provide finer-grained measurements under more realistic settings. We evaluate V2iFi both in lab environments and during real-life road tests; the results demonstrate that respiratory rate, heart rate, and heart rate variability can all be estimated accurately. Based on these estimation results, we further discuss how machine learning models can be applied on top of V2iFi so as to improve both physiological and psychological wellbeing in driving environments.

V2iFi: in-Vehicle Vital Sign Monitoring via Compact RF Sensing

In recent years, radio frequency (RF) sensing has gained increasing popularity due to its pervasiveness, low cost, non-intrusiveness, and privacy preservation. However, realizing the promises of RF sensing is highly nontrivial, given typical challenges such as multipath and interference. One potential solution leverages deep learning to build direct mappings from the RF domain to target domains, hence avoiding complex RF physical modeling. While earlier solutions exploit only simple feature extraction and classification modules, an emerging trend adds functional layers on top of elementary modules for more powerful generalizability and flexible applicability. To better understand this potential, this article takes a layered approach to summarize RF sensing enabled by deep learning. Essentially, we present a four-layer framework: physical, backbone, generalization, and application. While this layered framework provides readers a systematic methodology for designing deep interpreted RF sensing, it also facilitates making improvement proposals and hints at future research opportunities.

Tianyue Zheng

Papers

V2iFi: in-Vehicle Vital Sign Monitoring via Compact RF Sensing

MoVi-Fi: motion-robust vital signs waveform recovery via deep interpreted RF sensing

RF-net: a unified meta-learning framework for RF-enabled one-shot human activity recognition

MoRe-Fi: Motion-robust and Fine-grained Respiration Monitoring via Deep-Learning UWB Radar

Enhancing RF Sensing with Deep Learning: A Layered Approach