Digital twin can be defined as a virtual representation of a physical asset enabled through data and simulators for real-time prediction, optimization, monitoring, controlling, and improved decision making. Recent advances in computational pipelines, multiphysics solvers, artificial intelligence, big data cybernetics, data processing and management tools bring the promise of digital twins and their impact on society closer to reality. Digital twinning is now an important and emerging trend in many applications. Also referred to as a computational megamodel, device shadow, mirrored system, avatar or a synchronized virtual prototype, there can be no doubt that a digital twin plays a transformative role not only in how we design and operate cyber-physical intelligent systems, but also in how we advance the modularity of multi-disciplinary systems to tackle fundamental barriers not addressed by the current, evolutionary modeling practices. In this work, we review the recent status of methodologies and techniques related to the construction of digital twins mostly from a modeling perspective. Our aim is to provide a detailed coverage of the current challenges and enabling technologies along with recommendations and reflections for various stakeholders.

/pdf/digital-twin-values-challenges-and-enablers-from-a-modeling-3hpoelaogn.pdf

Digital Twin: Values, Challenges and Enablers From a Modeling Perspective

This paper proposes a novel machine learning architecture, specifically designed for radio-frequency based gesture recognition. We focus on high-frequency (60]GHz), short-range radar based sensing, in particular Google's Soli sensor. The signal has unique properties such as resolving motion at a very fine level and allowing for segmentation in range and velocity spaces rather than image space. This enables recognition of new types of inputs but poses significant difficulties for the design of input recognition algorithms. The proposed algorithm is capable of detecting a rich set of dynamic gestures and can resolve small motions of fingers in fine detail. Our technique is based on an end-to-end trained combination of deep convolutional and recurrent neural networks. The algorithm achieves high recognition rates (avg 87%) on a challenging set of 11 dynamic gestures and generalizes well across 10 users. The proposed model runs on commodity hardware at 140 Hz (CPU only).

https://dl.acm.org/doi/pdf/10.1145/2984511.2984565

Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition in the Radio-Frequency Spectrum

Driven by a wide range of real-world applications, significant efforts have recently been made to explore device-free human activity recognition techniques that utilize the information collected by various wireless infrastructures to infer human activities without the need for the monitored subject to carry a dedicated device. Existing device free human activity recognition approaches and systems, though yielding reasonably good performance in certain cases, are faced with a major challenge. The wireless signals arriving at the receiving devices usually carry substantial information that is specific to the environment where the activities are recorded and the human subject who conducts the activities. Due to this reason, an activity recognition model that is trained on a specific subject in a specific environment typically does not work well when being applied to predict another subject's activities that are recorded in a different environment. To address this challenge, in this paper, we propose EI, a deep-learning based device free activity recognition framework that can remove the environment and subject specific information contained in the activity data and extract environment/subject-independent features shared by the data collected on different subjects under different environments. We conduct extensive experiments on four different device free activity recognition testbeds: WiFi, ultrasound, 60 GHz mmWave, and visible light. The experimental results demonstrate the superior effectiveness and generalizability of the proposed EI framework.

https://dl.acm.org/doi/pdf/10.1145/3241539.3241548

Towards Environment Independent Device Free Human Activity Recognition

Since human bodies are good reflectors of wireless signals, human activities can be recognized by monitoring changes in WiFi signals. However, existing WiFi-based human activity recognition systems do not build models that can quantify the correlation between WiFi signal dynamics and human activities. In this paper, we propose a Channel State Information (CSI)-based human Activity Recognition and Monitoring system (CARM). CARM is based on two theoretical models. First, we propose a CSI-speed model that quantifies the relation between CSI dynamics and human movement speeds. Second, we propose a CSI-activity model that quantifies the relation between human movement speeds and human activities. Based on these two models, we implemented the CARM on commercial WiFi devices. Our experimental results show that the CARM achieves recognition accuracy of 96% and is robust to environmental changes.

Device-Free Human Activity Recognition Using Commercial WiFi Devices

Synergistic design of communications and radar systems with common spectral and hardware resources is heralding a new era of efficiently utilizing a limited radio-frequency (RF) spectrum. Such a joint radar communications (JRC) model has advantages of low cost, compact size, less power consumption, spectrum sharing, improved performance, and safety due to enhanced information sharing. Today, millimeter-wave (mmwave) communications have emerged as the preferred technology for short distance wireless links because they provide transmission bandwidth that is several gigahertz wide. This band is also promising for short-range radar applications, which benefit from the high-range resolution arising from large transmit signal bandwidths. Signal processing techniques are critical to the implementation of mm-wave JRC systems. Major challenges are joint waveform design and performance criteria that would optimally trade off between communications and radar functionalities. Novel multiple-input, multiple-output (MIMO) signal processing techniques are required because mm-wave JRC systems employ large antenna arrays. There are opportunities to exploit recent advances in cognition, compressed sensing, and machine learning to reduce required resources and dynamically allocate them with low overheads. This article provides a signal processing perspective of mm-wave JRC systems with an emphasis on waveform design.

/pdf/toward-millimeter-wave-joint-radar-communications-a-signal-4gn576kyat.pdf

Toward Millimeter-Wave Joint Radar Communications: A signal processing perspective

This paper presents Soli, a new, robust, high-resolution, low-power, miniature gesture sensing technology for human-computer interaction based on millimeter-wave radar. We describe a new approach to developing a radar-based sensor optimized for human-computer interaction, building the sensor architecture from the ground up with the inclusion of radar design principles, high temporal resolution gesture tracking, a hardware abstraction layer (HAL), a solid-state radar chip and system architecture, interaction models and gesture vocabularies, and gesture recognition. We demonstrate that Soli can be used for robust gesture recognition and can track gestures with sub-millimeter accuracy, running at over 10,000 frames per second on embedded hardware.

/pdf/soli-ubiquitous-gesture-sensing-with-millimeter-wave-radar-mcplexjo41.pdf

Soli: ubiquitous gesture sensing with millimeter wave radar

This document describes techniques for radio frequency (RF) based micro-motion tracking. These techniques enable even millimeter-scale hand motions to be tracked. To do so, radar signals are used from radar systems that, with conventional techniques, would only permit resolutions of a centimeter or more.

RF-Based Micro-Motion Tracking for Gesture Tracking and Recognition

This document describes techniques and devices for type-agnostic radio frequency (RF) signal representations. These techniques and devices enable use of multiple different types of radar systems and fields through type-agnostic RF signal representations. By so doing, recognition and application-layer analysis can be independent of various radar parameters that differ between different radar systems and fields.

Type-agnostic RF signal representations

Techniques and apparatuses are described that enable power management using a low-power radar The described techniques enable a radar system to reduce overall power consumption, thereby facilitating incorporation and utilization of the radar system within power-limited devices In one aspect, the radar system can replace other power-hungry sensors and provide improved performance in the presence of different environmental conditions, such as low lighting, motion, or overlapping targets In another aspect, the radar system can cause other components within the electronic device to switch to an off-state based on detected activity in an external environment By actively switching the components between an on-state or the off-state, the radar system enables the computing device to respond to changes in the external environment without the use of an automatic shut-off timer or a physical touch or verbal command from a user

Power management using a low-power radar

Techniques and apparatuses are described that enable low-power radar. The described techniques enable a radar system to reduce overall power consumption, thereby facilitating incorporation and utilization of the radar system within power-limited devices. Power consumption is reduced through customization of the transmission or processing of radar signals within the radar system. During transmission, different duty cycles, transmit powers, or framing structures can be utilized to collect appropriate data based on detected activity in an external environment. During processing, different hardware or different radar pipelines can be utilized to appropriately analyze the radar data. Instead of disabling the radar system, the described techniques enable the radar system to continuously monitor a dynamic environment and maintain responsiveness while conserving power.

Amihood Patrick M

Papers

Soli: ubiquitous gesture sensing with millimeter wave radar

RF-Based Micro-Motion Tracking for Gesture Tracking and Recognition

Type-agnostic RF signal representations

Power management using a low-power radar

Low-Power Radar