Deep learning performs remarkably well on many time series analysis tasks recently. The superior performance of deep neural networks relies heavily on a large number of training data to avoid overfitting. However, the labeled data of many real-world time series applications may be limited such as classification in medical time series and anomaly detection in AIOps. As an effective way to enhance the size and quality of the training data, data augmentation is crucial to the successful application of deep learning models on time series data. In this paper, we systematically review different data augmentation methods for time series. We propose a taxonomy for the reviewed methods, and then provide a structured review for these methods by highlighting their strengths and limitations. We also empirically compare different data augmentation methods for different tasks including time series anomaly detection, classification, and forecasting. Finally, we discuss and highlight five future directions to provide useful research guidance.

/pdf/time-series-data-augmentation-for-deep-learning-a-survey-8b9ejcq10z.pdf

Time Series Data Augmentation for Deep Learning: A Survey

For more than two decades, capacitive sensing has played a prominent role in human-computer interaction research. Capacitive sensing has become ubiquitous on mobile, wearable, and stationary devices - enabling fundamentally new interaction techniques on, above, and around them. The research community has also enabled human position estimation and whole-body gestural interaction in instrumented environments. However, the broad field of capacitive sensing research has become fragmented by different approaches and terminology used across the various domains. This paper strives to unify the field by advocating consistent terminology and proposing a new taxonomy to classify capacitive sensing approaches. Our extensive survey provides an analysis and review of past research and identifies challenges for future work. We aim to create a common understanding within the field of human-computer interaction, for researchers and practitioners alike, and to stimulate and facilitate future research in capacitive sensing.

/pdf/finding-common-ground-a-survey-of-capacitive-sensing-in-5aw8g2v2ro.pdf

Finding Common Ground: A Survey of Capacitive Sensing in Human-Computer Interaction

Template matching is an alternative to perform speech recognition. However, the template matching encountered problems due to speaking rate variability, in which there exist timing differences between the two utterances. Speech has a constantly changing signal, thus it is almost impossible to get the same signal for two same utterances. The problem of time differences can be solved through DTW algorithm: warping the template with the test utterance based on their similarities. So, DTW algorithm actually is a procedure, which combines both warping and distance measurement. DTW is considered as one effective method in speech pattern recognition, however the bad side of this method is that it requires a long processing time plus large storage capacity, especially for real time recognitions.

Dynamic time warping

Combining machine learning in neural networks with multimodal fusion strategies offers an interesting potential for classification tasks but the optimum fusion strategies for many applications have yet to be determined. Here we address this issue in the context of human activity recognition, making use of a state-of-the-art convolutional network architecture (Inception I3D) and a huge dataset (NTU RGB+D). As modalities we consider RGB video, optical flow, and skeleton data. We determine whether the fusion of different modalities can provide an advantage as compared to uni-modal approaches, and whether a more complex early fusion strategy can outperform the simpler late-fusion strategy by making use of statistical correlations between the different modalities. Our results show a clear performance improvement by multi-modal fusion and a substantial advantage of an early fusion strategy,

Early vs Late Fusion in Multimodal Convolutional Neural Networks

Capacitive proximity sensors (CPSs) are ubiquitous because of their simple design, low cost and low consumption. Capacitive displacement sensing, as one of the three sensing modalities, works for long distance and can be unitized to measure more physical quantities compared with capacitive volume and deformation sensing. In this paper, we firstly introduce the concept of capacitive displacement sensing. After that, we present applications of capacitive displacement sensing under three broad categories: distance measurements, indirect measurements, and the applications applied in smart environments. Finally, we discuss the challenges and possible solutions for CPSs development. We show that both the detection range and accuracy of CPS can be improved by multi-sensor fusion, and the application scenarios can be extensive through machine/deep learning approaches. We aim to provide a comprehensive, and state-of-the-art review of the capacitive displacement sensing, and inspire more researchers and developers to find wide application perspectives.

/pdf/a-review-on-applications-of-capacitive-displacement-sensing-2exh6zirht.pdf

A Review on Applications of Capacitive Displacement Sensing for Capacitive Proximity Sensor

Sensors are devices that quantify the physical aspects of the world around us. This ability is important to gain knowledge about human activities. Human Activity recognition plays an import role in people's everyday life. In order to solve many human-centered problems, such as health care, and individual assistance, the need to infer various simple to complex human activities is prominent. Therefore, having a well defined categorization of sensing technology is essential for the systematic design of human activity recognition systems. By extending the sensor categorization proposed by White, we survey the most prominent research works that utilize different sensing technologies for human activity recognition tasks. To the best of our knowledge, there is no thorough sensor-driven survey that considers all sensor categories in the domain of human activity recognition with respect to the sampled physical properties, including a detailed comparison across sensor categories. Thus, our contribution is to close this gap by providing an insight into the state-of-the-art developments. We identify the limitations with respect to the hardware and software characteristics of each sensor category and draw comparisons based on benchmark features retrieved from the research works introduced in this survey. Finally, we conclude with general remarks and provide future research directions for human activity recognition within the presented sensor categorization.

/pdf/sensing-technology-for-human-activity-recognition-a-26dwl81jg0.pdf

Sensing Technology for Human Activity Recognition: A Comprehensive Survey

Platypus is the first system to localize and identify people by remotely and passively sensing changes in their body electric potential which occur naturally during walking. While it uses three or more electric potential sensors with a maximum range of 2 m, as a tag-free system it does not require the user to carry any special hardware. We describe the physical principles behind body electric potential changes, and a predictive mathematical model of how this affects a passive electric field sensor. By inverting this model and combining data from sensors, we infer a method for localizing people and experimentally demonstrate a median localization error of 0.16 m. We also use the model to remotely infer the change in body electric potential with a mean error of 8.8 % compared to direct contact-based measurements. We show how the reconstructed body electric potential differs from person to person and thereby how to perform identification. Based on short walking sequences of 5 s, we identify four users with an accuracy of 94 %, and 30 users with an accuracy of 75 %. We demonstrate that identification features are valid over multiple days, though change with footwear.

/pdf/platypus-indoor-localization-and-identification-through-37x8kargi5.pdf

Platypus: Indoor Localization and Identification through Sensing of Electric Potential Changes in Human Bodies

Location-based services or smart home applications all depend on an accurate indoor positioning system. Basically one divides these systems into token-based and token-free localization systems. In this work, we focus on the token-free system based on smart floor technology. Smart floors can typically be built using pressure sensors or capacitive sensors. However, these set-ups are often hard to deploy as mechanical or electrical features are required below the surface and even harder to replace when detected a sensor malfunctioning. Therefore we present a novel indoor positioning system using an uncommon form of passive electric field sensing (EPS), which detects the electric potential variation caused by body movement. The EPS-based smart floor set-up is easy to install by deploying a grid of passive electrode wires underneath any non-conductive surfaces. Easy maintenance is also ensured by the fact that the sensors are not placed underneath the surface, but on the side. Due to the passive measuring nature, low power consumption is achieved as opposed to active capacitive measurement. Since we do not collect image data as in visual-based systems and all sensor data is processed locally, we preserve the user’s privacy. The proposed architecture achieves a high position accuracy and an excellent spatial resolution. Based on our evaluation conducted in our living lab, we measure a mean positioning error of only 12.7 cm.

Performing indoor localization with electric potential sensing

Nowadays activity recognition on smartphones is ubiquitously applied, for example to monitor personal health. The smartphone's sensors act as a foundation to provide information on movements, the user's location or direction. Incorporating ultrasound sensing using the smartphone's native speaker and microphone provides additional means for perceiving the environment and humans. In this paper, we outline possible usage scenarios for this new and promising sensing modality. Based on a custom implementation, we provide results on various experiments to assess the opportunities for activity recognition systems. We discuss various limitations and possibilities when wearing the smartphone on the human body. In stationary deployments, e.g. while placed on a night desk, our implementation is able to detect movements in proximities up to 2m as well as discern several gestures performed above the phone.

/pdf/opportunities-for-activity-recognition-using-ultrasound-20x2gefmyl.pdf

Opportunities for activity recognition using ultrasound doppler sensing on unmodified mobile phones

Quantified Self has seen an increased interest in recent years, with devices including smartwatches, smartphones, or other wearables that allow you to monitor your fitness level. This is often combined with mobile apps that use gamification aspects to motivate the user to perform fitness activities, or increase the amount of sports exercise. Thus far, most applications rely on accelerometers or gyroscopes that are integrated into the devices. They have to be worn on the body to track activities. In this work, we investigated the use of a speaker and a microphone that are integrated into a smartphone to track exercises performed close to it. We combined active sonar and Doppler signal analysis in the ultrasound spectrum that is not perceivable by humans. We wanted to measure the body weight exercises bicycles, toe touches, and squats, as these consist of challenging radial movements towards the measuring device. We have tested several classification methods, ranging from support vector machines to convolutional neural networks. We achieved an accuracy of 88% for bicycles, 97% for toe-touches and 91% for squats on our test set.

https://www.mdpi.com/2227-9709/5/2/24/pdf

Biying Fu

Papers

Sensing Technology for Human Activity Recognition: A Comprehensive Survey

Platypus: Indoor Localization and Identification through Sensing of Electric Potential Changes in Human Bodies

Performing indoor localization with electric potential sensing

Opportunities for activity recognition using ultrasound doppler sensing on unmodified mobile phones

Fitness Activity Recognition on Smartphones Using Doppler Measurements