Discovering patterns in time series data is essential to many key tasks in intelligent sensing systems, such as human activity recognition and event detection. These tasks involve the classification of sensory information from physical measurements such as inertial or temperature change measurements. Due to differences in the underlying physics, existing methods for classification use handcrafted features combined with traditional learning algorithms, or employ distinct deep neural models to directly learn from raw data. We propose here a unified neural architecture, UniTS, for sensory time series classification in various tasks, which obviates the need for domain-specific feature, model customization or polished hyper-parameter tuning. This is possible as we believe that discriminative patterns in sensory measurements would manifest when we combine information from both the time and frequency domains. In particular, to reveal the commonality of sensory signals, we integrate Short-Time Fourier Transform (STFT) into neural networks by initializing convolutional filter weights as the Fourier coefficients. Instead of treating STFT as a static linear transform with fixed coefficients, we make these weights optimizable during network training, which essentially learns to weigh each frequency channel. Recognizing that time-domain signals might represent intuitive physics such as temperature and acceleration, we combine linearly transformed time-domain hidden features with the frequency components within each time chunk. We further extend our model to multiple branches with different time-frequency resolutions to avoid the need of hyper-parameter search. We conducted experiments on four public datasets containing time-series data from various IoT systems, including motion, WiFi, EEG, and air quality, and compared UniTS with numerous recent models. Results demonstrate that our proposed method achieves an average F1 score of 91.85% with a 2.3-point improvement over the state of the art. We also verified the efficacy of STFT-inspired structures through numerous quantitative studies.

https://dl.acm.org/doi/pdf/10.1145/3485730.3485942

UniTS: Short-Time Fourier Inspired Neural Networks for Sensory Time Series Classification

In recent years, the adoption of machine learning 
has grown steadily in different fields affecting the day-to-day decisions of individuals. This paper presents an intelligent system for recognizing human’s daily activities in a complex IoT environment. An enhanced model of capsule neural network called 1D-HARCapsNe is proposed. This proposed model consists of convolution layer, primary capsule layer, activity capsules flat layer and output layer. It is validated using WISDM dataset collected via smart devices and normalized using the random-SMOTE algorithm to handle the imbalanced behavior of the dataset. The experimental results indicate the potential and strengths of the proposed 1D-HARCapsNet that achieved enhanced performance with an accuracy of 98.67%, precision of 98.66%, recall of 98.67%, and F1-measure of 0.987 which shows major performance enhancement compared to the Conventional CapsNet (accuracy 90.11%, precision 91.88%, recall 89.94%, and F1-measure 0.93).

/pdf/intelligent-system-for-human-activity-recognition-in-iot-28mzbovcce.pdf

Intelligent system for human activity recognition in IoT environment

Human activity recognition (HAR) has emerged as a significant area of research due to its numerous possible applications, including ambient assisted living, healthcare, abnormal behaviour detection, etc. Recently, HAR using WiFi channel state information (CSI) has become a predominant and unique approach in indoor environments compared to others (i.e., sensor and vision) due to its privacy-preserving qualities, thereby eliminating the need to carry additional devices and providing flexibility of capture motions in both line-of-sight (LOS) and non-line-of-sight (NLOS) settings. Existing deep learning (DL)-based HAR approaches usually extract either temporal or spatial features and lack adequate means to integrate and utilize the two simultaneously, making it challenging to recognize different activities accurately. Motivated by this, we propose a novel DL-based model named spatio-temporal convolution with nested long short-term memory (STC-NLSTMNet), with the ability to extract spatial and temporal features concurrently and automatically recognize human activity with very high accuracy. The proposed STC-NLSTMNet model is mainly comprised of depthwise separable convolution (DS-Conv) blocks, feature attention module (FAM) and NLSTM. The DS-Conv blocks extract the spatial features from the CSI signal and add feature attention modules (FAM) to draw attention to the most essential features. These robust features are fed into NLSTM as inputs to explore the hidden intrinsic temporal features in CSI signals. The proposed STC-NLSTMNet model is evaluated using two publicly available datasets: Multi-environment and StanWiFi. The experimental results revealed that the STC-NLSTMNet model achieved activity recognition accuracies of 98.20% and 99.88% on Multi-environment and StanWiFi datasets, respectively. Its activity recognition performance is also compared with other existing approaches and our proposed STC-NLSTMNet model significantly improves the activity recognition accuracies by 4% and 1.88%, respectively, compared to the best existing method.

/pdf/stc-nlstmnet-an-improved-human-activity-recognition-method-39cat9ax.pdf

STC-NLSTMNet: An Improved Human Activity Recognition Method Using Convolutional Neural Network with NLSTM from WiFi CSI

Smart living, an increasingly prominent concept, entails incorporating sophisticated technologies in homes and urban environments to elevate the quality of life for citizens. A critical success factor for smart living services and applications, from energy management to healthcare and transportation, is the efficacy of human action recognition (HAR). HAR, rooted in computer vision, seeks to identify human actions and activities using visual data and various sensor modalities. This paper extensively reviews the literature on HAR in smart living services and applications, amalgamating key contributions and challenges while providing insights into future research directions. The review delves into the essential aspects of smart living, the state of the art in HAR, and the potential societal implications of this technology. Moreover, the paper meticulously examines the primary application sectors in smart living that stand to gain from HAR, such as smart homes, smart healthcare, and smart cities. By underscoring the significance of the four dimensions of context awareness, data availability, personalization, and privacy in HAR, this paper offers a comprehensive resource for researchers and practitioners striving to advance smart living services and applications. The methodology for this literature review involved conducting targeted Scopus queries to ensure a comprehensive coverage of relevant publications in the field. Efforts have been made to thoroughly evaluate the existing literature, identify research gaps, and propose future research directions. The comparative advantages of this review lie in its comprehensive coverage of the dimensions essential for smart living services and applications, addressing the limitations of previous reviews and offering valuable insights for researchers and practitioners in the field.

/pdf/human-action-recognition-in-smart-living-services-and-1xadeikz.pdf

Human Action Recognition in Smart Living Services and Applications: Context Awareness, Data Availability, Personalization, and Privacy

In recent years, research on Wi-Fi sensing technology has developed rapidly. This technology automatically senses human activities through commercial Wi-Fi devices, such as lying down, falling, walking, waving, sitting down, and standing up. Because the movement of human parts affects the transmission of Wi-Fi signals, resulting in changes in CSI. In the context of indoor monitoring of human health through daily behavior, we propose Wi-CAL. More precisely, CSI fingerprints were collected at six events in two indoor locations, and data enhancement technology Dynamic Time Warping Barycentric Averaging (DBA) was used to expand the data. Then the feature weighting algorithm and convolution layer are combined to select the most representative CSI data features of human action. Finally, a classification model suitable for multiple scenes was obtained by blending the softmax classifier and CORrelation ALignment (CORAL) loss. Experiments are carried out on public data sets and the data sets before and after the expansion collected in this paper. Through comparative experiments, it can be seen that our method can achieve good recognition performance.

/pdf/wi-cal-a-cross-scene-human-motion-recognition-method-based-3cfx4vls.pdf

Wi-CAL: A Cross-Scene Human Motion Recognition Method Based on Domain Adaptation in a Wi-Fi Environment

Human Activity Recognition (HAR) has had a diverse range of applications in various fields such as health, security and smart homes. Among different approaches of HAR, WiFi-based solutions are getting popular since it solves the problem of deployment cost, privacy concerns and restriction of the applicable environment. In this paper, we propose a WiFi-based human activity recognition system that can identify different activities via the channel state information from WiFi devices. A special deep learning framework, Long Short-Term Memory-Convolutional Neural Network (LSTM-CNN), is designed for accurate recognition. LSTM-CNN is going to be compared with the LSTM network and the experimental results demonstrate that LSTM-CNN outperforms existing models and has an average accuracy of 94.14% in multi-activity classification.

Shunuo Shang

Papers

LSTM-CNN network for human activity recognition using WiFi CSI data