Accidental falls are a major source of loss of autonomy, deaths, and injuries among the elderly. Accidental falls also have a remarkable impact on the costs of national health systems. Thus, extensive research and development of fall detection and rescue systems are a necessity. Technologies related to fall detection should be reliable and effective to ensure a proper response. This article provides a comprehensive review on state-of-the-art fall detection technologies considering the most powerful deep learning methodologies. We reviewed the most recent and effective deep learning methods for fall detection and categorized them into three categories: Convolutional Neural Network (CNN) based systems, Long Short-Term Memory (LSTM) based systems, and Auto-encoder based systems. Among the reviewed systems, three dimensional (3D) CNN, CNN with 10-fold cross-validation, LSTM with CNN based systems performed the best in terms of accuracy, sensitivity, specificity, etc. The reviewed systems were compared based on their working principles, used deep learning methods, used datasets, performance metrics, etc. This review is aimed at presenting a summary and comparison of existing state-of-the-art deep learning based fall detection systems to facilitate future development in this field.

/pdf/deep-learning-based-systems-developed-for-fall-detection-a-14un7ui8jj.pdf

Deep Learning Based Systems Developed for Fall Detection: A Review

This paper revises the main advances in assistive computer vision recently fostered by deep learning. To this aim, we first discuss how the application of deep learning in computer vision has contributed to the development of assistive techinologies, then analyze the recent advances in assistive technologies achieved in five main areas, namely, object classification and localization, scene understanding, human pose estimation and tracking, action/event recognition and anticipation. The paper is concluded with a discussion and insights for future directions.

/pdf/deep-learning-for-assistive-computer-vision-1ys5mbwrrn.pdf

Deep Learning for Assistive Computer Vision

Standard feature engineering involves manually designing measurable descriptors based on some expert knowledge in the domain of application, followed by the selection of the best performing set of designed features for the subsequent optimisation of an inference model. Several studies have shown that this whole manual process can be efficiently replaced by deep learning approaches which are characterised by the integration of feature engineering, feature selection and inference model optimisation into a single learning process. In the following work, deep learning architectures are designed for the assessment of measurable physiological channels in order to perform an accurate classification of different levels of artificially induced nociceptive pain. In contrast to previous works, which rely on carefully designed sets of hand-crafted features, the current work aims at building competitive pain intensity inference models through autonomous feature learning, based on deep neural networks. The assessment of the designed deep learning architectures is based on the BioVid Heat Pain Database (Part A) and experimental validation demonstrates that the proposed uni-modal architecture for the electrodermal activity (EDA) and the deep fusion approaches significantly outperform previous methods reported in the literature, with respective average performances of 84.57 % and 84.40 % for the binary classification experiment consisting of the discrimination between the baseline and the pain tolerance level ( T 0 vs. T 4 ) in a Leave-One-Subject-Out (LOSO) cross-validation evaluation setting. Moreover, the experimental results clearly show the relevance of the proposed approaches, which also offer more flexibility in the case of transfer learning due to the modular nature of deep neural networks.

https://www.mdpi.com/1424-8220/19/20/4503/pdf

Exploring Deep Physiological Models for Nociceptive Pain Recognition.

Several approaches have been proposed for the analysis of pain-related facial expressions. These approaches range from common classification architectures based on a set of carefully designed handcrafted features, to deep neural networks characterised by an autonomous extraction of relevant facial descriptors and simultaneous optimisation of a classification architecture. In the current work, an end-to-end approach based on attention networks for the analysis and recognition of pain-related facial expressions is proposed. The method combines both spatial and temporal aspects of facial expressions through a weighted aggregation of attention-based neural networks' outputs, based on sequences of Motion History Images (MHIs) and Optical Flow Images (OFIs). Each input stream is fed into a specific attention network consisting of a Convolutional Neural Network (CNN) coupled to a Bidirectional Long Short-Term Memory (BiLSTM) Recurrent Neural Network (RNN). An attention mechanism generates a single weighted representation of each input stream (MHI sequence and OFI sequence), which is subsequently used to perform specific classification tasks. Simultaneously, a weighted aggregation of the classification scores specific to each input stream is performed to generate a final classification output. The assessment conducted on both the BioVid Heat Pain Database (Part A) and SenseEmotion Database points at the relevance of the proposed approach, as its classification performance is on par with state-of-the-art classification approaches proposed in the literature.

Two-Stream Attention Network for Pain Recognition from Video Sequences.

The performance of a conventional information fusion architecture is greatly affected by its ability to detect and combine useful and complementary information from heterogeneous representations stemming from a set of distinctive modalities. Moreover, manually designing a set of relevant and complementary features for a specific pattern recognition task is a complex and tedious endeavour. Therefore, enabling pattern recognition architectures to autonomously generate and select relevant descriptors directly from the set of preprocessed raw data is a favourable alternative to the more conventional manual feature engineering. In the following work, multimodal information fusion approaches based on Deep Denoising Convolutional Autoencoders (DDCAEs) are proposed for the classification of pain intensities based on physiological signals (electrodermal activity (EDA), electromyogram (EMG) and electrocardiogram (ECG)). The approaches are characterized by the simultaneous optimization of both the joint representation of the input channels generated by the multimodal DDCAE and the feed-forward neural network performing the classification of the pain intensities. The assessment performed on the BioVid Heat Pain Database (Part A) points at the relevance of the proposed approaches. In particular, the introduction of trainable weighting parameters for the generation of an aggregated latent representation outperforms most of the previously proposed methods in related works, each based on a set of carefully selected hand-crafted features.

Multimodal Deep Denoising Convolutional Autoencoders for Pain Intensity Classification based on Physiological Signals

The subjective nature of pain makes it a very challenging phenomenon to assess. Most of the current pain assessment approaches rely on an individual’s ability to recognise and report an observed pain episode. However, pain perception and expression are affected by numerous factors ranging from personality traits to physical and psychological health state. Hence, several approaches have been proposed for the automatic recognition of pain intensity, based on measurable physiological and audiovisual parameters. In the current paper, an assessment of several fusion architectures for the development of a multi-modal pain intensity classification system is performed. The contribution of the presented work is two-fold: (1) 3 distinctive modalities consisting of audio, video and physiological channels are assessed and combined for the classification of several levels of pain elicitation. (2) An extensive assessment of several fusion strategies is carried out in order to design a classification architecture that improves the performance of the pain recognition system. The assessment is based on the SenseEmotion Database and experimental validation demonstrates the relevance of the multi-modal classification approach, which achieves classification rates of respectively $83.39\%$ 83 . 39 % , $59.53\%$ 59 . 53 % and $43.89\%$ 43 . 89 % in a 2-class, 3-class and 4-class pain intensity classification task.

/pdf/multi-modal-pain-intensity-recognition-based-on-the-2sjxvirl74.pdf

Multi-Modal Pain Intensity Recognition Based on the SenseEmotion Database

Detecting unintended falls is essential for ambient intelligence and healthcare of elderly people living alone. In recent years, deep convolutional nets are widely used in human action analysis, based on which a number of fall detection methods have been proposed. Despite their highly effective performances, the behaviors of how the convolutional nets recognize falls are still not clear. In this paper, instead of proposing a novel approach, we perform a systematical empirical study, attempting to investigate the underlying fall recognition process. We propose four tasks to investigate, which involve five types of input modalities, seven net instances and different training samples. The obtained quantitative and qualitative results reveal the patterns that the nets tend to learn, and several factors that can heavily influence the performances on fall recognition. We expect that our conclusions are favorable to proposing better deep learning solutions to fall detection systems.

/pdf/an-empirical-study-towards-understanding-how-deep-448fq3unlw.pdf

Yan Zhang

Papers

Multi-Modal Pain Intensity Recognition Based on the SenseEmotion Database

An Empirical Study Towards Understanding How Deep Convolutional Nets Recognize Falls

An Empirical Study towards Understanding How Deep Convolutional Nets Recognize Falls