Falling is one of the causes of accidental death of elderly people over 65 years old in Taiwan. If the fall incidents are not detected in a timely manner, it could lead to serious injury or even death of those who fell. General fall detection approaches require the users to wear sensors, which could be cumbersome for the users to put on, and misalignment of sensors could lead to erroneous readings. In this paper, we propose using computer vision and applied machine-learning algorithms to detect fall without any sensors. We applied OpenPose real-time multi-person 2D pose estimation to detect movement of a subject using two datasets of 570 × 30 frames recorded in five different rooms from eight different viewing angles. The system retrieves the locations of 25 joint points of the human body and detects human movement through detecting the joint point location changes. The system is able to effectively identify the joints of the human body as well as filtering ambient environmental noise for an improved accuracy. The use of joint points instead of images improves the training time effectively as well as eliminating the effects of traditional image-based approaches such as blurriness, light, and shadows. This paper uses single-view images to reduce equipment costs. We experimented with time series recurrent neural network, long- and short-term memory, and gated recurrent unit models to learn the changes in human joint points in continuous time. The experimental results show that the fall detection accuracy of the proposed model is 98.2%, which outperforms the baseline 88.9% with 9.3% improvement.

https://www.mdpi.com/2076-3417/11/1/329/pdf

A Framework for Fall Detection Based on OpenPose Skeleton and LSTM/GRU Models

In recent years, the scientific community has found increasing interest in technological systems for the evaluation of the mobility performance of the elderly population. The reduced quantity of dataset for gait and balance analysis of elderly people is a serious issue in studying the link between cognitive impairment and motor dysfunction, particularly in people suffering from neurodegenerative diseases. In this context, this work aims to provide a dataset with skeletal information of people aged 60 years and older, while they perform well-established tests for stability assessment. 27 healthy people and 20 patients affected by neurodegenerative diseases, housed at two different nursing institutes, have been selected for the stability analysis. Subjects have been observed and evaluated by clinical therapists while executing three motion tests, namely balance, sit-to-stand and walking. The stability postural and gait control of each subject has been analyzed using a video-based system, made of three low-cost cameras, without the need for wearable and invasive sensors. The dataset provided in this work contains the skeletal information and highly-discriminant features of the balance, sit-to-stand and walking tests performed by each subject. To evaluate the efficiency of the balance dataset, the estimated risk of fall of the subjects has been processed considering the extracted features, and compared with the expected one. Final results have proven a good estimation of the risk of fall of the people under analysis, underlining the effectiveness of the dataset.

Image-Based Mobility Assessment in Elderly People from Low-Cost Systems of Cameras: A Skeletal Dataset for Experimental Evaluations

For patients suffering from neurodegenerative disorders, the behavior and activities of daily living are an indicator of a change in health status, and home-monitoring over a prolonged period of time by unobtrusive sensors is a promising technology to foster independent living and maintain quality of life. The aim of this pilot case study was the development of a multi-sensor system in an apartment to unobtrusively monitor patients at home during the day and night. The developed system is based on unobtrusive sensors using basic technologies and gold-standard medical devices measuring physiological (e.g., mobile electrocardiogram), movement (e.g., motion tracking system), and environmental parameters (e.g., temperature). The system was evaluated during one session by a healthy 32-year-old male, and results showed that the sensor system measured accurately during the participant’s stay. Furthermore, the participant did not report any negative experiences. Overall, the multi-sensor system has great potential to bridge the gap between laboratories and older adults’ homes and thus for a deep and novel understanding of human behavioral and neurological disorders. Finally, this new understanding could be utilized to develop new algorithms and sensor systems to address problems and increase the quality of life of our aging society and patients with neurological disorders.

https://www.mdpi.com/1424-8220/22/4/1657/pdf?version=1645352105

An Instrumented Apartment to Monitor Human Behavior: A Pilot Case Study in the NeuroTec Loft

In recent years, the number of older people living alone has increased rapidly. Innovative vision systems to remotely assess people’s mobility can help healthy, active, and happy aging. In the related literature, the mobility assessment of older people is not yet widespread in clinical practice. In addition, the poor availability of data typically forces the analyses to binary classification, e.g. normal/anomalous behavior, instead of processing exhaustive medical protocols. In this paper, real videos of elderly people performing three mobility tests of a clinical protocol are automatically categorized, emulating the complex evaluation process of expert physiotherapists. Videos acquired using low-cost cameras are initially processed to obtain skeletal information. A proper data augmentation technique is then used to enlarge the dataset variability. Thus, significant features are extracted to generate a set of inputs in the form of time series. Four deep neural network architectures with feedback connections, even aided by a preliminary convolutional layer, are proposed to label the input features in discrete classes or to estimate a continuous mobility score as the result of a regression task. The best results are achieved by the proposed Conv-BiLSTM classifier, which achieves the best accuracy, ranging between 88.12% and 90%. Further comparisons with shallow learning classifiers still prove the superiority of the deep Conv-BiLSTM classifier in assessing people’s mobility, since deep networks can evaluate the quality of test executions. 

/pdf/video-based-mobility-monitoring-of-elderly-people-using-deep-hwp6y3rg.pdf

Video Based Mobility Monitoring of Elderly People Using Deep Learning Models

In recent years, the number of older people living alone has increased rapidly. Innovative vision systems to remotely assess people’s mobility can help healthy, active, and happy aging. In the related literature, the mobility assessment of older people is not yet widespread in clinical practice. In addition, the poor availability of data typically forces the analyses to binary classification, e.g. normal/anomalous behavior, instead of processing exhaustive medical protocols. In this paper, real videos of elderly people performing three mobility tests of a clinical protocol are automatically categorized, emulating the complex evaluation process of expert physiotherapists. Videos acquired using low-cost cameras are initially processed to obtain skeletal information. A proper data augmentation technique is then used to enlarge the dataset variability. Thus, significant features are extracted to generate a set of inputs in the form of time series. Four deep neural network architectures with feedback connections, even aided by a preliminary convolutional layer, are proposed to label the input features in discrete classes or to estimate a continuous mobility score as the result of a regression task. The best results are achieved by the proposed Conv-BiLSTM classifier, which achieves the best accuracy, ranging between 88.12% and 90%. Further comparisons with shallow learning classifiers still prove the superiority of the deep Conv-BiLSTM classifier in assessing people’s mobility, since deep networks can evaluate the quality of test executions.

Falls represent one of the most serious clinical problems in the elderly population. This risk is even more important in people suffering from neurodegenerative problems. This work aims to instrumentally assess the balance performance of elderly people and specifically those suffering from neurodegenerative diseases, to obtain an objective evaluation of their risk of falls. This paper presents a vision-based system made of three low-cost cameras, able to automatically infer important mobility parameters by observing the execution of well-established tests for stability assessment. This result is achieved by a dedicated image processing pipeline, which processes videos to get dynamic user skeletons, and the following strategy for information management, which targets to feature extraction. This information finally feeds a classifier, namely a decision tree, trained to predict the risk of fall of patients within 5 classes of interest. Actual experiments performed on actual video recordings prove a good agreement of results with those expected, labeled by expert therapists, with final prediction accuracy of 79.1%.

Nicola Lorusso

Papers

Vision-based Assessment of Balance Control in Elderly People