An overview of the self-organizing map algorithm, on which the papers in this issue are based, is presented in this article.

The Self-Organizing Map

A biometric monitoring device is used to determine a user's heart rate by using a heartbeat waveform sensor and a motion detecting sensor. In some embodiments, the device collects collecting concurrent output data from the heartbeat waveform sensor and output data from the motion detecting sensor, detects a periodic component of the output data from the motion detecting sensor, and uses the periodic component of the output data from the motion detecting sensor to remove a corresponding periodic component from the output data from the heartbeat waveform sensor. From this result, the device may determine and present the user's heart rate.

Wearable heart rate monitor

The present inventions, in one aspect, are directed to a portable activity monitoring device comprising a housing having a physical size and shape that is adapted to couple to the user's body, a plurality of sensors (for example, motion sensor and altitude sensor) disposed in the housing. The monitoring device may further include processing circuitry, disposed in the housing and electrically coupled to the plurality of sensor, to calculate the activity points corresponding to the physical activity of the user using the sensor data, wherein the activity points correlate to an amount and intensity of the physical activity of the user, and output the data which is representative of the activity points. The monitoring device may also include a display, coupled to the processing circuitry, may output the data which is representative of the activity points to the user.

Portable monitoring devices and methods of operating same

A number of review or survey articles have previously appeared on human action recognition where either vision sensors or inertial sensors are used individually. Considering that each sensor modality has its own limitations, in a number of previously published papers, it has been shown that the fusion of vision and inertial sensor data improves the accuracy of recognition. This survey article provides an overview of the recent investigations where both vision and inertial sensors are used together and simultaneously to perform human action recognition more effectively. The thrust of this survey is on the utilization of depth cameras and inertial sensors as these two types of sensors are cost-effective, commercially available, and more significantly they both provide 3D human action data. An overview of the components necessary to achieve fusion of data from depth and inertial sensors is provided. In addition, a review of the publicly available datasets that include depth and inertial data which are simultaneously captured via depth and inertial sensors is presented.

/pdf/a-survey-of-depth-and-inertial-sensor-fusion-for-human-mqkujoyxbg.pdf

A survey of depth and inertial sensor fusion for human action recognition

Convolutional Neural Networks and Long Short-Term Memory for skeleton-based human activity and hand gesture recognition

We propose an automatic, privacy-preserving, fall detection method for indoor environments, based on the usage of the Microsoft Kinect® depth sensor, in an “on-ceiling” configuration, and on the analysis of depth frames. All the elements captured in the depth scene are recognized by means of an Ad-Hoc segmentation algorithm, which analyzes the raw depth data directly provided by the sensor. The system extracts the elements, and implements a solution to classify all the blobs in the scene. Anthropometric relationships and features are exploited to recognize one or more human subjects among the blobs. Once a person is detected, he is followed by a tracking algorithm between different frames. The use of a reference depth frame, containing the set-up of the scene, allows one to extract a human subject, even when he/she is interacting with other objects, such as chairs or desks. In addition, the problem of blob fusion is taken into account and efficiently solved through an inter-frame processing algorithm. A fall is detected if the depth blob associated to a person is near to the floor. Experimental tests show the effectiveness of the proposed solution, even in complex scenarios.

https://www.mdpi.com/1424-8220/14/2/2756/pdf

A depth-based fall detection system using a Kinect® sensor.

The aim of Active and Assisted Living is to develop tools to promote the ageing in place of elderly people, and human activity recognition algorithms can help to monitor aged people in home environments. Different types of sensors can be used to address this task and the RGBD sensors, especially the ones used for gaming, are cost-effective and provide much information about the environment. This work aims to propose an activity recognition algorithm exploiting skeleton data extracted by RGBD sensors. The system is based on the extraction of key poses to compose a feature vector, and a multiclass Support Vector Machine to perform classification. Computation and association of key poses are carried out using a clustering algorithm, without the need of a learning algorithm. The proposed approach is evaluated on five publicly available datasets for activity recognition, showing promising results especially when applied for the recognition of AAL related actions. Finally, the current applicability of this solution in AAL scenarios and the future improvements needed are discussed.

/pdf/a-human-activity-recognition-system-using-skeleton-data-from-22115p9u05.pdf

A Human Activity Recognition System Using Skeleton Data from RGBD Sensors

Fall injury issues represent a serious problem for elderly in our society. These people want to live in their home as long as possible and technology can improve their security and independence. In this work we study the joint use of a camera based system and wearable devices, in the so called data fusion approach, to design a fall detection solution. The synchronization issues between the heterogeneous data provided by the devices are properly treated, and three different fall detection algorithms are implemented. Experimental results are also provided, to compare the proposed solutions.

Proposal and Experimental Evaluation of Fall Detection Solution Based on Wearable and Depth Data Fusion

The Microsoft Kinect sensor has gained attention as a tool for gait analysis for several years. Despite the many advantages the sensor provides, however, the lack of a native capability to extract joints from the side view of a human body still limits the adoption of the device to a number of relevant applications. This paper presents an algorithm to locate and estimate the trajectories of up to six joints extracted from the side depth view of a human body captured by the Kinect device. The algorithm is then applied to extract data that can be exploited to provide an objective score for the “Get Up and Go Test”, which is typically adopted for gait analysis in rehabilitation fields. Starting from the depth-data stream provided by the Microsoft Kinect sensor, the proposed algorithm relies on anthropometric models only, to locate and identify the positions of the joints. Differently from machine learning approaches, this solution avoids complex computations, which usually require significant resources. The reliability of the information about the joint position output by the algorithm is evaluated by comparison to a marker-based system. Tests show that the trajectories extracted by the proposed algorithm adhere to the reference curves better than the ones obtained from the skeleton generated by the native applications provided within the Microsoft Kinect (Microsoft Corporation, Redmond, WA, USA, 2013) and OpenNI (OpenNI organization, Tel Aviv, Israel, 2013) Software Development Kits.

Kinect as a tool for gait analysis: validation of a real-time joint extraction algorithm working in side view.

Ambient Assisted Living applications often need to integrate data from multiple sensors, to provide consistent information on the observed phenomena Data fusion based on samples from several sensors requires accurate time synchronization with sufficient resolution, depending on the sensor sampling frequency This work presents a technical platform for the efficient and accurate synchronization of the data captured from RGB-Depth cameras and wearable inertial sensors, that can be integrated in AAL solutions A case study of sensor data fusion for Timed Up and Go test is also presented and discussed

Samuele Gasparrini

Papers

A depth-based fall detection system using a Kinect® sensor.

A Human Activity Recognition System Using Skeleton Data from RGBD Sensors

Proposal and Experimental Evaluation of Fall Detection Solution Based on Wearable and Depth Data Fusion

Kinect as a tool for gait analysis: validation of a real-time joint extraction algorithm working in side view.

Time synchronization and data fusion for RGB-Depth cameras and inertial sensors in AAL applications