Showing papers by "Majid Mirmehdi published in 2016"

A comparative study of pose representation and dynamics modelling for online motion quality

Abstract: Quantitative assessment of the quality of motion is increasingly in demand by clinicians in healthcare and rehabilitation monitoring of patients. We study and compare the performances of different pose representations and HMM models of dynamics of movement for online quality assessment of human motion. In a general sense, our assessment framework builds a model of normal human motion from skeleton-based samples of healthy individuals. It encapsulates the dynamics of human body pose using robust manifold representation and a first-order Markovian assumption. We then assess deviations from it via a continuous online measure. We compare different feature representations, reduced dimensionality spaces, and HMM models on motions typically tested in clinical settings, such as gait on stairs and flat surfaces, and transitions between sitting and standing. Our dataset is manually labelled by a qualified physiotherapist. The continuous-state HMM, combined with pose representation based on body-joints’ location, outperforms standard discrete-state HMM approaches and other skeleton-based features in detecting gait abnormalities, as well as assessing deviations from the motion model on a frame-by-frame basis.

Designing a video monitoring system for AAL applications: the SPHERE case study

Abstract: IT based Healthcare platforms have been widely recognized by research communities and institutions as key players in the future of home-based health monitoring and care. Features like personalised care, continuous monitoring, and reduced costs are fostering the research and use of these technologies. In this paper, we describe the design and implementation of the video monitoring system of the SPHERE platform (Sensor Platform for Healthcare in a Residential Environment). SPHERE aims to develop a smart home platform based on low cost, non-medical sensors. We present a detailed description of the hardware and software infrastructure designed and tested in real life scenarios, with particular emphasis on the design considerations employed to foster collaboration, the real time and budget constraints, and mid-scale deployment plan of our case study.

Calorie Counter: RGB-Depth Visual Estimation of Energy Expenditure at Home

Abstract: We present a new framework for vision-based estimation of calorific expenditure from RGB-D data - the first that is validated on physical gas exchange measurements and applied to daily living scenarios. Deriving a person's energy expenditure from sensors is an important tool in tracking physical activity levels for health and lifestyle monitoring. Most existing methods use metabolic lookup tables (METs) for a manual estimate or systems with inertial sensors which ultimately require users to wear devices. In contrast, the proposed pose-invariant and individual-independent vision framework allows for a remote estimation of calorific expenditure. We introduce, and evaluate our approach on, a new dataset called SPHERE-calorie, for which visual estimates can be compared against simultaneously obtained, indirect calorimetry measures based on gas exchange. % based on per breath gas exchange. We conclude from our experiments that the proposed vision pipeline is suitable for home monitoring in a controlled environment, with calorific expenditure estimates above accuracy levels of commonly used manual estimations via METs. With the dataset released, our work establishes a baseline for future research for this little-explored area of computer vision.

3D Data Acquisition and Registration Using Two Opposing Kinects

Abstract: We present an automatic, open source data acquisition and calibration approach using two opposing RGBD sensors (Kinect V2) and demonstrate its efficacy for dynamic object reconstruction in the context of monitoring for remote lung function assessment. First, the relative pose of the two RGBD sensors is estimated through a calibration stage and rigid transformation parameters are computed. These are then used to align and register point clouds obtained from the sensors at frame level. We validated the proposed system by performing experiments on known-size box objects with the results demonstrating accurate measurements. We also report on dynamic object reconstruction by way of human subjects undergoing respiratory functional assessment.

Recognition of unscripted kitchen activities and eating behaviour for health monitoring

Abstract: Nutrition related health conditions such as diabetes and obesity can seriously impact quality of life for those who are affected by them. A system able to monitor kitchen activities and patients’ eating behaviours could provide clinicians with important information helping them to improve patients’ treatments. We propose a symbolic model able to describe unscripted kitchen activities and eating habits of people in home settings. This model consists of an ontology which describes the problem domain, and a Computational State Space Model (CSSM) which is able to reason in a probabilistic manner about a subject’s actions, goals, and causes of any problems during task execution. To validate our model we recorded 15 unscripted kitchen activities involving 9 subjects, with the video data being annotated according to the proposed ontology schemata. We then evaluated the model’s ability to recognise activities and potential goals from action sequences by simulating noisy observations from the annotations. The results showed that our model is able to recognise kitchen activities with an average accuracy of 80% when using specialised models, and with an average accuracy of 40% when using the general model.

Multiple Human Tracking in RGB-D Data: A Survey

Abstract: Multiple human tracking (MHT) is a fundamental task in many computer vision applications. Appearance-based approaches, primarily formulated on RGB data, are constrained and affected by problems arising from occlusions and/or illumination variations. In recent years, the arrival of cheap RGB-Depth (RGB-D) devices has {led} to many new approaches to MHT, and many of these integrate color and depth cues to improve each and every stage of the process. In this survey, we present the common processing pipeline of these methods and review their methodology based (a) on how they implement this pipeline and (b) on what role depth plays within each stage of it. We identify and introduce existing, publicly available, benchmark datasets and software resources that fuse color and depth data for MHT. Finally, we present a brief comparative evaluation of the performance of those works that have applied their methods to these datasets.

Calorie counter: RGB-depth visual estimation of energy expenditure at home

Abstract: We present a new framework for vision-based estimation of calorific expenditure from RGB-D data - the first that is validated on physical gas exchange measurements and applied to daily living scenarios. Deriving a person’s energy expenditure from sensors is an important tool in tracking physical activity levels for health and lifestyle monitoring. Most existing methods use metabolic lookup tables (METs) for a manual estimate or systems with inertial sensors which ultimately require users to wear devices. In contrast, the proposed pose-invariant and individual-independent vision framework allows for a remote estimation of calorific expenditure. We introduce, and evaluate our approach on, a new dataset called SPHERE-calorie, for which visual estimates can be compared against simultaneously obtained, indirect calorimetry measures based on gas exchange. We conclude from our experiments that the proposed vision pipeline is suitable for home monitoring in a controlled environment, with calorific expenditure estimates above accuracy levels of commonly used manual estimations via METs. With the dataset released, our work establishes a baseline for future research for this little-explored area of computer vision.

Beyond Action Recognition: Action Completion in RGB-D Data

Abstract: An action is completed when its goal has been successfully achieved. Using current state-of-the-art depth features, designed primarily for action recognition, an incomplete sequence may still be classified as its complete counterpart due to the overlap in evidence. In this work we show that while features can perform comparably for action recognition, they vary in their ability to recognise incompletion. Experimenting on a novel dataset of 414 complete/incomplete object interaction sequences, spanning six actions and captured using an RGB-D camera, we test for completion using binary classification on labelled data. Results show that by selecting the suitable feature per action, we achieve 95.7% accuracy for recognising action completion.

Cost-based Feature Transfer for Vehicle Occupant Classification

Abstract: Knowledge of human presence and interaction in a vehicle is of growing interest to vehicle manufacturers for design and safety purposes. We present a framework to perform the tasks of occupant detection and occupant classification for automatic child locks and airbag suppression. It operates for all passenger seats using a single overhead camera. A transfer learning technique is introduced to make full use of training data from all seats, whilst still maintaining some control over the bias necessary for a system designed to penalize certain misclassifications more than others. An evaluation is performed on a challenging dataset with both weighted and unweighted classifiers that demonstrates the effectiveness of the transfer process.

Registration and Modeling From Spaced and Misaligned Image Volumes

Abstract: We address the problem of object modeling from 3D and 3D+T data made up of images, which contain different parts of an object of interest, are separated by large spaces, and are misaligned with respect to each other. These images have only a limited number of intersections, hence making their registration particularly challenging. Furthermore, such data may result from various medical imaging modalities and can, therefore, present very diverse spatial configurations. Previous methods perform registration and object modeling (segmentation and interpolation) sequentially. However, sequential registration is ill-suited for the case of images with few intersections. We propose a new methodology, which, regardless of the spatial configuration of the data, performs the three stages of registration, segmentation, and shape interpolation from spaced and misaligned images simultaneously. We integrate these three processes in a level set framework, in order to benefit from their synergistic interactions. We also propose a new registration method that exploits segmentation information rather than pixel intensities, and that accounts for the global shape of the object of interest, for increased robustness and accuracy. The accuracy of registration is compared against traditional mutual information based methods, and the total modeling framework is assessed against traditional sequential processing and validated on artificial, CT, and MRI data.

Real-time Estimation of Physical Activity Intensity for Daily Living

Abstract: Estimating a person's energy expenditure and activity intensity over time is an important component in managing various health conditions or tracking lifestyle choices. To implement an automatic estimation, most current systems ultimately require users to wear sensor devices. In contrast, this paper presents a framework for the contact-free, real-time estimation of energy expenditure, applicable to daily living scenarios. This is a new application in real-time computer vision. We demonstrate the effectiveness and the benefits of utilising a basic set of features and evaluate the resulting framework on the challenging SPHERE-calorie dataset. To ensure accurate evaluation, automated estimates are compared against a simultaneously taken indirect calorimetry ground truth based on per breath gas exchange. Following detailed experiments, we conclude that the proposed real-time vision pipeline is suitable for monitoring physical activity levels in a controlled environment with higher accuracy than the commonly used manual estimation via metabolic lookup tables (METs), whilst being significantly faster than existing automated methods.

Right ventricle segmentation using a 3D cylindrical shape model

Abstract: Accurate segmentation of the right ventricle is a necessary precursor for the assessment of cardiac function. However, the large shape variations exhibited by the right ventricle make automated segmentation a difficult problem. In this work, we explore the ability of a cylindrical shape model to compactly represent and accurately segment this wide range of morphologies. The novelty of this method lies in the design of the fitting function which incorporates learned shape information into a Markov Random Field formulation. Furthermore, the shape model is integrated with a 2D image-based segmentation method, further refining the accuracy of the extracted regions. To evaluate our method, we applied it to the independently evaluated MICCAI RV Segmentation Challenge dataset. Our method performed as well as, or better than, the state-of-the-art methods, validating its suitability for this difficult application.