Development of a kinect-based physical rehabilitation system
21 Dec 2015-pp 387-392
TL;DR: This system intends to help the patient by keeping track of daily exercise routine, advising for improvements and maintaining records for doctor to access by using Hidden Markov Models for recognition and a histogram-based comparison for computing the accuracy score.
Abstract: Shoulder injuries are very common in sports and certain labour intensive occupations. While some injuries are minor and full recovery is within 1–2 weeks, some major injuries requires the person to consult a physiotherapist and follow an exercise plan for months for full recovery. With the advent of consumer accessible motion capture (MoCap) technologies, the task of conducting daily rehabilitation routine and evaluation which was previously done by a trained physiotherapist can be now done with computers which can be set up in home. In this paper, we propose a system for medical rehabilitation of patients suffering from shoulder injuries. We use Hidden Markov Models (HMM) for recognition and a histogram-based comparison for computing the accuracy score. The Microsoft Kinect sensor is used to obtain 3D coordinates of human joints. Important features are extracted from the skeletal coordinates which are then quantized into 16 intermediate upper-body poses. The temporal patterns of these upper-body poses are modelled by training an HMM for each exercise. Our system recognizes different exercises performed by the patient and assigns an accuracy score for each exercise carried out in a session. It intends to help the patient by keeping track of daily exercise routine, advising for improvements and maintaining records for doctor to access.
Citations
More filters
TL;DR: The presented study reviews computational approaches for evaluating patient performance in rehabilitation programs using motion capture systems and places an emphasis on the application of machine learning methods for movement evaluation in rehabilitation.
Abstract: Recent advances in data analytics and computer-aided diagnostics stimulate the vision of patient-centric precision healthcare, where treatment plans are customized based on the health records and needs of every patient. In physical rehabilitation, the progress in machine learning and the advent of affordable and reliable motion capture sensors have been conducive to the development of approaches for automated assessment of patient performance and progress toward functional recovery. The presented study reviews computational approaches for evaluating patient performance in rehabilitation programs using motion capture systems. Such approaches will play an important role in supplementing traditional rehabilitation assessment performed by trained clinicians, and in assisting patients participating in home-based rehabilitation. The reviewed computational methods for exercise evaluation are grouped into three main categories: discrete movement score, rule-based, and template-based approaches. The review places an emphasis on the application of machine learning methods for movement evaluation in rehabilitation. Related work in the literature on data representation, feature engineering, movement segmentation, and scoring functions is presented. The study also reviews existing sensors for capturing rehabilitation movements and provides an informative listing of pertinent benchmark datasets. The significance of this paper is in being the first to provide a comprehensive review of computational methods for evaluation of patient performance in rehabilitation programs.
28 citations
Cites methods from "Development of a kinect-based physi..."
...Mishra et al. [81] designed a remote home-based rehabilitation program that uses Kinect v1 for motion tracking and streams the recorded videos in real-time to a clinic....
[...]
...[43] (data from [119]) Kinect v1 D None PE Uttarwar and Mishra [75] Kinect v1 S Hand-crafted MC Saraee et al....
[...]
TL;DR: A hybrid solution (software and hardware) integrating the computer and the Kinect sensor is presented, named GoNet v2, an instrument for the dynamic and automatic evaluation of biomechanical rehabilitation processes, showing its optimum precision.
Abstract: This paper presents a hybrid solution (software and hardware) integrating the computer and the Kinect sensor. The proposed solution, named GoNet v2, is an instrument for the dynamic and automatic evaluation of biomechanical rehabilitation processes. Experimental tests to evaluate the range of motion of body joints, especially for elbow flexion, elbow extension, shoulder abduction, shoulder flexion, radial deviation, and ulnar deviation, are presented and discussed. We also presented the exergamers for rehabilitation tests, based on Kabat diagonal and squatting. Ten healthy individuals were evaluated using the GoNet v2 and the universal goniometer, and twelve professionals evaluated the instrument through a survey. The intraclass correlation coefficient (ICC) was used to analyze the reproducibility, and for the accuracy analysis the errors were compared using the mean of the worst cases with movement. Regarding intra-examiner and inter-examiner reproducibility, high ICC values were found for the range of flexion/extension of the shoulder, abduction of the shoulder, and ulnar deviation, thus showing its optimum precision. According to the evaluation of the specialists, the GoNet v2 gave better results for the flexion/extension of the shoulder (3.61%) and elbow (3.17%), and also the abduction (2.11%) of the shoulder compared with the goniometer. The results showed that the GoNet v2 had a high reproducibility, except for radial deviation. The accuracy results were good for the abduction measurements of the shoulder and the flexion/extension measurements of the elbow and shoulder.
20 citations
04 Feb 2021
TL;DR: There are various HETs, ranging from simple videoconferencing systems to complex sensor-based technologies for telerehabilitation, that assist patients with musculoskeletal shoulder disorders when exercising at home that are not ready for practical use.
Abstract: Background: Health-enabling technologies (HETs) are information and communication technologies that promote individual health and well-being. An important application of HETs is telerehabilitation for patients with musculoskeletal shoulder disorders. Currently, there is no overview of HETs that assist patients with musculoskeletal shoulder disorders when exercising at home.
Objective: This scoping review provides a broad overview of HETs that assist patients with musculoskeletal shoulder disorders when exercising at home. It focuses on concepts and components of HETs, exercise program strategies, development phases, and reported outcomes.
Methods: The search strategy used Medical Subject Headings and text words related to the terms upper extremity, exercises, and information and communication technologies. The MEDLINE, Embase, IEEE Xplore, CINAHL, PEDro, and Scopus databases were searched. Two reviewers independently screened titles and abstracts and then full texts against predefined inclusion and exclusion criteria. A systematic narrative synthesis was performed. Overall, 8988 records published between 1997 and 2019 were screened. Finally, 70 articles introducing 56 HETs were included.
Results: Identified HETs range from simple videoconferencing systems to mobile apps with video instructions to complex sensor-based technologies. Various software, sensor hardware, and hardware for output are in use. The most common hardware for output are PC displays (in 34 HETs). Microsoft Kinect cameras in connection with related software are frequently used as sensor hardware (in 27 HETs). The identified HETs provide direct or indirect instruction, monitoring, correction, assessment, information, or a reminder to exercise. Common parameters for exercise instructions are a patient’s range of motion (in 43 HETs), starting and final position (in 32 HETs), and exercise intensity (in 20 HETs). In total, 48 HETs provide visual instructions for the exercises; 29 HETs report on telerehabilitation aspects; 34 HETs only report on prototypes; and 15 HETs are evaluated for technical feasibility, acceptance, or usability, using different assessment instruments. Efficacy or effectiveness is demonstrated for only 8 HETs. In total, 18 articles report on patients’ evaluations. An interdisciplinary contribution to the development of technologies is found in 17 HETs.
Conclusions: There are various HETs, ranging from simple videoconferencing systems to complex sensor-based technologies for telerehabilitation, that assist patients with musculoskeletal shoulder disorders when exercising at home. Most HETs are not ready for practical use. Comparability is complicated by varying prototype status, different measurement instruments, missing telerehabilitation aspects, and few efficacy studies. Consequently, choosing an HET for daily use is difficult for health care professionals and decision makers. Prototype testing, usability, and acceptance tests with the later target group under real-life conditions as well as efficacy or effectiveness studies with patient-relevant core outcomes for every promising HET are required. Furthermore, health care professionals and patients should be more involved in the product design cycle to consider relevant practical aspects.
11 citations
Cites background from "Development of a kinect-based physi..."
...In total, 27 articles do not directly name the proposed target group [29,34,38,40,43,44,46,51,52,55-57,59,61,65-67,69,70,72,74,78,80,82,84-87]....
[...]
...All identified articles and related HETs, grouped by telerehabilitation aspects, are shown in a table in Multimedia Appendix 3 [18-87]....
[...]
...Of these, 18 articles refer to a specific target group for their use [20,23,26,36,37,45,47-50,52,54,58,62,68,71,73,77]....
[...]
TL;DR: This research paper compares two HMM classifiers and Hidden Conditional Random Fields plus two types of posture descriptors, based on points and based on angles, and point representation proves to be superior to angle representation, although the latter is still acceptable.
Abstract: Rehabilitation systems are becoming more impor-tant now because patients can access motor skills recovery treatment from home, reducing the limitations of time, space and cost of treatment in a medical facility. Traditional rehabilitation systems served as movement guides, later as movement mirrors, and in recent years research has sought to generate feedback messages to the patient based on the evaluation of his or her movements. Currently the most commonly used algorithms for exercise evaluation are Dynamic time warping (DTW), Hidden Markov model (HMM), Support vector machine (SVM). However, the larger the set of exercises to be evaluated, the less accurate the recognition becomes, generating confusion between exercises that have similar posture descriptors. This research paper compares two HMM classifiers and Hidden Conditional Random Fields (HCRF) plus two types of posture descriptors, based on points and based on angles. Point representation proves to be superior to angle representation, although the latter is still acceptable. Similar results are found in HCRF and HMM.
6 citations
Cites background or methods from "Development of a kinect-based physi..."
...In this paper we applied HMM described in [4] by Utarwar et al....
[...]
...[4], propose a rehabilitation system for shoulder injuries....
[...]
References
More filters
TL;DR: This work takes an object recognition approach, designing an intermediate body parts representation that maps the difficult pose estimation problem into a simpler per-pixel classification problem, and generates confidence-scored 3D proposals of several body joints by reprojecting the classification result and finding local modes.
Abstract: We propose a new method to quickly and accurately predict human pose---the 3D positions of body joints---from a single depth image, without depending on information from preceding frames. Our approach is strongly rooted in current object recognition strategies. By designing an intermediate representation in terms of body parts, the difficult pose estimation problem is transformed into a simpler per-pixel classification problem, for which efficient machine learning techniques exist. By using computer graphics to synthesize a very large dataset of training image pairs, one can train a classifier that estimates body part labels from test images invariant to pose, body shape, clothing, and other irrelevances. Finally, we generate confidence-scored 3D proposals of several body joints by reprojecting the classification result and finding local modes.The system runs in under 5ms on the Xbox 360. Our evaluation shows high accuracy on both synthetic and real test sets, and investigates the effect of several training parameters. We achieve state-of-the-art accuracy in our comparison with related work and demonstrate improved generalization over exact whole-skeleton nearest neighbor matching.
3,034 citations
16 Jun 2012
TL;DR: This paper presents a novel approach for human action recognition with histograms of 3D joint locations (HOJ3D) as a compact representation of postures and achieves superior results on the challenging 3D action dataset.
Abstract: In this paper, we present a novel approach for human action recognition with histograms of 3D joint locations (HOJ3D) as a compact representation of postures. We extract the 3D skeletal joint locations from Kinect depth maps using Shotton et al.'s method [6]. The HOJ3D computed from the action depth sequences are reprojected using LDA and then clustered into k posture visual words, which represent the prototypical poses of actions. The temporal evolutions of those visual words are modeled by discrete hidden Markov models (HMMs). In addition, due to the design of our spherical coordinate system and the robust 3D skeleton estimation from Kinect, our method demonstrates significant view invariance on our 3D action dataset. Our dataset is composed of 200 3D sequences of 10 indoor activities performed by 10 individuals in varied views. Our method is real-time and achieves superior results on the challenging 3D action dataset. We also tested our algorithm on the MSR Action 3D dataset and our algorithm outperforms Li et al. [25] on most of the cases.
1,453 citations
Additional excerpts
...In [6], authors have used histograms of 3D joint locations human action recognition....
[...]
12 Nov 2012
TL;DR: This paper compares the Kinect pose estimation (skeletonization) with more established techniques for pose estimation from motion capture data, examining the accuracy of joint localization and robustness of pose estimation with respect to the orientation and occlusions.
Abstract: The Microsoft Kinect camera is becoming increasingly popular in many areas aside from entertainment, including human activity monitoring and rehabilitation. Many people, however, fail to consider the reliability and accuracy of the Kinect human pose estimation when they depend on it as a measuring system. In this paper we compare the Kinect pose estimation (skeletonization) with more established techniques for pose estimation from motion capture data, examining the accuracy of joint localization and robustness of pose estimation with respect to the orientation and occlusions. We have evaluated six physical exercises aimed at coaching of elderly population. Experimental results present pose estimation accuracy rates and corresponding error bounds for the Kinect system.
356 citations
"Development of a kinect-based physi..." refers background in this paper
...Some authors tested the accuracy of Kinect in context of coaching elderly population [2]....
[...]
Posted Content•
01 Jul 2011TL;DR: This paper uses a RGBD sensor as the input sensor, and presents learning algorithms to infer the activities of a person based on a hierarchical maximum entropy Markov model (MEMM), and infers the two-layered graph structure using a dynamic programming approach.
Abstract: Being able to detect and recognize human activities is essential for several applications, including personal assistive robotics. In this paper, we perform detection and recognition of unstructured human activity in unstructured environments. We use a RGBD sensor (Microsoft Kinect) as the input sensor, and compute a set of features based on human pose and motion, as well as based on image and pointcloud information. Our algorithm is based on a hierarchical maximum entropy Markov model (MEMM), which considers a person's activity as composed of a set of sub-activities. We infer the two-layered graph structure using a dynamic programming approach. We test our algorithm on detecting and recognizing twelve different activities performed by four people in different environments, such as a kitchen, a living room, an office, etc., and achieve good performance even when the person was not seen before in the training set.
311 citations
11 Dec 2014
TL;DR: Technical and clinical impact of the Microsoft Kinect in physical therapy and rehabilitation covers the studies on patients with neurological disorders including stroke, Parkinson's, cerebral palsy, and MS as well as the elderly patients.
Abstract: This paper reviews technical and clinical impact of the Microsoft Kinect in physical therapy and rehabilitation. It covers the studies on patients with neurological disorders including stroke, Parkinson’s, cerebral palsy, and MS as well as the elderly patients. Search results in Pubmed and Google scholar reveal increasing interest in using Kinect in medical application. Relevant papers are reviewed and divided into three groups: (1) papers which evaluated Kinect’s accuracy and reliability, (2) papers which used Kinect for a rehabilitation system and provided clinical evaluation involving patients, and (3) papers which proposed a Kinect-based system for rehabilitation but fell short of providing clinical validation. At last, to serve as technical comparison to help future rehabilitation design other sensors similar to Kinect are reviewed.
311 citations