Automated evaluation of physical therapy exercises using multi-template dynamic time warping on wearable sensor signals

TLDR: The multi-template multi-match dynamic time warping (MTMM-DTW) algorithm is proposed as a natural extension of DTW to detect multiple occurrences of more than one exercise type in the recording of a physical therapy session and for providing feedback to the patient.

About: This article is published in Computer Methods and Programs in Biomedicine.The article was published on 2014-11-01 and is currently open access. It has received 50 citations till now. The article focuses on the topics: Dynamic time warping.

Figures

Fig. 2 – The flowchart of the MTMM-DTW algorithm.

Fig. 4 – Sensor placement on the human body. (a) The first and (b) second configurations are designed for leg and arm exercises, respectively. The arrow perpendicular to the sensor unit indicates the z direction. The cable goes into the sensor term

Fig. 7 – Detection and classification of exercise executions performed by subject 5 in the eight experiments corresponding to the eight exercises. Each detected execution is shown as a bar whose width is the execution’s duration and whose height is the normalized DTW distance between the detected subsequence and the matching template. Black/gray-filled (black/blue online) bars indicate correctly/incorrectly detected executions of the correct exercise type. Unfilled (red online) b xecu F

Fig. 5 – (a–h) The eight physical therapy exercises, labeled 1–8, considered in this study. In each exercise, the subject moves his leg/arm from the initial position (in solid lines) to the position in dotted lines, holds the position for 5 s, and returns to the initial position.

Fig. 1 – Comparison of the Euclidean and DTW distance measures. (a) The Euclidean measure compares the samples at the same time instants, whereas (b) the DTW measure compares samples with similar shapes to minimize the distance. R ons

Fig. 8 – (a) Number of FAs for each exercise of each subject in L1EO in tabular form. (b) Histogram of the normalized DTW distances of all the detections in L1EO. Detections with DTW distances below the threshold are considered FAs.

Frequently asked questions

Q1. What have the authors contributed in "Automated evaluation of physical therapy exercises using multi-template dynamic time warping on wearable sensor signals" ?

The authors propose the multi-template multi-match dynamic time warping ( MTMM-DTW ) algorithm as a natural extension of DTW to detect multiple occurrences of more than one exercise type in the recording of a physical therapy session. While allowing some distortion ( warping ) in time, the algorithm provides a quantitative measure of similarity between an exercise execution and previously recorded templates, based on DTW distance. To evaluate the algorithm ’ s performance, the authors record a data set consisting of one reference template and 10 test executions of three execution types of eight exercises performed by five subjects. Even at the hospital, specialists can not follow each patient continuously during their exercise sessions because the s i n 190 c o m p u t e r m e t h o d s a n d p r o g r a m former often alternate between several patients or have other tasks to do, a situation that can result in insufficient, inaccurate, and often subjective feedback [ 5,6 ]. Below, the authors provide a summary of studies aiming to assess the accuracy of physical therapy exercises or classify them as correct/incorrect: Fergus et al. [ 5 ] propose a tele-rehabilitation system that collects and stores the patient ’ s motion data, utilizing body area and sensor networks, including inertial sensors. This proposed solution is impractical and does not significantly improve inspection time because the system provides no information regarding the patient ’ s movement capability, movement accuracy, or progress. The system provides real-time feedback to the patient with an average classification accuracy of 85 %. In [ 4 ], strain sensors worn on the arm are used to provide real-time feedback to patients undergoing motor rehabilitation. The authors believe that wearable sensors are superior to camera systems in these respects. 

Q2. How many templates can be removed from the MTMM-DTW algorithm?

Since the computational complexity of the MTMM-DTW algorithm is directly proportional to the number of templates used, to further improve the computational efficiency, thes i n 202 c o m p u t e r m e t h o d s a n d p r o g r a mnumber of reference templates can be reduced by removing the two incorrect execution types of the exercises. 

Q3. How many discrete-time sequences were recorded during each experiment?

Because each unit contains three tri-axial devices, 45 (= 9 axes × 5 units) discrete-time sequences were recorded during each experiment. 

Q4. What is the way to match a template to a subsequence?

if ̨ is elected too small (say, less than 0.2), then the template can e matched to subsequences much shorter than itself, which re likely not to contain any exercise execution. 

Q5. What is the way to provide feedback to the patient?

To save time, feedback can also be in the form of alerts that inform physicians/therapists only when needed, for example, when the activity level of the patient is too low, the majority of the executions are incorrect or too fast, etc. 

Q6. What is the only requirement for a physician to do a new exercise?

in their system, if a new exercise needs to be added, the only requirement is to record the templates of the patient performing the different execution types of that exercise, which the physician/therapist can easily do instead of having to rely on the engineer who developed the system. 

Q7. What are the main reasons why the subjects may not perform the exercise properly?

the subjects may not always perform the complete set of exercises properly due to fatigue, lack of concentration or interest, etc. 

Q8. How many times does the subject wait for the same exercise?

At the 100th second, he starts repeating the same exercise 10 times with a type-1 error, and again waits idly, this time until the 160th second. 

Q9. What are the exercises that are more focused on improving functional activities?

The assigned exercises are more directly focused on improving functional activities (e.g., grasping or squeezing an object, holding a cup) and more often involve application of forces to the patient’s body manually or using robotic devices. 

Q10. What are the common errors that patients make during exercise sessions?

Two common types of errors that patients make during exercise sessions are:• Performing the movements too fast; patients do not hold the position for the necessary amount of time because they want to quickly complete the number of repetitions required. 

Q11. What can be done with the sensor units?

The sensor units can be freely configured to properly capture the exercise movements and units can be easily added or removed when needed. 

Q12. Why did the authors use two suitably designed sensor configurations to capture the movements of the right hand?

Because the exercises the authors consider in this study involve only arm or only leg movements, the authors used two suitably designed sensor configurations to capture these motions (see Fig. 4 for details). 

Q13. How many false positives are there in the idle intervals?

The number of samples in the idle intervals can be estimated by dividing the interval’s duration by the duration of the correctly executed template of the exercise in each experiment, obtaining the number of negative (idle) samples.