Ballester et al. Journal of NeuroEngineering and Rehabilitation
(2016) 13:74
DOI 10.1186/s12984-016-0178-x
RESEARCH Open Access
Counteracting learned non-use in chronic
stroke patients with reinforcement-induced
movement therapy
Belén Rubio Ballester
1†
, Martina Maier
1†
, Rosa María San Segundo Mozo
2
, Victoria Castañeda
2
,ArminDuff
1
andPaulF.M.J.Verschure
1,3*
Abstract
Background: After stroke, patients who suffer from hemiparesis tend to suppress the use of the affected extremity, a
condition called learned non-use. Consequently, the lack of training may lead to the progressive deterioration of
motor function. Although Constraint-Induced Movement Therapies (CIMT) have shown to be effective in treating this
condition, the method presents several limitations, and the high intensity of its protocols severely compromises its
adherence. We propose a novel rehabilitation approach called Reinforcement-Induced Movement Therapy (RIMT),
which proposes to restore motor function through maximizing arm use. This is achieved by exposing the patient to
amplified goal-oriented movements in VR that match the intended actions of the patient. We hypothesize that
through this method we can increase the patients self-efficacy, reverse learned non-use, and induce long-term motor
improvements.
Methods: We conducted a randomized, double-blind, longitudinal clinical study with 18 chronic stroke patients.
Patients performed 30 minutes of daily VR-based training during six weeks. During training, the experimental group
experienced goal-oriented movement amplification in VR. The control group followed the same training protocol but
without movement amplification. Evaluators blinded to group designation performed clinical measurements at the
beginning, at the end of the training and at 12-weeks follow-up. We used the Fugl-Meyer Assessment for the upper
extremities (UE-FM) (Sanford et al., Phys Ther 73:447–454, 1993) as a primary outcome measurement of motor
recovery. Secondary outcome measurements included the Chedoke Arm and Hand Activity Inventory (CAHAI-7)
(Barreca et al., Arch Phys Med Rehabil 6:1616–1622, 2005) for measuring functional motor gains in the performance of
Activities of Daily Living (ADLs), the Barthel Index (BI) for the evaluation of the patient’s perceived independence
(Collin et al., Int Disabil Stud 10:61–63, 1988), and the Hamilton scale (Knesevich et al., Br J Psychiatr J Mental Sci
131:49–52, 1977) for the identification of improvements in mood disorders that could be induced by the
reinforcement-based intervention. In order to study and predict the effects of this intervention we implemented a
computational model of recovery after stroke.
Results: While both groups showed significant motor gains at 6-weeks post-treatment, only the experimental group
continued to exhibit further gains in UE-FM at 12-weeks follow-up (p < .05). This improvement was accompanied by
a significant increase in arm-use during training in the experimental group.
(Continued on next page)
*Correspondence: paul.verschure@upf.edu
†
Equal contributors
1
Laboratory of Synthetic, Perceptive, Emotive and Cognitive Systems,
Universitat Pompeu Fabra, Roc Boronat 138, Barcelona, Spain
3
Institució Catalana de Recerca I Estudis Avançats (ICREA), Barcelona, Spain
Full list of author information is available at the end of the article
© 2016 The Author(s). Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the
Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver
(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Ballester et al. Journal of NeuroEngineering and Rehabilitation
(2016) 13:74
Page 2 of 15
(Continued from previous page)
Conclusions: Implicitly reinforcing arm-use by augmenting visuomotor feedback as proposed by RIMT seems
beneficial for inducing significant improvement in chronic stroke patients. By challenging the patients’ self-limiting
believe system and perceived low self-efficacy this approach might counteract learned non-use.
Trial registration: Clinical Trials NCT02657070.
Keywords: Stroke, Rehabilitation, Deductive medicine, Learned non-use, Virtual reality
Background
After stroke, a neural shock leads to a learning process
in which the brain progressively suppresses the use of
the affected extremity [1]. This phenomenon is commonly
referred to as learned non-use [2, 3]. Constraint-Induced
Movement Therapy (CIMT) [1] implements a technique
that aims to re-integrate the affected arm in the perfor-
mance of Activities of Daily Living (ADLs) and reduce
learned non-use. In order to achieve this goal, CIMT
proposes to restrict the movement of the patient’s less-
affected arm for about 90 % of the patient’s waking hours,
which physically forces the use of the affected arm dur-
ing performance of ADLs. Although a number of studies
have shown the effectivity of CIMT [4], the high inten-
sity of its protocols severely compromises its adherence
[5] and can be physically and mentally tiring [6]. More-
over, its application is restricted to patients without severe
cognitive impairments and with mild hemiparesis, which
only accounts for about 15 % of all stroke cases [7]. Due
to this limitations, several studies have tested variants
of CIMT with reduced intensity protocols, giving rise
to a Modified Constraint-Induced Movement Therapy
(mCIMT) [8] and the so called Distributed Constraint-
Induced Movement Therapy (dCIMT) [9]. However, the
inclusion criteria of this type of therapy still remains
excessively stringent [8, 10], and its efficacy at the chronic
stage is unclear [11]. Given these limitations, there is a
need for developing alternative methods that build on
CIMT principles to foster the usage of the paretic limb,
while mitigate its limitations.
A better understanding of the different factors deter-
mining hand selection could provide valuable insights for
the development of new treatments that effectively coun-
teract learned non-use and promote functional recovery.
Previous studies have shown that the history of rewards
may strongly bias action selection and habit learning
[12–15]. Indeed, perceived self-efficacy, i.e. one’s own
belief in his or her capabilities to successfully execute
actions that are required for a desired outcome [16],
appears to be an important driver for health behavior
improvements [17]. In addition, the minimization of the
expected cost/effort associated to a given action may
as well regulate the decision making process [18]. The
strong influence of these two factors on hand selection
(i.e. expected cost and expected reward) may be sufficient
to approximate the prediction of hand selection patterns,
and may provide a direct explanation of our general pref-
erence for the execution of ipsilateral movements [19].
Following this line of research, we have shown in previ-
ous studies that hemiparetic stroke patients may be highly
sensitive to failure when using the affected limb, therefore
exposure to goal-oriented movement amplification in VR
when using the affected extremity may serve as implicit
reinforcement and promote arm use [20]. The resulting
bias in hand selection patterns may rapidly emerge via
action selection mechanisms, both reducing the expected
cost and increasing the expected outcome associated to
those movements executed with the paretic limb. It is gen-
erally known that motor learning is driven by motor error,
and the high redundancy of the human motor system
allows for the optimization of performance through deci-
sion making processes (i.e. effector selection). Thus, by
virtually reducing sensorimotor error, these decision mak-
ing processes can be modulated through intrinsic evalu-
ation mechanisms [21, 22]. Previous studies have further
proposed that a successful action outcome might reinforce
not only the intended action but also any movement that
drives the ideomotor system during the course of its exe-
cution [23–25]. This theory suggests that accidental suc-
cess after action selection may be an effective mechanism
for the spontaneous emergence of compensatory move-
ments [26]. On this basis, by reducing sensorimotor feed-
back of those goal-oriented movements performed with
the paretic limb, we may reinforce the future selection of
the executed action. Indeed, a fMRI study on one stroke
patient suggests that activations in the sensorimotor cor-
tex of the affected hemisphere (the “inactive” cortex)
were significantly increased simply by providing feed-
back of the contralateral hand [27]. This effect was also
observed in healthy subjects [27]. In more recent studies,
the effect of visuomotor modulations in motor adaptation
has been also explored, showing that diminished error
feedback and goal-oriented movement amplification does
not necessarily compromise error-based learning [22, 28].
Building on these findings and grounding them on the
Distributed Adaptive Control (DAC) theory of mind and
brain, which proposes that restoring impaired sensorimo-
tor contingencies is the key for promoting recovery [29],
Ballester et al. Journal of NeuroEngineering and Rehabilitation
(2016) 13:74
Page 3 of 15
we propose a new motor rehabilitation technique that we
term Reinforcement-Induced Movement Therapy (RIMT)
[20]. This strategy is a combination of the following meth-
ods: 1) Shaping through training, while increasing the task
difficulty according the patient’s performance; 2) limiting
the use of the non-affected arm by introducing contex-
tual restrictions in VR (i.e. restricted and symmetrically
matched workspace for each arm); 3) providing explicit
feedback about performance to the patient; and 4) aug-
menting goal-directed movements of the paretic limb in
virtual reality (VR), in such a way that the patient execut-
ing the movement is exposed to diminished visuomotor
errors, both in terms of distance and directional accu-
racy, thus increasing the expected action outcome (i.e.
expected success) and decreasing the expected action cost
(i.e. expected effort) [21]. While principles one to three
of RIMT are similarly present in CIMT and Occupa-
tional Therapy protocols, the novelty of RIMT resides in
its fourth principle: the provision of implicit reinforce-
ment through the reduction of sensorimotor errors. This
unique component of RIMT is the only variable that will
be manipulated in the present study.
We hypothesize that by reducing visuomotor error
within RIMT protocols, we may be able to boost the
patients’ perceived performance of the paretic limb, lead-
ing to an increased use over time. Consequently, the
increased spontaneous use of the paretic limb may facil-
itate intense practice and induce use-dependent plastic
changes, therefore establishing a closed loop of recovery
in which arm use and motor recovery reinforce each other.
In this vein, a recent computational model of motor recov-
ery suggested that there may be a functional threshold that
predicts the use of the paretic limb after therapy [13, 30].
According to this model, only therapies that enable the
patient to exceed a given functional threshold will recur-
sively increase the spontaneous use of the paretic limb
and induce functional improvement, leading to a com-
plete motor recovery. This principle of use it or loose it can
as well predict the effectiveness of RIMT. Furthermore,
based on simulations from a computational model, we
propose that reinforcement-based and constraint-based
protocols can be combined to maximally promote the use
of the paretic limb and induce functional gains in the
chronic phase after the stroke. To test our hypothesis we
conduct a randomized, double-blind, longitudinal clini-
cal study with chronic stroke patients, and we analyze
the effects of RIMT intervention on counteracting learned
non-use and inducing motor recovery.
Methods
In the following section we first briefly describe a compu-
tational functional model of motor recovery after stroke
that grounds our hypothesis, next we present a behavioral
clinical study with chronic stroke patients.
Theoretical grounding
In order to study the effects and possible applications of
reinforcement-based therapies, we implemented a com-
putational model of recovery after stroke to simulate
different variations of RIMT and CIMT combinations.
The model thus allowed us to study optimal combina-
tions of these two therapies for an effective rehabilitation.
Recently the influence of arm use on motor recovery has
been explored through a bi-stable model of motor recov-
ery after stroke developed by [13]. This functional model
simulated planar unimanual reaching movements towards
a target. In this simulations, movement outcome informed
the system to maximize future performance. We extended
this model by integrating the planing of movement extent
as an indicator of motor performance, and by incorporat-
ing the expected cost of a movement as a parameter for
action selection. Detailed description of the model can be
found in Additional file 1, section Computational model
description and has been published elsewhere [20]. Simu-
lations showed that the averaged probability of choosing
the paretic limb and mean directional errors across trials
increased in both constraint-based and reinforcement-
based treatment conditions (Additional file 1, section
Results from the model). We identified a threshold of per-
formance and a threshold of arm use that initiated a
virtuous loop of recovery by promoting the spontaneous
use of the paretic limb. This bistable dynamics induced
further performance improvement and restored typical
hand selection patterns at follow-up. Contrarily, the no-
therapy condition progressively discouraged the use of the
affected limb and predicted further deterioration.
Experimental protocol and set-up
Subjects
From January 2014 until May 2015, 23 hemiparetic
stroke patients from Hospital Universitari Joan XXIII in
Tarragona, Spain, were recruited according to the fol-
lowing inclusion criteria: a) patients with upper-limb
hemiparesis due to a first-ever ischemic or hemorrhagic
stroke (at least > four weeks post-stroke); b) between 25
and 75 years old; c) demonstrating an upper limb motor
deficit superior to two points as measured by the Medical
Research Council Scale for proximal muscle strength;
d) a spasticity in the affected upper limb of less than
three points as measured through the Modified Ashworth
Scale; e) sufficient cognitive capacity to be able to follow
the instruction of the intervention training as measured
through the Mini Mental State Evaluation (superior than
24 on the scale). Exclusion criteria were defined as: a)
severe cognitive deficits that impede the correct execution
or understanding of the intervention training; b) severe
impairments in vision or visual perception abilities (such
as vision loss or spatial neglect), in spasticity, in communi-
cation abilities (such as aphasia or apraxia), severe pain as
Ballester et al. Journal of NeuroEngineering and Rehabilitation
(2016) 13:74
Page 4 of 15
Table 1 Patient’s characteristics at baseline (n=18)
Characteristics EG n (%) CG p-values
Subjects 12 (52 %) 11 (48 %)
Dropouts 3 (13 %) 2 (9 %)
Compliants 9 (39 %) 9 (39 %)
Gender .578
Female 2 (11 %) 1 (6 %)
Male 7 (39 %) 8 (44 %)
Etiology 1.000
Hemorrhagic 1 (6 %) 3 (17 %)
Ischemic 8 (44 %) 6 (33 %)
Lesion side 1.000
Right 5 (28 %) 4 (22 %)
Left 4 (22 %) 5 (28 %)
Mean (SD) - Median
[25th–75th percentiles]
Age, years 63.40 (9.40) – 63 54.80 (12.00) – 57 .154
[57.80–68.50] [50.80–63.30]
Days 1298.44 (1968.48) – 400 1387.33 (1455.12) – 735 .232
poststroke [269.25–1373.00] [493.50 – 1826.00]
Clinical scales
Total UE-FM 32.33 (16.09) – 38 36.89 (12.29) – 40 .651
[25.50–40.75] [50.80–63.30]
UE-FM-Proximal 17.00 (7.40) – 17 18.89 (6.01) – 19 .88
[12.50–21.50] [16.88–21.13]
UE-FM-Wrist 5.78 (3.60) – 8 4.78 (3.31) – 5 .49
[5.75–10.25] [2.25–7.75]
UE-FM-Hand 7.44 (4.69) – 8 11.44 (4.72) – 12 .15
[4.63–11.38] [8.50–15.50]
UE-FM- 2.56 (1.67) – 3 2.78 (1.30) – 3 .99
Coordination [1.75–4.25] [2.00–4.00]
CAHAI 32.56 (14.47) – 36 36.89 (12.29) – 40 .475
[25.50–42.25] [16.00–45.00]
Table 1 Patient’s characteristics at baseline (n=18) (Continuation)
BI 85.33 (10.82) – 88 90.56 (7.32) – 90 .445
[80.00–91.00] [84.00–96.25]
Hamilton 14.44 (9.61) – 8 12.44 (9.10) – 10 .649
[6.75–24.75] [5.50–19.50]
Statistical test used for p-value: Wilcoxon rank-sum test
well as other neuromuscular or orthopedic changes that
impede the correct execution of the intervention training;
d) mental dysfunctions during the acute or subacute phase
after the stroke. All patients were right-handed.
The study was approved by the local Ethical Committee
at Hospital Universitari Joan XXIII, and the written con-
sent to participate in the experiment was obtained from
all patients involved.
The 23 patients were recruited through the administra-
tive staff of the rehabilitation center of the Hospital Uni-
versitari Joan XXIII and then randomly assigned to two
groups, an Experimental Group (EC) or a Control Group
(CG), by the experimenter who ensured a balanced allo-
cation in the two groups (see Additional file 2). Patients’
demographics and characteristics are shown in Table 1.
Clinicians, that were blinded regarding the group alloca-
tion, conducted the clinical assessments at the beginning
of the experiment (baseline, T0), after six weeks at the end
of the treatment (T1) and at follow-up after 12 weeks (T2).
The experiment concluded in August 2015. Patients were
instructed not to follow any specific therapy during the
participation period.
From the 23 patients recruited, five were excluded due
to the following reasons: a) two patients presented spatial
neglect; b) two patients that were assigned to EG, failed
to complete the intervention training of six weeks; and c)
one patient dropped out after the recruitment. The final
analysis was therefore performed on a total of 18 patients
(n=18), nine in each group.
The Rehabilitation Gaming System (RGS)
In order to provide RIMT as an intervention for motor
recovery, we used the Rehabilitation Gaming System, a
virtual reality-based rehabilitation tool that has shown to
be a valid approach to provide augmented multimodal
feedback and effective sensorimotor training in clinical
setups [21, 31, 32]. RGS incorporates the neurorehabili-
tation paradigm that action execution and observation of
the same action might activate the functional reorganiza-
tion of the motor and pre-motor systems that are affected
by a stroke, potentially by recruiting undamaged primary
or secondary motor areas through alternative sensorimo-
tor pathways [33]. This can be achieved as the patient
Ballester et al. Journal of NeuroEngineering and Rehabilitation
(2016) 13:74
Page 5 of 15
controls with his own movements a virtual body (avatar)
on a computer screen and observes the digital movement
from the first-perspective. By modulating this visuomo-
tor feedback we can provide goal-oriented movement
amplification in VR, consequently exposing the subject to
diminished errors.
Set-up
The clinical set-up of RGS consisted of a desktop touch
screen computer with integrated CPU that displays the
scenarios to the patients and a Microsoft Kinect motion
capture system (Microsoft, US) for tracking the upper-
limb movement of the patient and mapping it to the virtual
arms of an avatar. The computer and the Kinect were
placed in front of an acrylic table that allowed the patients
to rest their arms during the session (Fig. 1a). In addition,
a metallic frame was placed on top of the table, where a
second Kinect and an overhead projector facing the table
were mounted. This additional set-up was needed for one
of the evaluation scenarios that are described after the
following section.
Training scenarios
The three training scenarios used in this study (Fig. 1b-d)
which are called Spheroids, Whack-a-mole and Collector
were game-like intervention protocols that incorporated
various features that aimed to promote the usage of the
paretic limb, either forced or voluntarily. In the Spheroids
and Collector scenarios the patients were required to
intercept colored or patterned spheres by performing hor-
izontal lateral arm movement. A bar in the middle of the
scenery split the virtual workspace in two sides, herewith
forcing the patient to perform ipsilateral movements only;
targets that appeared in the paretic side of the screen
had to be intercepted with the paretic limb, whereas the
less-affected limb could only be used for the targets that
appeared in the workspace ipsilateral to the less-affected
side. As targets could occasionally appear simultaneously
in both work spaces, the patient was prompted to do bi-
manual training. Since the avatar’s arm movement was
controlled by the patient’s joints of the upper extremi-
ties and the avatar’s arm length was fixed, the distance
from the avatar’s hand to the target was equal across
patients. For every successfully intercepted sphere the
patient was rewarded with a point. Within the Collector
scenario the spheres fell from the upper part of the screen
to the bottom, where the patients could intercept them.
In contrast to Spheroids [34], did the Collector scenario
possess an additional cognitive component. In the third
scenario themed Whack-a-mole, patients executed a hor-
izontal reaching movement to eliminate targets (moles)
that appeared sequentially on a planar surface. The loca-
tion of the target did not determine which hand had to
be used, the patients were free in choosing one or the
other limb for any given target, therefore applying ispi-
and contralateral movements. In contrast to the other sce-
nariosthehandshadtobeplacedonstartpositions,that
were indicated by two red cylinders of 7.5 cm in diam-
eter and that were located 48 cm apart from each other,
to initiate the appearance of a target respectively a trial.
The hands had to be maintained on the start positions for
avariabletimeof1±0.5 seconds, after which the start
Fig. 1 Set-up and scenarios. a RGS setup in the hospital showing the transparent acrylic table in front of which the desktop computer with the
Kinect (on a tadpole that elevates it above the screen) is placed. In order to use the second Kinect and the overhead projector on the scaffold above
the table for the real world evaluation scenario, a white cover can be placed over the acrylic surface. During a training session, the user sits in a chair
facing the screen while resting his/her arms on the table. b Spheroids scenario, where sets of colored spheres are launched towards the player who
has to intercept them. c Whack-a-mole scenario, where the user freely chooses which limb to use in order to reach towards an appearing mole.
d Collector scenario, where a set of patterned spheroids as indicated in the upper-left corner of the screen need to be collected. e Virtual evaluation
scenario, an abstract version of the Whack-a-mole scenario, where the patient has to reach towards an appearing cylinder. f Real-world scenario,
where the user has to reach towards randomly appearing dots that are projected from above on the table surface in front of him or her