![Fig. 1. System setup and corresponding DOF; a) Combined system, b) Mechanical support, c) Human arm. (Source: [10])](/figures/fig-1-system-setup-and-corresponding-dof-a-combined-system-b-22hun39v.webp)
Iterative learning control for stroke rehabilitation with input dependent
muscle fatigue modeling
Fons Luijten, Bing Chu and Eric Rogers
Abstract— The consequences of a stroke is a major and
increasing problem world wide. Many people who suffer a
stroke are left with permanent impairment but the possibil-
ity exists that suitable rehabilitation could increase mobility
and, for example, enable independent living. This, in turn,
requires effective rehabilitation where it is known that currently
available methods are relatively poor and are not well suited
to home use, where the latter aspect is critical to improving
practice and reducing costs. An accepted method to relearn lost
function, such as reaching out to an object, is repeated attempts
with learning from previous from those already completed
with the application of applied stimulation if required. This
requirement is analogous to iterative learning control and much
progress, with supporting clinical trials data, has been reported
on using this engineering design method to regulate the applied
stimulation such that patient improvement in completing the
task corresponds to increasing voluntary input and reduced
stimulation. The applied stimulation in this application can
induce muscle fatigue and this paper gives new result on
enhancing the control laws to mitigate this unwanted effect.
I. INTRODUCTION
Annually, 15 million people world-wide suffer a stroke and
up to a third of these are left with permanent impairment.
Other demographic patterns and, in particular, aging popu-
lations place even more strain on the resources for patient
care and rehabilitation. Stroke is an age-related disease [1]
and all of these factors contribute to an increasing burden
on long-term health and related resources. Hence there is a
pressing need to improve the effectiveness of treatments to
achieve independence.
A common cause of a stroke is blockage of a blood
vessel in the brain, where as a result regions downstream are
starved of blood. Consequently, the connecting nerve cells
die and this usually leads to partial paralysis on one side
of the body, termed hemiplegia. The brain cells that die as
a result of a stroke cannot regrow but new connections can
be made using the brain’s spare capacity. In particular, the
brain is continually and rapidly changing and as new skills
are learned, new connections are formed and redundant ones
disappear. Relearning skills after a stroke is the same process
as a person learning an everyday task, such as reaching out to
a cup, and requires sensory feedback during repeated practice
This work was undertaken when the first author was on an Erasmus ex-
change at the Department of Electronics and Computer Science, University
of Southampton, UK.
Fons Luijten is with the Department of Mechanical Engineering, Eind-
hoven University of Technology
Bing Chu and Eric Rogers are with the Department of Electronics and
Computer Science, University of Southampton, Southampton SO17 1BJ,
UK b.chu(etar)@ecs.soton.ac.uk
of a task. This requires movement skills but the affects of
the stroke means that these are almost always very poor and
hence feedback on performance is not obtained.
Stroke survivors commonly have a complex pattern of
upper limb motor impairments with a loss in functional
abilities such as reaching. The coupling between reaching
and independence is reflected in measures of function inde-
pendence, including the Bartel index [2] where the ability to
reach is essential for approximately 50% of activities that
make up daily living tasks. Currently, the level of upper
limb recovery following a stroke is poor and it has been
reported [3] that complete recovery occurs in less than 15%
of patients with initial paralysis. This and the age-related
factor are among the major reasons why there is a critical
need to improve the effectiveness of treatments. If the stage
were reached where rehabilitation could be moved outside
the hospital, which requires mobile technology, then reduced
costs could result.
The literature on conventional therapy plus motor learning
theory, e.g., [4], provides evidence that functional recovery
can be achieved through the facilitation of motor control and
skill acquisition and restoration of muscle power through
repetitive resistance exercises [5], in addition to the vari-
ety of tasks and feedback. This knowledge has motivated
the development of novel treatments, such as robot-aided
therapy, which could provide the basis longer-term for a
translation of rehabilitation clinics from labor-intensive work
to technology-assisted operations and also an opportunity for
repetitive movement practice. Reviews of the robotic therapy
literature, available in [6], [7] and the cited references. For
the upper limb suggest that robot-assisted treatment improves
motor control of the proximal upper limb and may improve
functional outcomes.
Rehabilitation robots are power driven or mechanically
supported devices that assist a patient with limited physical
capability to undertake repetitive exercises. The resulting
sensory feedback is known to be associated with cortical
changes that facilitate the recovery of functional move-
ment. Functional electrical stimulation (FES) has been found
to be applicable as another method in promoting cortical
connectivity to enable recovery, which is motivated by a
growing body of clinical evidence and theoretical support
from neurophysiology and motor learning research, again
see [6], [7] for references to the literature and [8] for an
overview of FES with a control systems perspective.
Application of FES to a muscle causes electrical impulses
to travel along the nerves in the same way as electrical
impulses from the brain and if the stimulation is carefully
regulated a useful movement can be made. In stroke reha-
bilitation FES is applied in combination with the patients
voluntary effort with the aim of a specific recovery of
voluntary power. A wide range of algorithms have been
applied to the control of FES for both the upper and lower
limbs, where again the literature is covered in [6], [7] and the
cited references. In recent work Iterative Learning Control
(ILC), see the references cited in [6], [7] has been applied
to regulate the FES applied in robotic-assisted upper limb
stroke rehabilitation.
This research started with a planar daily living motivated
task, reaching out over a table top, where the patient was
asked to track a supplied reference trajectory whilst attached
to a robotic arm with assistive FES applied to the relevant
muscle, i.e., the triceps. During each attempt, the error
between the desired trajectory and that produced by the
patient was measured, the arm reset to the starting location
and in the time before the next attempt an ILC law was used
to compute the FES to be applied. This work proceeded to a
clinical trial where the required property that as the patient
improves with repeated attempts voluntary effort increased
and the level of FES required decreased [6], [7] was detected.
This application area for ILC has been extended to 3D tasks,
such as reaching and extending the forearm, where there is a
need to stimulate more than one muscle and again supporting
clinical trial results are available [6], [9], [7].
In application, FES applied to muscles is at a higher
frequency and is hence a contributory factor to muscle
fatigue. If the muscle suffers from fatigue then the force
output drops and the treatment session has to stop to allow
recovery, which almost certainly means the session must end
and the patient return at another time, see [8] for a detailed
control systems/modeling discussion of this area. The previ-
ous research on ILC for upper-limb stroke rehabilitation did
not explicitly account for muscle fatigue in the model used
for control law design but this aspect must be addressed if the
use of model based control laws in this and related problem
areas is to proceed.
One approach to overcoming, or at least reducing, the
effects of fatigue was considered in [10], which introduced a
representation for the effects of fatigue into the model for the
response of the muscle to applied FES with a compensating
feedback loop around the model used. The results of a
detailed simulation based evaluation of the new design is
given where the dynamic model is constructed from data
collected from patients participating in a previous clinical
study of this ILC application. Such an evaluation is an
essential step before seeking ethical approval for patient-
based trials. This paper gives new results based on an
enhanced muscle model and gives a performance between
no compensation for muscle fatigue, the previous work and
the new design.
II. BACKGROUND
The same setup as in [10] is considered and starts with
Figure 1, which consists of the human arm supported by
a mechanical rig. This rig can measure the position of the
arm and contains springs to counteract gravity. In Figure 1
a) the combined system is shown, Figure 1 b) shows the
Structure of the mechanical support with all Degrees Of
Freedom (DOF) and Figure 1 c) shows the DOF of the human
arm.
Fig. 1. System setup and corresponding DOF; a) Combined system, b)
Mechanical support, c) Human arm. (Source: [10])
The position of the human arm can be described by Φ =
ϑ
a
ϑ
b
ϑ
c
ϑ
d
ϑ
e
>
. Using a Lagrangian approach,
the differential equations describing the dynamics of the
support and arm can be described as
B(Φ)
¨
Φ + C(Φ,
˙
Φ)
˙
Φ + F (Φ,
˙
Φ) + G(Φ) + K(Φ) = τ (u, Φ,
˙
Φ),
(1)
in which B(Φ) represents the inertial matrix, C(Φ,
˙
Φ) the
Coriolis matrix and F (Φ,
˙
Φ) the vector of non-conservative
forces acting on the system. The term G(Φ) contains mo-
ments from gravity acting on the system and K(Φ) is the
vector of moments from the springs designed to counteract
gravity. An extended description of these terms can be found
in [7]. The input
τ(u, Φ,
˙
Φ) =
0 τ
b
(u
b
, Φ
b
,
˙
Φ
b
) 0 0 τ
e
(u
e
, Φ
e
,
˙
Φ
e
)
>
is created by the stimulated muscle. The task for the pa-
tient considered in [10] is that of lifting the affected arm
and then reaching out from the elbow. It is well known
that stroke patients experience great difficulty in lifting the
affected arm and hence part of this robot compensates for
gravity. The muscles involved are the triceps u
b
(t)) and the
anterior deltoid (u
e
(t)) and complete details of the robot
configuration, how the target is presented and the supporting
software development can again be found in [6], [9], [7] (and
the relevant cited references). Hence only ϑ
b
and ϑ
e
are the
axes in that are actuated by means of electrical stimulation.
The torques in this vector τ (u, Φ,
˙
Φ) are created as described
next.
The relationship between the torques τ
b
and τ
e
and the
applied FES is referred to as the muscle model. Many models
are available to describe the muscle dynamics. A comparison
of some of these models is given in [8]. In this paper, the
model consists of the non-linear activation dynamics which
are modelled using a Hammerstein structure. The model
can be split up in two parts, the linear activation dynamics
h
LAD
and a static non-linearity h
IRC
(u
i
). The non-linearity
h
IRC
(u
i
) describes the Isometric Recruitment Curve (IRC),
which maps the stimulation input u
i
to the steady-state
torque T
m,i
and is given by
h
IRC,i
(u
i
) : T
m,i
= c
1,i
e
c
2,i
u
i
− 1
e
c
2,i
u
i
+ c
3,i
, i = {b, e}. (2)
In this equation c
1,i
, c
2,i
and c
3,i
are parameters specific
for each muscle and can be experimentally determined, see,
e.g., [11], [12] and the relevant cited references in these
papers.. The linear activation dynamics h
LAD
describe the
relation between T
m,i
and the fatigue free torque τ
m,i
. These
dynamics are modelled by a second-order critically damped
linear system given by the state-space model
˙x
i
=
0 1
−ω
2
n
−2ω
n
x
i
+
0
1
h
IRC,i
(u
i
)
τ
m,i
=
ω
2
n
0
x
i
, i = {b, e},
(3)
where ω
n
is the natural frequency and x
i
=
x
i,1
x
i,2
>
is
the state for muscle i. The muscle model is used to simulate
the non-fatigued torque τ
m,i
and the modeling representation
of fatigue used in this paper is discussed next.
A muscle is in general subjected to fatigue. The torque
provided by the muscle will decrease over time when the
applied FES is constant. Previous research [10] used a time
dependent and iteration dependent fatigue model f(t, k)
given by τ
i
= (1 − k
f
t)λ
k−1
τ
m,i
in which k
f
and λ are
constants determining the time and iteration fatigue rates
respectively. The model implies that the muscle will fatigue
during a trial even if no input is applied and the arm does
not move, which is counter-intuitive. A more intuitive model
is used in [13], which proposes a dynamic model describing
fatigue as well as recovery from fatigue as a function of
input. This model is given by
τ
i
= φ
i
τ
m,i
,
˙
φ
i
=
1
T
fat,i
(φ
min,i
− φ
i
)¯τ
m,i
+
1
T
rec,i
(1 − φ
i
)(1 − ¯τ
m,i
),
¯τ
m,i
=
1
τ
m,i
max
τ
m,i
i = {b, e},
(4)
in which φ
min,i
∈ [0, 1] is the minimum fatigue constant
indicating the minimum level of fatigue the muscle can reach.
φ
i
∈ [φ
min,i
, 1] is the fatigue factor and T
fat,i
and T
rec,i
are the time constants for fatigue and recovery respectively,
which need to be estimated based on measurements. ¯τ
m,i
∈
[0, 1] is the normalized positive input. Note that if no fatigue
is present in the system, it holds that φ
i
= 1. Between
two trials, the system is reset to its initial condition during
which the new ILC input is calculated and the muscle has
a chance to recover from fatigue. The resting time between
trials is denoted as t
rec
, during which the muscle model will
be simulated with ¯τ
m,i
= 0 as input.
The control design is split up in two parts. First the feed-
back controllers are designed consisting of a slave controller
and a master controller. Next the ILC design is discussed.
The structure of the control design is given in Figure 2. The
subscript k serves as an indicator for the trial number for
ILC purposes and is left out for the feedback design. It is
assumed that the angles ϑ
b
and ϑ
e
of the human arm can
be measured as well as the fatigued torques τ
b
and τ
e
. All
other DOF’s are not accounted for in the control design.
ILC
Master
Controller
Slave
Controller
h
IRC
h
Fatigue
Model
System
Dynamics
LAD
Φ
Φ
-
-
+
+
+
τ
u
T
e
v
v
v
ref
k
k
fb
k
k
k
k
m
τ
m
k
ILC
Fig. 2. Control structure of the rehabilitation system.
III. CONTROL LAW DESIGN
The slave controller is designed to achieve fast tracking
of the applied input and to deal with fatigue present in the
system. In the previous research [10] the fatigue is considered
as an unknown disturbance on the muscle model and a
linearising controller is applied disregarding any fatigue. The
non-linearity of the muscle h
IRC
is assumed to be monotonic
and known. Implementing the inverse of h
IRC
in the slave
controller will result in the linearising feedback law
u =
u
b
u
e
=
"
h
−1
IRC,b
(ω
2
n
x
b,1
+ 2ω
n
x
b,2
+
v
0
b
ω
2
n
)
h
−1
IRC,e
(ω
2
n
x
e,1
+ 2ω
n
x
e,2
+
v
0
e
ω
2
n
)
#
, (5)
in which v
0
b
and v
0
e
are the new inputs to be designed.
Applying this linearising controller results in the transfer-
function matrix P
muscle
(s) and, using v
0
i
= C
s
i
(v
i
− τ
i
), the
closed-loop transfer-function matrix G
muscle
(s) given by
P
muscle
(s) =
φ
b
s
2
0
0
φ
e
s
2
, (6)
G
muscle
(s) =
"
C
s
b
φ
b
s
2
+C
s
b
φ
b
0
0
C
s
e
φ
e
s
2
+C
s
e
φ
e
#
, (7)
in which C
s
i
is the slave controller chosen as a proportional
gain, C
s
i
= K
s
p
i
. The input to this transfer-function matrix
is v =
v
b
v
e
>
. Also if the patient at any instant reaches
full fatigue (φ
i
= 0) the treatment session cannot continue.
Next the master controller is designed as a proportional gain
plus derivative controller such that the input v is given by
v = v
fb
+ v
ILC
=
K
m
p
b
(ϑ
b,ref
− ϑ
b
) + K
m
d
b
(
˙
ϑ
b,ref
−
˙
ϑ
b
) + v
ILC
b
K
m
p
e
(ϑ
e,ref
− ϑ
e
) + K
m
d
e
(
˙
ϑ
e,ref
−
˙
ϑ
e
) + v
ILC
e
,(8)
where v
ILC
is the feedforward calculated by the ILC con-
troller discussed next.
In this paper the ILC controller is designed Newton-based
ILC, see, e.g. [14]. The Newton method is chosen for the
ILC design but alternatives are possible. First the overall
system is rewritten as a function of the ILC input v
ILC
,
which includes the controlled slave and master loops. The
result is
˙x = f (x, v
ILC
)
y = h(x) =
ϑ
b
ϑ
e
>
(9)
In this paper two cases are considered. Case (1) disregards
any fatigue present in the system in the ILC design and is
designed using
x =
Φ
>
˙
Φ
>
x
>
b
x
>
e
>
,
f(x, v
ILC
) =
˙
Φ
p(Φ,
˙
Φ) + B(Φ)
−1
0
K
s
p
b
x
b,1
0
0
K
s
p
e
x
e,1
x
b,2
−K
s
p
b
x
b,1
+ v
b
x
e,2
−K
s
p
e
x
e,1
+ v
e
,
(10)
with p(Φ,
˙
Φ) = −B(Φ)
−1
(C(Φ,
˙
Φ)
˙
Φ + F (Φ,
˙
Φ) + G(Φ) +
K(Φ)) and where the inputs v
b
and v
e
are given in 8. Case
(2) includes the fatigue model in the ILC design, starting
from
x =
Φ
>
˙
Φ
>
x
>
b
x
>
e
φ
b
φ
e
>
,
f(x, v
ILC
) =
˙
Φ
p(Φ,
˙
Φ) + B(Φ)
−1
0
φ
b
K
s
p
b
x
b,1
0
0
φ
e
K
s
p
e
x
e,1
x
b,2
−K
s
p
b
φ
b
x
b,1
+ v
b
x
e,2
−K
s
p
e
φ
e
x
e,1
+ v
e
1
T
f at,b
(φ
min,b
− φ
b
)
K
s
p
b
x
b,1
τ
m,b
max
+ ˆa
1
T
f at,e
(φ
min,e
− φ
e
)
K
s
p
e
x
e,1
τ
m,e
max
+
ˆ
b
with ˆa =
1
T
rec,b
(1 − φ
b
)(1 −
K
s
p
b
x
b,1
τ
m,b
max
)
and
ˆ
b =
1
T
rec,e
(1 − φ
e
)(1 −
K
s
p
e
x
e,1
τ
m,e
max
).
(11)
In order to calculate the ILC inputs for trial k + 1, the
system in (9) is first sampled with sampling time T
s
to obtain
the discrete time state space model
x
k
(n + 1) = f(x
k
(n), v
ILC
k
(n))
y
k
(n) = h(x
k
(n)),
(12)
with the sample number n ∈ [0, 1, ..., N] in which N =
T
end
/T
s
. All bold symbols are the discrete time equivalents
of their continuous-time counterparts. The system starts each
trial from the same initial condition x
k
(0) = x
0
and can
therefore be written as
y
k
(n) = g(v
ILC
k
(n)).
(13)
Using the Newton method, the ILC input for the next trial
is given by
v
ILC
k+1
= v
ILC
k
+ z
k+1
z
k+1
= g
0
(v
ILC
k
)
−1
e
k
,
(14)
wheree
k
= Φ
ref
− y
k
is the tracking error and g
0
(v
ILC
k
)
is equivalent to the linearization of g(v
ILC
k
) around v
ILC
k
.
This linearization results in the linear time-varying system
˜x(n + 1) = A(n)˜x(n) + B(n)˜u(n)
˜y(n) = C(n)˜x(n)
with
A(n) =
∂f
∂x
k
v
ILC
k
(n),x
k
(n)
,
B(n) =
∂f
∂v
ILC
k
v
ILC
k
(n),x
k
(n)
C(n) =
∂h
∂x
k
v
ILC
k
(n),x
k
(n)
,
(15)
in which ˜u = z
k+1
and ˜y needs to track e
k
. Solving this
system using a second ILC loop will result in the input
z
k+1
that will regulate the trial error dynamics. For this
loop, Norm Optimal ILC (NOILC) [15] is used, where the
subscript j is used to indicate the iteration number. On
iteration j the input and output to the system are ˜u
j
and ˜y
j
respectively. Using NOILC the next input ˜u
j+1
is calculated
by minimizing the cost function
J(˜u
j+1
) =
N−1
X
n=0
((e
k
− ˜y
j
)
>
Q(e
k
− ˜y
j
)
+ (˜u
j+1
− ˜u
j
)
>
R(˜u
j+1
− ˜u
j
)), (16)
in which Q ∈ R
+
and R ∈ R
+
are symmetric weighting
matrices. When j = 10 or when the error e
k
− ˜y
j
is
sufficiently small, the ILC loop is stopped and the resulting
˜u
j
= z
k+1
is used to calculate the new ILC input for the
Newton method using (14).
IV. PERFORMANCE EVALUATION
The performance of the two designs has been compared
in simulation using a system model built from stroke patient
data using estimates for the fatigue model parameters. First
case (1) is evaluated, in which no compensation for fatigue
is implemented. The 2-norm of the error does not converge
in this case which is caused by the fatigue varying too
much from trial-to-trial without compensation. Allowing for
a longer recovery time between successive trials such that
the muscle will recover more from the fatigue is the only
option if fatigue arises is the only way achieving error
convergence in this case. However, with stroke patients the
required ethical approval will specify a maximum time that
a session can last. In this time, all required measurements
necessary to construct the model must be made and also the
task explained and demonstrated to the patient. Moreover,
failure to complete a session for any reason can have a de-
motivating for some patients.
For case (2) the 2-norm of the error converges as seen in
Figure 3.
Fig. 3. 2-norm of the error for Newton based ILC in case (2).
In this case the fatigue is taken into account in the ILC
loop. Because the ILC loop is constructed model based, the
fatigue model is assumed to be known. Since in the first trial
no ILC is applied and fatigue is not compensated for in the
slave controller, the closed loop system has to be stable under
influence of the fatigue. The assumption is made here that the
muscle is not fatigued at the start of the ILC trials. Note that
the next ILC input is calculated from a linearization around
the previous measured/estimated state x. Since the fatigue
on the next trial is different from the previous trial, this
method will only work for small changes in fatigue between
trials. Multiple simulations confirm this result, however more
change is allowed than in case (1). The error is shown in
Figure 4 for trials k = 1, 2 and 5.
This plot exhibits some oscillations in the error signal of
e
e
near the end of the trial after applying the ILC law. These
oscillations were also observed for case (1) and are expected
to be introduced into the system by solving the linear system
in the NOILC problem. More research into the origin of
these oscillations can still be done. Figure 5 shows the input
Fig. 4. The error for Newton based ILC in case (2) for trials k = 1, 2, 5.
to the system at trial k = 6. The input satisfies bounds for
clinical trial approval, but is larger than in case (1). This was
expected since the fatigue is now compensated for in the
input. The results in this part are superior when compared
with those in [10].
Time [s]
Input signal u
b
(mus)
Time [s]
Input signal u
e
(mus)
Fig. 5. The input for Newton based ILC in case (2).
V. CONCLUSIONS
Previous researhc has established that ILC can be used to
regulate the level of FES applied to the muscles of patients
undergoing robotic-assisted upper limb stroke rehabilitation,
where the patient makes repeated attempts at a prescribed
finite duration task with FES applied to the relevant muscles.