To Appear in Neural Computation 1
A Cerebellar Model of Timing and Prediction
in the Control of Reaching
Andrew G. Barto
†
, Andrew H. Fagg
†
,
Nathan Sitkoff
†
, and James C. Houk
∗
†
Department of Computer Science, University of Massachusetts
∗
Department of Physiology, Northwestern University Medical School
May 1998
Abstract
A simplified model of the cerebellum was developed to explore
its potential for adaptive, predictive control based on delayed feed-
back information. An abstract representation of a single Purkinje cell
with multistable properties was interfaced, via a formalized premotor
network, with a simulated single degree-of-freedom limb. The limb
actuator was a nonlinear spring-mass system based on the nonlin-
ear velocity dependence of the stretch reflex. By including realistic
mossy fiber signals, as well as realistic conduction delays in afferent
and efferent pathways, the model allowed the investigation of tim-
ing and predictive processes relevant to cerebellar involvement in the
control of movement. The model regulates movement by learning to
react in an anticipatory fashion to sensory feedback. Learning de-
pends on training information generated from corrective movements
and uses a temporally-asymmetric form of plasticity for the parallel
fiber synapses on Purkinje cells.
1 Introduction
The neural commands that control rapid limb movements appear to be com-
prised of pulse components followed by smaller step components (Ghez 1979;
Ghez and Martin 1982), analogous to the pulse-step commands that control
rapid eye movements (Robinson 1975). In the case of eye movements, the
pulse component serves to overcome the internal viscosity of the muscles,
thus moving the eye rapidly to the target, whereupon the step component
holds the eye at its final position. Limb movements involve more inertia than
eye movements, so the pulse activation of the agonist muscle must end part
way through the movement, and a braking pulse in the antagonist muscle is
needed to decelerate the mass of the limb. Ghez and Martin (1982) showed
that the braking pulse is produced by a stretch reflex in the antagonist mus-
cle. The central control problem, therefore, is to terminate the pulse phase
of the command sent to the agonist muscle at an appropriate time during
the movement. The dynamics of the stretch reflex should then bring the
movement to a halt at a desired endpoint. Since the pulse must terminate
well in advance of the achievement of the desired endpoint, this is a problem
of timing and prediction in control. In this article we present a model of how
the cerebellum may contribute to the predictive control of limb movements.
The model is a simplified version of the adjustable pattern generator (APG)
model being developed by Houk and colleagues (Berthier et al. 1993; Houk
et al. 1990; Sinkjær et al. 1990) to test the computational competence of
a conceptual framework for understanding the brain mechanisms of motor
control (Houk 1989; Houk and Barto 1992; Houk et al. 1993; Houk and
Wise 1995; Houk et al. 1996). The model has a modular architecture in
which single modules generate elemental motor commands with adjustable
time courses, and multiple modules cooperatively produce more complex
commands. The APG model is constrained by the modular anatomy of the
cerebellar cortex and its connections with the limb premotor network, by
the physiology of the neurons comprising this network, and by properties
of cerebellar Purkinje cells (PCs). However, it is purposefully abstract to
allow us to explore control and learning issues in a computationally feasible
manner. The model presented here corresponds to a single module of the
APG model consisting of a single unit representing a PC. This unit is modeled
as a collection of nonlinear switching elements, which we call dendritic zones,
representing segments of a PC dendritic tree.
Our previous modeling studies dealt mainly with two issues: (1) demonstra-
tion that a single module can learn to generate appropriate one-dimensional,
2
variable-duration velocity commands (Houk et al. 1990) and (2) a prelimi-
nary demonstration that an array of 48 modules can learn to function co-
operatively in the control of a simulated non-dynamic, two-joint planar limb
(Berthier et al. 1993). In these previous simulations, the input layer of the
cerebellum, the representation of PCs, and the complexity of the learning
problem were greatly simplified. In the present article, we employ a more re-
alistic input representation based on what is known about movement-related
mossy fiber (MF) signals in the intermediate cerebellum of the monkey (Van
Kan et al. 1993a) and the Marr-Albus architecture of the granular layer
(Tyrrell and Willshaw 1992). In addition, we use a more complex dynamic
spring-mass system (although it is still one-dimensional), and we include re-
alistic conduction delays in the relevant signal pathways. The model also
makes use of a trace mechanism in its learning rule. Preliminary results
appear in Buckingham et al. (1995) and Barto et al. (1996).
We first describe the nonlinear spring-mass system and discuss some of its
properties from a control point of view. The following section presents the
details of the model. We then present simulation results demonstrating the
learning and control abilities of a single dendritic zone followed by similar
results for a model with multiple dendritic zones. We conclude with a dis-
cussion of these results.
2 Pulse-Step Control of a Nonlinear Plant
The limb motor plant has prominent nonlinearities that have a strong influ-
ence on movement and its control. The plant model used in this study is
a spring-mass system with a form of nonlinear damping based on studies of
human wrist movement (Gielen and Houk 1984; Wu et al. 1990):
M ¨x + B( ˙x)
1
5
+ K(x − x
eq
) = 0, (1)
where x is the position (in meters) of an object of mass M (kg) attached
to the spring, x
eq
is the resting, or equilibrium, position, B is the damping
coefficient, and K is the spring stiffness (Fig. 1A). This fractional-power
form of nonlinear damping is derived from a combination of nonlinear muscle
properties and spinal reflex mechanisms, the latter driven mainly by feedback
from muscle spindle receptors (Gielen and Houk 1987). Setting M = 1,
B = 3, and K = 30 produces trajectories that are qualitatively similar to
those observed in human wrist movement (Wu et al. 1990).
Nonlinear damping of this kind enables fast movements that terminate with
little oscillation. Fig. 1B is a graph of the damping force as a function of
3
M
xx
eq
Damping
Force
Velocity
0
A B
0
5
10
15
Xeq (cm)
x
p
x
s
x
0
0
10
20
30
Vel (cm/s)
0 100 200 300 400 500
0
5
10
15
Pos (cm)
Time (ms)
x
p
x
s
x
T
0 2 4 6 8 10
−5
0
5
10
15
20
25
30
35
x
0
x
T
x
s
x
p
Pos (cm)
Vel (cm/s)
C D
Figure 1: Pulse-Step Control of a Simplified Motor Plant. Panel A: Spring-
Mass System. M , mass; x, position; x
eq
, resting, or equilibrium, position.
Panel B: Nonlinear Damping Force as a Function of Velocity. The plant’s
effective damping coefficient (the graph’s slope) increases rapidly as the ve-
locity magnitude decreases to zero. Panel C: Pulse-Step Control. Control
of a movement from initial position x
0
= 0 to target endpoint x
T
= 5 cm.
Top—The pulse-step command. Middle—Velocity as a function of time.
Bottom—Position as a function of time. Panel D: Phase-Plane Trajectory.
The bold line is the phase-plane trajectory of the movement of Panel C. The
dashed line is a plot of the states of the spring-mass system at which the
command should switch from pulse to step so that the mass will stick at the
endpoint x
T
= 5 cm starting from a variety of different initial states.
4
velocity. As velocity decreases, the effective damping coefficient (the curve’s
slope) increases radically when the velocity gets sufficiently close to zero.
This causes a decelerating mass generally to “stick” at a non-equilibrium
position, thereafter drifting extremely slowly toward x
eq
. We call the position
at which the mass sticks (defined here as the position at which the absolute
value of its velocity falls and remains below 0.9 cm/sec) the endpoint of a
movement, denoted x
e
. For all practical purposes, this is where the movement
stops.
The control signal in our model sets the equilibrium value x
eq
, which rep-
resents a central motor command setting the threshold of the stretch reflex
(Feldman 1966; Houk and Rymer 1981). Pulse-step control is effective in
producing rapid and well-controlled positioning of the mass in this system.
As shown in Fig. 1C, the control signal switches from a pulse level x
eq
= x
p
,
to a smaller step level x
eq
= x
s
. Also shown are the time courses of the veloc-
ity and position (middle and bottom) for the resulting movement. Inserting
a low-pass filter in the command pathway, a common feature of muscle mod-
els, would produce velocity profiles more closely matching those of actual
movements, but we have not been concerned with this issue.
Fig. 1D shows the phase-plane trajectory (velocity plotted against position)
followed by the state of the spring-mass system during pulse-step control.
When the pulse is being applied, the state follows a trajectory that would
end at the equilibrium position x
p
= 10 cm if the pulse were to continue.
When the step begins, the state switches to the trajectory that ends at the
equilibrium position x
s
= 4 cm, but the mass sticks at the target endpoint
x
T
= 5 cm before reaching this equilibrium position. Thus, simply setting the
equilibrium position to the target endpoint as suggested by the equilibrium-
point hypothesis (Bizzi et al. 1992; Feldman 1966, 1974) is not a practical
solution to the endpoint positioning task for this system. The dashed line in
Fig. 1D is an approximate plot of the states at which the switch from pulse to
step should occur so that movements starting from a variety of initial states
will stick at x
T
= 5 cm. This switching curve has to vary as a function of
the target endpoint. If the switch from pulse to step occurs too soon (late),
the mass will undershoot (overshoot) x
T
.
In developing a model of pulse-step control of the limb, one can profit from
analogies, where appropriate, with the extensive literature on pulse-step con-
trol of saccadic eye movements. However, an important difference between
eye and limb control is the absence of a stretch reflex for regulating primate
eye muscle activity (Keller and Robinson 1971). As a consequence, models of
the eye motor plant do not contain the nonlinear damping mechanism present
in Eq. 1. As mentioned above, the stretch reflex is important in generating a
5
