Edinburgh Research Explorer
Dynamical Movement Primitives: Learning Attractor Models for
Motor Behaviors
Citation for published version:
Ijspeert, AJ, Nakanishi, J, Hoffmann, H, Pastor, P & Schaal, S 2013, 'Dynamical Movement Primitives:
Learning Attractor Models for Motor Behaviors', Neural Computation, vol. 25, no. 2, pp. 328-373.
https://doi.org/10.1162/NECO_a_00393
Digital Object Identifier (DOI):
10.1162/NECO_a_00393
Link:
Link to publication record in Edinburgh Research Explorer
Document Version:
Publisher's PDF, also known as Version of record
Published In:
Neural Computation
General rights
Copyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)
and / or other copyright owners and it is a condition of accessing these publications that users recognise and
abide by the legal requirements associated with these rights.
Take down policy
The University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorer
content complies with UK legislation. If you believe that the public display of this file breaches copyright please
contact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately and
investigate your claim.
Download date: 10. Aug. 2022
LETTER Communicated by Hirokazu Tanaka
Dynamical Movement Primitives: Learning Attractor
Models for Motor Behaviors
Auke Jan Ijspeert
auke.ijspeert@epfl.ch
Ecole Polytechnique F
´
ed
´
erale de Lausanne, Lausanne CH-1015, Switzerland
Jun Nakanishi
jun.nakanishi@ed.ac.uk
School of Informatics, University of Edinburgh, Edinburgh EH8 9AB, U.K.
Heiko Hoffmann
heikohof@usc.edu
Peter Pastor
pastorsa@usc.edu
Computer Science, Neuroscience, and Biomedical Engineering, University
of Southern California, Los Angeles, CA 90089, U.S.A.
Stefan Schaal
sschaal@usc.edu
Computer Science, Neuroscience, and Biomedical Engineering, University
of Southern California, Los Angeles, CA 90089, U.S.A.; Max-Planck-Institute
for Intelligent Systems, T
¨
ubingen 72076, Germany; and ATR Computational
Neuroscience Laboratories, Kyoto 619-0288, Japan
Nonlinear dynamical systems have been used in many disciplines to
model complex behaviors, including biological motor control, robotics,
perception, economics, traffic prediction, and neuroscience. While often
the unexpected emergent behavior of nonlinear systems is the focus of
investigations, it is of equal importance to create goal-directed behavior
(e.g., stable locomotion from a system of coupled oscillators under per-
ceptual guidance). Modeling goal-directed behavior with nonlinear sys-
tems is, however, rather difficult due to the parameter sensitivity of these
systems, their complex phase transitions in response to subtle parameter
changes, and the difficulty of analyzing and predicting their long-term
behavior; intuition and time-consuming parameter tuning play a major
role. This letter presents and reviews dynamical movement primitives, a
line of research for modeling attractor behaviors of autonomous nonlin-
ear dynamical systems with the help of statistical learning techniques.
The essence of our approach is to start with a simple dynamical system,
such as a set of linear differential equations, and transform those into a
weakly nonlinear system with prescribed attractor dynamics by means
Neural Computation 25, 328–373 (2013)
c
2013 Massachusetts Institute of Technology
Dynamical Movement Primitives 329
of a learnable autonomous forcing term. Both point attractors and limit
cycle attractors of almost arbitrary complexity can be generated. We ex-
plain the design principle of our approach and evaluate its properties in
several example applications in motor control and robotics.
1 Introduction
In the wake of the development of nonlinear systems theory (Guckenheimer
& Holmes, 1983; Strogatz, 1994; Scott, 2005), it has become common practice
in several branches of science to model natural phenomena with systems of
coupled nonlinear differentialequations. Such approachesare motivated by
the insight that coupling effects of nonlinear systems exhibit rich abilities for
forming complex coordinated patterns without the need to explicitly plan
or supervise the details of such pattern formation. Among the many dif-
ferent forms of nonlinear systems (e.g., high-dimensional, weakly coupled,
strongly coupled, chaotic, Hamiltonian, dissipative), this letter addresses
low-dimensional nonlinear systems, for example, as typically used to model
phenomena of motor coordination or cognitive science (Kelso, 1995; Thelen
& Smith, 1994).
1
In this domain, there are often two modeling objectives.
First, a model of a baseline behavior is required, as in generating a basic
pattern for bipedal locomotion or reach-and-grasp in arm movement. Such
behaviors are goal oriented; the focus is less on emergent coordination phe-
nomena and more on achieving a task objective. After this baseline model
has been accomplished, the second objective is to use this model to account
for more complex phenomena with the help of the coupling dynamics of
nonlinear systems. For instance, a typical example is the modulation of lo-
comotion due to resonance entrainment of the pattern generator with the
dynamics of a physical body (Nakanishi et al., 2004; Hatsopoulos & War-
ren, 1996). Another example is the coupling between motor control and
perception (Dijkstra, Schoner, Giese, & Gielen, 1994; Kelso, 1995; Swinnen
et al., 2004). In order to allow investigations of such second objectives, a
dynamical systems model has to be found first.
Finding an appropriate dynamical systems model for a given behavioral
phenomenon is nontrivial due to the parameter sensitivity of nonlinear
differential equations and their lack of analytical predictability. Thus, mod-
eling is often left to the intuition and the trial-and-error patience of the
researchers. Many impressive studies have been generated in this manner
(Schoner & Kelso, 1988; Sch
¨
oner, 1990; Taga, Yamaguchi, & Shimizu, 1991;
Schaal & Sternad, 1998; Kelso, 1995), but the lack of a generic modeling tool
is unsatisfactory.
In this letter, we propose a generic modeling approach to generate
multidimensional systems of weakly nonlinear differential equations to
1
With low-dimensional, we refer to systems with less than about 100 degrees of
freedom.
330 Ijspeert et al.
capture an observed behavior in an attractor landscape. The essence of
our methodology is to transform well-understood simple attractor systems
with the help of a learnable forcing function term into a desired attractor
system. Both point attractor and limit cycle attractors of almost arbitrary
complexity can be achieved. Multiple degrees of freedom can be coordi-
nated with arbitrary phase relationships. Stability of the model equations
can be guaranteed. Our approach also provides a metric to compare differ-
ent dynamical systems in a scale-invariant and temporally invariant way.
We evaluate our approach in the domain of motor control for robotics,
where desired kinematic motor behaviors will be coded in attractor land-
scapes and then converted into control commands with inverse dynamics
controllers. Importantly, perceptual variables can be coupled back into the
dynamic equations, such that complex closed-loop motor behaviors are
created out of one relatively simple set of equations. Inspired by the bio-
logical concept of motor primitives (Giszter, Mussa-Ivaldi, & Bizzi, 1993;
Mussa-Ivaldi, 1999), we call our system dynamical movement primitives,as
we see them as building blocks that can used and modulated in real time
for generating complex movements.
The followingsections first introduceourmodeling approach (see section
1), then, examine its theoretical properties (see section 2), and finally explore
our approach in the example domain of motor control in various scenarios
(see section 3). Matlab code is provided as supplemental material to allow
readers to explore properties of the system.
2
Early versions of the dynamical
system presented in this letter have been published elsewhere in short
format (Ijspeert, Nakanishi, & Schaal, 2002b, 2003) or some review articles
(Schaal, Mohajerian, & Ijspeert, 2007; Schaal, Ijspeert, & Billard, 2003). Here,
we review previous work and present our system in more detail, introduce
examples of spatial and temporal couplings, and discuss issues related to
generalization and coordinate systems. In the end, this letter presents a
comprehensive and mature account of our dynamic modeling approach
with discussions of related work, which will allow readers to apply or
improve research on this topic.
2 A Learnable Nonlinear Attractor Systems
Before developing our model equations, it will be useful to clarify the
specific goals pursued with this model:
1. Both learnable point attractor and limit cycle attractors need to be
represented. This is useful to encode both discrete (i.e., point to point)
and rhythmic (periodic) trajectories.
3
2
The code can be downloaded from http://www-clmc.usc.edu/Resources/Software.
3
Note that we borrowed the terminology discrete trajectories from the motor control
literature (Schaal, Sternad, Osu, & Kawato, 2004) to denote point-to-point (nonperiodic
Dynamical Movement Primitives 331
2. The model should be an autonomous system, without explicit time
dependence.
3. The model needs to be able to coordinate multidimensional dynam-
ical systems in a stable way.
4. Learning the open parameters of the system should be as simple as
possible, which essentially opts for a representation that is linear in
the open parameters.
5. The system needs to be able to incorporate coupling terms, for exam-
ple, as typically used in synchronization studies or phase resetting
studies and as needed to implement closed-loop perception-action
systems.
6. The system should allow real-time computation as well as arbitrary
modulation of control parameters for online trajectory modulation.
7. Scale and temporal invariance would be desirable; for example,
changing the amplitude or frequency of a periodic system should
not affect a change in geometry of the attractor landscape.
2.1 Model Development. The basic idea of our approach is to use an
analytically well-understood dynamical system with convenient stability
properties and modulate it with nonlinear terms such that it achieves a
desired attractor behavior (Ijspeert et al., 2003). As one of the simplest
possible systems, we chose a damped spring model,
4
τ
¨
y =α
z
(β
z
(g − y) −
˙
y) + f,
which, throughout this letter, we write in first-order notation,
τ
˙
z =α
z
(β
z
(g − y) −z) + f, (2.1)
τ
˙
y =z,
where τ isatimeconstantandα
z
and β
z
are positive constants. If the forcing
term f = 0, these equations represent a globally stable second-order linear
system with (z, y) = (0, g) as a unique point attractor. With appropriate val-
ues of α
z
and β
z
, the system can be made critically damped (with β
z
= α
z
/4)
in order for y to monotonically converge toward g. Such a system imple-
ments a stable but trivial pattern generator with g as single point attractor.
5
The choice of a second-order system in equation 2.1 was motivated
or episodic) trajectories—trajectories that are not repeating themselves, as rhythmic tra-
jectories do. This notation should not be confused with discrete dynamical systems,which
denotes difference equations—those that are time discretized.
4
As will be discussed below, many other choices are possible.
5
In early work (Ijspeert et al., 2002b, 2003), the forcing term f was applied to the second
˙
y equation (instead of the
˙
z equation), which is analytically less favorable. See section 2.1.8.
