*Published in "Human Confort and Security of Information Systems; Advanced Interface for the Information
Society", K. Varghese and S. Pfleger (Eds), Research Report ESPRIT, ISBN 3-540-62067-2, 305 pp, Springer
Verlag, Heidelberg, 1997
A Comparison of Design Strategies
for 3D Human Motions*
Ronan Boulic, Zhiyong Huang, Daniel Thalmann
Computer Graphics Lab (LIG), Swiss Federal Institute of Technology (EPFL)
CH-1015 Lausanne, Switzerland
Three-dimensional character animation and especially human animation becomes everyday more
popular for simulation, multimedia applications, and games. However the specification of human motion
in a computer animation system is still a tedious task for the user in most commercial systems. Based on
the experience on the ESPRIT projects HUMANOID and HUMANOID-2, we compare in this paper the
various strategies for describing and interacting with complex motions of the human body. We especially
analyze the advantages and disadvantages of the associated interfaces. A case study of object grasping
and handling is detailed within the framework of the TRACK system.
1 Introduction
In this paper we compare the design strategies for 3D human motion. We choose to concentrate on the
constraints, the objectives and the methodologies adopted for the body skeleton animation thus excluding
the facial animation field from our investigations. The comparison is organized according to the major
trade-off between the interaction refresh rate and the desired realism of the resulting motion. Although, it
is a major factor of computing cost and final realism of the human motion, we don't address in detail here
the problem of the human envelope deformation, either skin or cloth. It is clear that the same trade-off
also applies to that feature and we keep it in mind when comparing the different strategies. We can order
the various methodologies along a scale beginning at high realism for the motion design process in a
production context, then middle realism for the interactive process provided by a wide range of motion
modeling systems and finally low realism for real-time applications as simulators, virtual reality and
games. We review the characteristics of these various approaches by describing the strategies used to
animate and interact with 3D human characters.
We first examine the design process of realistic motions for productions as films, commercials and more
recently games. Then, in the second part we focus on the wide range of systems providing interactive
response time basically for design purpose or for some of them dedicated to human factors evaluation.
The objectives of the ESPRIT project HUMANOID 2 are recalled in that context. In this class of systems
the integration of the end user interaction flow is depending on the system load. Conversely, in the
simulators and the games a high input-output rate tightly constraints the system architecture as developed
in the third part. We especially stress the problems which have found recent improvements and those
intrinsically difficult to solve in a near future. The fifth part recalls the methodology of the TRACK
system developed in the framework of the ESPRIT project HUMANOID. We focus on a case-study of
complex goal-oriented grasping and handling motions. Finally, we conclude by summarizing the general
trends of the field.
2 Motion Design in Film Production Context
What is required in movie productions is much more than the mere physically-based realism ; it is rather a
high believability conveying the intention of the motion and the emotional state of the character.
Animating a 3D human with that objective in mind makes the whole process extremely difficult as the
models are still desperately simple compared to any real actor.
The animators and directors know very well that the body postures and movements express nearly as
much as the face and the speech themselves [1]. This has been partly exploited in traditional 2D
animation together with other observations regarding subjective interpretation of object proportions and
relative motion. Such practical knowledge can now guide the 3D animators [2][3] in bringing to "life"
cartoon-like or toy-like characters as recently demonstrated with the movie "Toy Story" [4].
- 2 - Autonomous Virtual Humans
Apart from that masterpiece which involved an important
team (110 persons at Pixar [4]), this type of work is limited
to short pieces for cost reasons. At this moment, most of
them are special effects, commercials and, more and more,
some sophisticated games [5].
In the production context, the animators have very detailed
specifications of each individual motions to design from the
storyboard and artistic directives about the characters. The
logical requirement on the software tools is to ask both for
the highest realism and the greatest freedom of design in
order to edit and improve any detail of the motion. However,
as appears on Figure 1 (inspired by [6]), the techniques
providing highly realistic motions, at least from the physical
aspect of the problem, are the ones providing the least design
freedom. We now review them and analyze why the live
recording, also known as Performance Animation, is now the
most popular technique is that field. In a second part we
recall what still prevent performance animation systems to be
more widely accepted in 3D human animation.
2.1 Physics alone does not bring “Life”
The major commercial systems for 3D human animation, as Alias-Wavefront-TDI and Softimage,
propose various degrees of motion realism from the standard Keyframe techniques, Inverse Kinematics,
Direct and Inverse Dynamics to the option of live recording. They ignore Optimization techniques and
Inverse Kinetics (see details in section 3). In the film production context, the motion design is an
incremental process that require the possibility for the animator to fine tune any degree of freedom
(further noted DOF) of the animated structure at any point in time. This is achieved with the large set of
tools manipulating keyframes [4]. Such fine control is also mandatory in cartesian space for fine
positioning and orienting of end effectors (hands, feet, head, others...). This is now very common practice
with Inverse Kinematics (further noted IK) [7][5]. Furthermore, the animator needs interactive
specification and a fast response system to work within an efficient "look, feel and adjust" design loop.
Such requirements discard language-based interfaces in this context [8][9]. Commercial systems now
integrate these techniques and design requirements on standard graphic workstations, allowing to handle
3D human figures with usually around 30 DOFs.
Although impressive results have been obtained with optimization techniques [10], [11] they still face
severe computation costs for such high dimension of animation space. As appears on Figure 1, the second
limitation of that technique comes from the insufficient amount of animator control over the resulting
motion. A recent advance in that field [6] improves these two aspects by combining the optimization with
the keyframe technique:
• the animator has a greater control by specifying keyframed postures, eventually with their associated
key time, as constraints. It is also possible to specify higher level constraints as velocity of the center
of mass or any end effector.
• the type of in-between interpolation is fixed, so it remains only the first derivative at each DOF and
the time of most keyframes to be optimized thus greatly reducing the computing cost.
Perhaps the most difficult problem faced by this approach, in term of designer control, is how to express
the objective function in order to reflect the character’s intentions and mood, i.e. what makes the
character looking alive while performing a desired motion.
This is a general problem also faced by more standard techniques (IK, Dynamics, functional models as
walking, grasping, etc...). At the moment it is solved by sampling the resulting motions into keyframe
motions and use the various techniques available at that bottom level of representation [12] [13]. Figure 1
has put the Live Recording technique at the top of the scale as providing the most realistic motions while,
at first sight, freeing the animator from any intervention. Indeed, recording the motion from a performing
actor allows to capture its natural dynamics along with the subtle attitudes and motions that are so
important to convey the underlying message of the shot [5]. On the other hand, it seems that this
technique transfers the responsibility of the character design from the animator to the actor. In fact, the
actor usually does not match the skeleton features of the virtual creature. Even in case of ideal
measurements, this technique still induces significant work of the animator after converting the motion
Techniques
Kinematics
(keyframe, IK)
Kinetics
Dynamics without
optimization
Optimization
(time, energy,
comfort,...)
Live Recording
Fig. 1: designer freedom
Vs motion realism
Autonomous Virtual Humans - 3 -
into the standard keyframe representation. So, in short, it provides both the realism and the design
freedom. This explain why the Performance Animation approach has been widely adopted in the film
production context [14][5]. We now explore more technically the limitations preventing a larger
acceptance.
2.2 Live recording techniques are still too “superficial”
Most of the Performance Animation systems dedicated to the recording of human body motion belong to
two groups depending on the sensing technologies they rely on, either optical or magnetic. Both allow the
real-time registration of the human motion, practically speaking from around 20Hz to 100Hz for
magnetic, and from 50Hz to 200Hz for optical. Although the optical technology is also suited to record
the hand motion, dedicated devices are proposed which are discussed in section 4. An extensive
discussion about their relative merits can be found in [14]; we just recall here the major facts :
• both approaches place the sensors on the external surface of a human performer
• the magnetic technology provides the position and orientation of sensors while the optical technology
provides the 3D position of reflective markers.
• real-time display is only possible with the magnetic technology ; the motion can be adjusted as a live
action shot (real-time applications are discussed in section 4).
• free movements in large areas are better performed with the wire-free optical approach. The
disadvantage comes from possible line-of-sight problems.
Optical Measurement Magnetic Measurement
Anatomical angles (and translations)
Anatomical Coordinate systems
Sensor Coordinate systems
Markers 3D position
Technical Coordinate system
reconstruction of the segment local frame
constant homogeneous transformations to joints' location and orientation
Interpretation of the relative transformation
between succesive Anatomical Coord. Syst.
3
2
1
0
Fig. 2: The live motion recording process with Performance Animation systems
From the first point we can state that the measurement is "superficial" and this has essential consequences
for its use. Figure 2 recalls the general process of translating the raw 3D trajectories into anatomical joint
angles trajectories (based on the methodology of clinical motion analysis [15][16]). We can distinguish
three fundamental transformations through the different levels of information :
• constructing the so-called technical frames associated with the body segments from the raw 3D
position of at least three markers.
• locating the anatomical frame at each joint of interest by applying a rigid transformation from the
technical frame (or magnetic sensor frame)
• deriving the anatomical angles from the relative orientation between anatomical frames belonging to
successive segments at a given joint.
At this time, the animated character calibration seriously hamper the effective reflection of the
performer's motion, generating uncertainties at the three processing levels :
• The rigid segments assumption is weak due to the deformation of muscles and the dynamics of
human tissues(see [17][18] for comparative measurements).
• The performer's skeleton parameters are difficult to identify, inducing inaccurate positioning of the
technical or sensor frame relative to the anatomical frame. Using bony landmarks is convenient but
subject to errors (see [19] for the hip joint or [15] for the knee).
• The Biomechanics of the human joints should be reflected in the virtual character as well. It is rarely
the case as real joints often exhibit complex behaviors as integrating translation DOFs, varying axis
of rotation, non mutually perpendicular axis, etc... (see [16] for the knee joint).
- 4 - Autonomous Virtual Humans
All these factors alter the realization of the cartesian constraints effectively performed by the real actor,
e.g. the animated character body may exhibit various self-collisions and external collisions (foot into
floor, etc...) or, conversely, no more realize some important contacts (hand in hand, hand on face etc...).
Moreover, in some cases, the imaginary character may have no real performer counterpart thus
amplifying these artifacts.
As a conclusion, the animator is still left a large responsibility in the editing of motion coming from
Performance Animation systems. There is also a need to improve motion editing methods in order to
enforce the cartesian constraints lost in the acquisition process while retaining the initial motion dynamics
[20] [12]. Recent advances in motion signal processing are also worth mentioning in that respect [21] [13]
3 The interactive simulation environment
Apart from the wide range of commercial systems providing interactive response time for the purpose of
animation design, we can consider here the systems dedicated to human factors evaluation, ergonomics,
human behaviors studies and autonomous agent simulations. In this paper we focus only on this second
class of systems. It is more rooted in robotics as the desired result is more quantitative than in the
production class of application. In that context the realism is more a matter of conformance with the
situations potentially faced by populations of future users of public environment [22], working place or
device [23]. Recent advances focus on extending the human model to allow a larger autonomy of the
virtual human agent. In the ESPRIT project HUMANOID II, the perception faculties of vision, audition
and general collision detection are basic features of the human model [24]. Modeling the perception of
balance is also very useful for motion design as developed later [9] [25].
In the interactive simulation context, a large use of functional models is made to access to a higher level
of specification and control of the human motion [9] [26]. Such motion modeling is usually kinematic due
to its low computation cost. As such it may lack the realism requested for full believability. However, it is
the price to pay for the flexibility, the higher levels of control and the longer duration of simulation.
Compared to the production context where one has to pay a high price for a high quality live recording of
says, a single walking motion, we have here models providing flexible and infinite duration of a walking
motion at low cost. Although the resulting motion is less artistic, it remains nearly as realistic as a
recorded one in term of space, time and phase characteristics [27] [28] [29]. The same remark is globally
valid for other classical functional models as grasping [30] or general goal-oriented motion with IK [31]
and general balance control with Inverse Kinetics [25] [7]. Regarding the evaluation of behaviors in
complex environments, the language-based interface now becomes a suitable approach to structure the
functional models activation resulting in a higher level plan similar to robots task planning [9].
The balance control is a fundamental problem in realistic human motion design as human subjects
perform a large class of motion while standing in equilibrium on one or two feet. Inverse Kinematics is
not suited to handle that problem as the mass distribution information is not integrated in the kinematic
jacobian [9]. Conversely, Inverse Kinetics evaluates the exact influence of each joint on the displacement
of the total center of mass [25]. An equally important property of this technique is the ability to combine
it with goal-oriented motions (defined with Inverse Kinematics) in a hierarchical fashion [7]. Such tool
can of course be used to design realistic postures later used as keyframes in a production context.
An important issue in that context is the management of the transition between successive actions. This is
generally made with the ease-in and ease-out technique realized with simple cubic steps. Such approach is
used in games where realistic prerecorded animation sequences, possibly with performance animation
systems, can be combined on the fly to provide fluid behaviors [5]. Some interesting generalization of the
transition management between multiple postures have be proposed to define simple behaviors that can
also be used in real-time applications [32]. Basically, a set of postures is structured in a so-called Posture
Transition Graph defining which posture can success to which posture with associated transition
conditions. The technique has been applied to model a simple soldier behavior with postures as stand,
squat, kneel on one knee, kneel on both knee, supine, crawl and prone.
Another branch of these systems focuses on the study of group and population behaviors for security
assessment of public environments. This branch has begun with simple flock of birds and animal herds
behaviors and now turns to simulate believable human behaviors in complex environments [22]. The
theoretical background behind complex behaviors involving multiple agents are grounded in AI studies
with recent applications to group behaviors of mobile robots [33]. In the human simulation context we
clearly need either language-based or finite state automata structuration to represent complex behaviors
Autonomous Virtual Humans - 5 -
emerging from the interaction of elementary behaviors. Intended applications are scenario testing in
multimedia applications and games with multiple human models.
4 The real-time simulation environment
The real-time simulation environment fully integrate the end-user within the animation loop in order to
drive strategic simulation parameters. In that context, only very small system lag is acceptable in response
to user input. So human motion control shrinks to :
• the playback and combination of prerecorded realistic motions (Cf. section 2) according to a scenario
integrating user generated events (games [5]).
• the use of Inverse Kinematics [34], functional models [35] and posture-based behavioral automata
[32] (Cf. section 3)
In some highly sophisticated real-time environment the system can integrate a real-time performance
animation system to either simulate a virtual character interacting with the end user (interactive TV or
real-time production environment [14]), or to simulate the virtual body of the operator in the virtual
environment [34], or to have bi-directional interaction between operator and virtual character in the
virtual world [36]. The techniques used there are the magnetic sensor technology (Cf. section 2), the real-
time image analysis [36] and various dedicated approaches to measure the hand posture with digital
gloves. The use of digital glove for real-time production of character animation is called digital puppetry
for two reasons :
• the interaction metaphor is close to puppetry as the movement is measured on a articulated structure
(the hand) rather different from the controlled one (the character) thus requiring some adjustment on
the part of the performer [14].
• only simple characters can be animated in such a way due to the limited number of measured DOFs
(even if more than one puppeteer are coordinating their performance, one usually animates the body
while the other one animates the face).
At the moment very few real-time simulation environments integrate the full human body representation
for an operator immersed and interacting with a virtual world [34] [35]. Most VR applications limit the
representation of the operator to the display of the hand posture when wearing a digital glove. Even in
that limited context it can be desirable to automatically alter the displayed hand posture in order to reflect
the virtual hand interaction with the virtual objects [37]. In such a way, the operator gets a feedback about
the relative position of the hand-object system and is able to perform grasping with a higher efficiency.
5 The HUMANOID environment
The HUMANOID environment is dedicated to the development of multimedia, VR and CAD applications
involving virtual humans [26]. This environment integrates highly heterogeneous components such as the
environment model, the humanoid model and various motion generators. Among others, it supports the
TRACK application providing :
• interactive manipulation of multiple humanoids on standard SGI workstations
• skin deformation of a human body, including the hands and the face
• collision detection and correction between multiple humanoid entities
• keyframing, inverse kinematics, dynamics, walking and grasping
In TRACK the motion designer can generate sequences with high level functional models as walking [27]
and grasping [30] [38] and later refine or blend them at the lower keyframe level (figure 3) [12]. Inverse
Kinematics is also one key component of the system especially regarding the ability to perform goal-
oriented motion with grasped objects. Owned to the hierarchical nature of its solution [7], we can
integrate secondary behaviors which significantly improve the realism of the resulting motion. In this
paper we especially focus on two major design issues:
• integrating self-collision avoidance and gravity optimization with IK.
• combining IK and keyframe for goal-oriented motions with grasped objects.
Self-collision is difficult to avoid when the goal-oriented motion is close to the body as can be seen on
figure 4a,b. With standard Inverse Kinematics the end effector usually performs a collision-free trajectory
as it is directly specified by the animator (in figure 4a, the hand first grasps the sphere and then moves to
the target on the left of the body). The self-collision occurs frequently with unused end effectors [39] or
intermediate segments of the articulated figure (figure 4b).