Recent years have witnessed the birth of a new paradigm for learning environments: animated pedagogical agents. These lifelike autonomous characters cohabit learning environments with students to create rich, face-to-face learning interactions. This opens up exciting new possibilities; for example, agents can demonstrate complex tasks, employ locomotion and gesture to focus students' attention on the most salient aspect of the task at hand, and convey emotional responses to the tutorial situation. Animated pedagogical agents offer great promise for broadening the bandwidth of tutorial communication and increasing learning environments' ability to engage and motivate students. This article sets forth the motivations behind animated pedagogical agents, describes the key capabilities they offer, and discusses the technical issues they raise. The discussion is illustrated with descriptions of a number of animated agents that represent the current state of the art.

Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments

Real-time control of three-dimensional avatars is an important problem in the context of computer games and virtual environments. Avatar animation and control is difficult, however, because a large repertoire of avatar behaviors must be made available, and the user must be able to select from this set of behaviors, possibly with a low-dimensional input device. One appealing approach to obtaining a rich set of avatar behaviors is to collect an extended, unlabeled sequence of motion data appropriate to the application. In this paper, we show that such a motion database can be preprocessed for flexibility in behavior and efficient search and exploited for real-time avatar control. Flexibility is created by identifying plausible transitions between motion segments, and efficient search through the resulting graph structure is obtained through clustering. Three interface techniques are demonstrated for controlling avatar motion using this data structure: the user selects from a set of available choices, sketches a path through an environment, or acts out a desired motion in front of a video camera. We demonstrate the flexibility of the approach through four different applications and compare the avatar motion to directly recorded human motion.

Interactive control of avatars animated with human motion data

Animated pedagogical agents that inhabit interactive learning environments can exhibit strikingly lifelike behaviors. In addition to providing problem-solving advice in response to students’ activities in the learning environment, these agents may also be able to play a powerful motivational role. To design the most effective agent-based learning environment software, it is essential to understand how students perceive an animated pedagogical agent with regard to affective dimensions such as encouragement, utility, credibility, and clarity. This paper describes a study of the affective impact of animated pedagogical agents on students’ learning experiences. One hundred middle school students interacted with animated pedagogical agents to assess their perception of agents’ affective characteristics. The study revealed the persona eflecr, which is that the presence of a lifelike character in an interactive learning environment~ven one that is not expressive— can have a strong positive effect on student’s perception of their learning experience. The study also demonstrates the interesting effect of multiple types of explanatory behaviors on both affective perception and learning performance.

/pdf/the-persona-effect-affective-impact-of-animated-pedagogical-2myghv8vh2.pdf

The persona effect: affective impact of animated pedagogical agents

There are many different techniques for allowing users to specify locomotion in human-scale, immersive virtual environments. These include flying with a hand-controller, using a treadmill, walking-in-place, and others. 
Real walking, where the user actually and physically walks in the lab, and virtually moves the same distance and in the same direction in the virtual scene, is better than flying. It is more input-natural, does not require learning a new interface, results in a greater sense of presence, and theoretically results in less simulator sickness. 
One serious problem with real walking, however, is that the size of the virtual scene is limited by the size of tracked area. For example, for an architect to really walk in a virtual prototype of a house, the tracked area must be as large as the house. This requirement makes real walking infeasible for many facilities and virtual scenes. 
To address this limitation, I have developed Redirected Walking, which by interactively and imperceptibly rotating the virtual scene around her, makes the user turn herself. Under the right conditions, Redirected Walking would cause the user to unknowingly walk in circles in the lab, while thinking she is walking in a straight and infinitely long path in the virtual scene. 
In this dissertation I develop Redirection, discuss its theoretical and physiological underpinnings, and presents results to show that it can be used: (1) to make the user turn themselves, (2) without causing the user to be aware of Redirection, (3) without unacceptably increasing the user's level of simulator sickness and, most importantly, (4) to useful effect: (A) In head-mounted display systems, the user can experience a virtual scene larger than the lab while also having the benefits of real walking. (B) In an open-backed, three-walled CAVE, users can have the increased presence and input-naturalness normally associated with a fully enclosed CAVE. 
I also present guidelines for VE practitioners wishing to use Redirection, based on the theory and observations reported in this dissertation.

Redirected walking

This article presentsa method for approximatingpolyhedralobjects to support a time-critical collision-detectionalgorithm. The approximationsare hierarchies of spheres, and they allow the time-critical algorithm to progressivelyrefine the accuracy of its detection, stopping as needed to maintain the real-time performanceessential for interactive applications.The key to this approachis a preprocessthat automaticallybuilds tightly fitting hierarchies for rigid and articulatedobjects.The preprocessuses medial-axis surfaces, which are skeletal representations of objects. These skeletonsguide an optimizationtechniquethat gives the hierarchies accuracy properties appropriate for collision detection. In a sample application, hierarchies built this way allow the time-criticalcollision-detectionalgorithmto have acceptableaccuracy, improving significantly on that possible with hierarchies built by previous techniques. The performanceof the time-critical algorithmin this application is consistently 10 to 100 times better than a previous collision-detection algorithm, maintaining low latency and a nearIy constant frame rate of 10 frames per second on a conventional graphics workstation. The time-critical algorithm maintains its real-time performanceas objects become more complicated, even as they exceed previouslyreported complexitylevels by a factor of more than 10.

Approximating polyhedra with spheres for time-critical collision detection

We track, in real-time, the position and posture of a human body, using a minimal number of six DOF sensors to capture full body standing postures. We use four sensors to create a good approximation of a human operator's position and posture, and map it on to our articulated computer graphics human model. The unsensed joints are positioned by a fast inverse kinematics algorithm. Our goal is to realistically recreate human postures while minimally encumbering the operator.

/pdf/real-time-control-of-a-virtual-human-using-minimal-sensors-12fya2lpvx.pdf

Real-time control of a virtual human using minimal sensors

This paper describes a technique for augmenting the process of 3D direct manipulation by automaticallyfinding an effective placement for the virtual camera. Many of the best techniques for direct manipulation of 3D geometric objects are sensitive to the angle of view, and can thus require that the user coordinate the placement of the viewpoint during the manipulation process. In some cases, this process can be automated. This means that the system can automatically avoid degenerate situations in which translations and rotations are difficult to perform. The system can also select viewpoints and viewing angles which make the object being manipulated visible, ensuring that it is not obstructed by other objects.

/pdf/automatic-viewing-control-for-3d-direct-manipulation-14drvlle7t.pdf

Automatic viewing control for 3D direct manipulation

A system for controlling the behaviors of an interactive human-like agent, and executing them in real-time, is presented. It relies on an underlying model of continuous behavior as well as a discrete scheduling mechanism for changing behavior over time. A multiprocessing framework executes the behaviors and renders the motion of the agents in real-time. Finally we discuss the current state of our implementation and some areas of future work.

/pdf/behavioral-control-for-real-time-simulated-human-agents-52zf2x9ch1.pdf

Behavioral control for real-time simulated human agents

Creation of compelling 3-dimensional, multi-user virtual worlds for education and training applications requires a high degree of realism in the appearance, interaction, and behavior of avatars within the scene. Our goal is to develop and/or adapt existing 3-dimensional technologies to provide training scenarios across the Internet in a form as close as possible to the appearance and interaction expected of live situations with human participants. We have produced a prototype system, JackMOO, which combines Jack, a virtual human system, and LambdaMOO, a multiuser; network-accessible, programmable, interactive server: Jack provides the visual realization of avatars and other objects. LambdaMOO provides the Web-accessible communication, programmability, and persistent object database. The combined JackMOO allows us to store the richer semantic information necessitated by the scope and range of human actions that an avatar must portray, and to express those actions in the form of imperative sentences. We describe JackMOO, its components, and a prototype application with five virtual human agents.

/pdf/smart-avatars-in-jackmoo-1qncadg3iw.pdf

Smart avatars in JackMOO

While interpolating between successive postures of an articulated figure is not mathematically difficult, it is much more useful to provide postural transitions that are behaviorally reasonable and that avoid collisions with nearby objects. We describe such a posture interpolator which begins with a number of pre-defined static postures. A finite state machine controls the transitions from any posture to a goal posture by finding the shortest path of required motion sequences between the two. If the motion between any two postures is not collision free, a collision avoidance strategy is invoked and the posture is changed to one that satisfies the required goal while respecting object and agent integrity. >

/pdf/posture-interpolation-with-collision-avoidance-3v54viwlcc.pdf

John P. Granieri

Papers

Real-time control of a virtual human using minimal sensors

Automatic viewing control for 3D direct manipulation

Behavioral control for real-time simulated human agents

Smart avatars in JackMOO

Posture interpolation with collision avoidance