An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

Recent years have witnessed the birth of a new paradigm for learning environments: animated pedagogical agents. These lifelike autonomous characters cohabit learning environments with students to create rich, face-to-face learning interactions. This opens up exciting new possibilities; for example, agents can demonstrate complex tasks, employ locomotion and gesture to focus students' attention on the most salient aspect of the task at hand, and convey emotional responses to the tutorial situation. Animated pedagogical agents offer great promise for broadening the bandwidth of tutorial communication and increasing learning environments' ability to engage and motivate students. This article sets forth the motivations behind animated pedagogical agents, describes the key capabilities they offer, and discusses the technical issues they raise. The discussion is illustrated with descriptions of a number of animated agents that represent the current state of the art.

Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments

The purpose of this article is to critically deconstruct the term engagement as it applies to peoples' experiences with technology. Through an extensive, critical multidisciplinary literature review and exploratory study of users of Web searching, online shopping, Webcasting, and gaming applications, we conceptually and operationally defined engagement. Building on past research, we conducted semistructured interviews with the users of four applications to explore their perception of being engaged with the technology. Results indicate that engagement is a process comprised of four distinct stages: point of engagement, period of sustained engagement, disengagement, and reengagement. Furthermore, the process is characterized by attributes of engagement that pertain to the user, the system, and user-system interaction. We also found evidence of the factors that contribute to nonengagement. Emerging from this research is a definition of engagement—a term not defined consistently in past work—as a quality of user experience characterized by attributes of challenge, positive affect, endurability, aesthetic and sensory appeal, attention, feedback, variety-novelty, interactivity, and perceived user control. This exploratory work provides the foundation for future work to test the conceptual model in various application areas, and to develop methods to measure engaging user experiences. © 2008 Wiley Periodicals, Inc.

https://open.library.ubc.ca/media/download/pdf/52383/1.0107445/1

What is user engagement? A conceptual framework for defining user engagement with technology

The Behavior Expression Animation Toolkit (BEAT) allows animators to input typed text that they wish to be spoken by an animated human figure, and to obtain as output appropriate and synchronized nonverbal behaviors and synthesized speech in a form that can be sent to a number of different animation systems. The nonverbal behaviors are assigned on the basis of actual linguistic and contextual analysis of the typed text, relying on rules derived from extensive research into human conversational behavior. The toolkit is extensible, so that new rules can be quickly added. It is designed to plug into larger systems that may also assign personality profiles, motion characteristics, scene constraints, or the animation styles of particular animators.

/pdf/beat-the-behavior-expression-animation-toolkit-3i6udjdeg3.pdf

BEAT: the Behavior Expression Animation Toolkit

Effective management of documents on computers has been a central user interface problem for many years. One common approach involves using 2D spatial layouts of icons representing the documents, particularly for information workspace tasks. This approach takes advantage of human 2D spatial cognition. More recently, several 3D spatial layouts have engaged 3D spatial cognition capabilities. Some have attempted to use spatial memory in 3D virtual environments. However, there has been no proof to date that spatial memory works the same way in 3D virtual environments as it does in the real world. We describe a new technique for document management called the Data Mountain, which allows users to place documents at arbitrary positions on an inclined plane in a 3D desktop virtual environment using a simple 2D interaction technique. We discuss how the design evolved in response to user feedback. We also describe a user study that shows that the Data Mountain does take advantage of spatial memory. Our study shows that the Data Mountain has statistically reliable advantages over the Microsoft Internet Explorer Favorites mechanism for managing documents of interest in an information workspace.

/pdf/data-mountain-using-spatial-memory-for-document-management-34rsai1awg.pdf

Data mountain: using spatial memory for document management

Comics have a rich visual vocabulary, and people find them app ing. They are also an effective form of communication. We ha built a system, called Comic Chat, that represents on-line com nications in the form of comics. Comic Chat automates numer aspects of comics generation, including balloon construction layout, the placement and orientation of comic characters, the fault selection of character gestures and expressions, the incorp tion of semantic panel elements, and the choice of zoom facto the virtual camera. This paper describes the mechanisms Comic Chat uses to perform this automation, as well as novel pects of the program’s user interface. Comic Chat is a working program, allowing groups of people to communicate over the Inter It has several advantages over other graphical chat programs cluding the availability of a graphical history, and a dynamic grap ical presentation.

Comic Chat

This article describes the Persona project at Microsoft Research, which is exploring social user interfaces that employ an explicitly anthropomorphic character to interact with the user in a natural spoken dialogue. The prototype system described here integrates spoken language input, a simple conversational dialogue manager, reactive 3D animation, speech output and sound effects to create Peedy the Parrot, a conversational assistant who accepts user requests for audio CDs and then plays them. 1. Lifelike Computer Characters: the Persona project at Microsoft Research ______________________________________________________1 1.

Lifelike computer characters: the persona project at Microsoft

https://kurlander.net/DJ/Pubs/peedycha.pdf

Lifelike Computer Characters: The Persona Project at Microsoft Research

Computer users have tasks they need to perform, and are therefore motivated to overcome poorly designed interfaces. With video games, there is no external motivation for the task — if the game’s interface is not compelling and entertaining, the product fails in the marketplace. Many aspects of game design, such as an attractor mode to draw users toward the game, have direct relevance to other domains, such as information kiosks. This panel will consist of video game designers who will relate their design methodologies, techniques, and other experiences which will help HCI designers create more compelling, engaging, and effective interfaces. The panel will consist of panelists’ presentations followed by a large allocation of time for interaction with the audience’s questions. The panelists’ presentations will include demonstration examples drawn from coin-operated and computer-based games.

What HCI designers can learn from video game designers

There are two similar, but equally pressing issues currently facing interface designers -how to make increasingly complex applications usable and how to design effective interfaces for a broad audience, such as might exist for the coming wave of interactive consumer products. This tutorial questions some assumptions behind current interface design and why conventional interaction techniques are often unsatisfying to users, experienced as well as novice. Also examined are some of the underlying social transactions that occur between human and machine, like those that emerge when voice and moving images are added to the interface. The psychology of media viewing is reviewed for insights that may help bridge the gap between passive viewer <andactive user.

Tim Skelly

Papers

Comic Chat

Lifelike computer characters: the persona project at Microsoft

Lifelike Computer Characters: The Persona Project at Microsoft Research

What HCI designers can learn from video game designers

Seductive interfaces: satisfying a mass audience