Surround-screen projection-based virtual reality: the design and implementation of the CAVE

http://lsc.univ-evry.fr/~davesne/ens/pub/intro_to_AR_full.pdf

Knowledge-based augmented reality

This paper presents an application that uses hand gesture input to control a computer while giving a presentation. In order to develop a prototype of this application, we have defined an interaction model, a notation for gestures, and a set of guidelines to design gestural command sets. This works aims to define interaction styles that work in computerized reality environments. In our application, gestures are used for interacting with the computer as well as for communicating with other people or operating other devices.

Charade: remote control of objects using free-hand gestures

Multimodal systems process two or more combined user input modes— such as speech, pen, touch, manual gestures, gaze, and head and body movements— in a coordinated manner with multimedia system output. This class of systems represents a new direction for computing, and a paradigm shift away from conventional WIMP interfaces. Since the appearance of Bolt’s (1980) “Put That There” demonstration system, which processed speech in parallel with touch-pad pointing, a variety of new multimodal systems has emerged. This new class of interfaces aims to recognize naturally occurring forms of human language and behavior, which incorporate at least one recognitionbased technology (e.g., speech, pen, vision). The development of novel multimodal systems has been enabled by the myriad input and output technologies currently becoming available, including new devices and improvements in recognition-based technologies. This chapter will review the main types of multimodal interfaces, their advantages and cognitive science underpinnings, primary features and architectural characteristics, and general research in the field of multimodal interaction and interface design.

Multimodal interfaces

This paper explores the issues involved in designing and developing network software architectures for large-scale virtual environments. We present our ideas in the context of NPSNET-IV, the first 3-D virtual environment that incorporates both the IEEE 1278 distributed interactive simulation DIS application protocol and the IP multicast network protocol for multiplayer simulation over the Internet.

Npsnet: A network software architecture for largescale virtual environments

A multi-user Virtual World has been implemented combining a flexible-object simulator with a multisensory user interface, including hand motion and gestures, speech input and output, sound output, and 3-D stereoscopic graphics with head-motion parallax. The implementation is based on a distributed client/server architecture with a centralized Dialogue Manager. The simulator is inserted into the Virtual World as a server. A discipline for writing interaction dialogues provides a clear conceptual hierarchy and the encapsulation of state. This hierarchy facilitates the creation of alternative interaction scenarios and shared multiuser environment.

Interactive simulation in a multi-person virtual world

This paper presents a system architecture for creating interactive, multisensory, three-dimensional environments called virtual worlds. The architecture specifically addresses the requirements of virtual worlds for high performance, flexibility, and coordination of concurrent events. Performance is enhanced by a distributed client/server system structure and by efficient overlap of processing time and input/output delay. All processes communicate via asynchronous messages. The functional partitioning of a virtual world requires relatively low bandwidth among the individual processes and the system can be implemented over a conventional local-area network.

A key element of this architecture is a central, event-driven dialogue manager that coordinates concurrent input and output events. The dialogue manager provides a clear separation of the interaction techniques from the content of the virtual world as defined by the application. The system is flexible and easily reconfigurable. An interaction technique can be readily changed or replaced because each interaction device is modularized into a separate server and each interaction modality into a separate subdialogue. Subdialogues can be loaded and dropped dynamically, enabling input/output device remapping and the selection of interaction techniques while a virtual world is running. As an initial test of this architecture we have implemented a virtual world for interacting with data from a computational fluid dynamics simulation.

An architecture for virtual worlds

We describe a software architecture for virtual worlds, built on a base of multiple processes communicating through a central event-driven user interface management system. The virtual world's behavior is specified by a dialogue composed of modular subdialogues or rule sets. In order to achieve high flexibility, device remappability and reusability, the rule sets should be written as independent modules, each encapsulating its own state. Each should be designed according to its purpose in a conceptual hierarchy: it can transform a specific device into a generic device, or transform a generic device into an interaction technique, or, at the top level, map interaction techniques to actions.

Daniel T. Ling

Papers

Interactive simulation in a multi-person virtual world

An architecture for virtual worlds

Dialogue structures for virtual worlds