Architecture for implementing a perceptual user interface. The architecture comprises alternative modalities for controlling computer application programs and manipulating on-screen objects through hand gestures or a combination of hand gestures and verbal commands. The perceptual user interface system includes a tracking component that detects object characteristics of at least one of a plurality of objects within a scene, and tracks the respective object. Detection of object characteristics is based at least in part upon image comparison of a plurality of images relative to a course mapping of the images. A seeding component iteratively seeds the tracking component with object hypotheses based upon the presence of the object characteristics and the image comparison. A filtering component selectively removes the tracked object from the object hypotheses and/or at least one object hypothesis from the set of object hypotheses based upon predetermined removal criteria.

Architecture for controlling a computer using hand gestures

Accurate simulation of sport to quantify and train performance constructs by employing sensing electronics for determining, in essentially real time, the player's three dimensional positional changes in three or more degrees of freedom (three dimensions); and computer controlled sport specific cuing that evokes or prompts sport specific responses from the player that are measured to provide meaningful indicia of performance. The sport specific cuing is characterized as a virtual opponent that is responsive to, and interactive with, the player in real time. The virtual opponent continually delivers and/or responds to stimuli to create realistic movement challenges for the player.

System and method for tracking and assessing movement skills in multidimensional space

Head-worn adaptive display

A chat system, method and computer program product for delivering a message between a sender and a recipient in a three-dimensional (3D) multi-user environment, wherein the 3D multi-user environment maintains respective digital representations of the sender and the recipient, uses a recipient interface to receive a message, map the message to a texture to generate a textured message, and render the textured message in the 3D multi-user environment so as to permit the recipient to visually ascertain the location of the digital representation of the sender in the 3D world. Received messages are mantained as two-dimensional elements on a recipient viewport.

Method, system and computer program product for the delivery of a chat message in a 3D multi-user environment

Techniques for facilitating interaction with an application in a motion capture system allow a person to easily begin interacting without manual setup. A depth camera system tracks a person in physical space and evaluates the person's intent to engage with the application. Factors such as location, stance, movement and voice data can be evaluated. Absolute location in a field of view of the depth camera, and location relative to another person, can be evaluated. Stance can include facing a depth camera, indicating a willingness to interact. Movements can include moving toward or away from a central area in the physical space, walking through the field of view, and movements which occur while standing generally in one location, such as moving one's arms around, gesturing, or shifting weight from one foot to another. Voice data can include volume as well as words which are detected by speech recognition.

Recognizing user intent in motion capture system

In a virtual reality system (20) an operator perceives being in a virtual reality world (74) of virtual reality images (62). The system (20) models the relative position and movement of objects (208 and 210) in a virtual reality world (74) by representing graphically a first (208) and second (210) object in a virtual reality world (74) on a graphical display (60 and 68). The representation selective obscures the first (208) and second (210) objects according to the relative position of the objects (208 and 210) to a predetermined observation point in the virtual reality world (74). The system (20) then determines a first partitioning plane (204) between the first (208) and second (210) objects. Then, the system (20) method determines a second partition plane (206) between the first (208) and second (210) object in response to either of said first (208) or second (210) objects moving across the first partitioning plane (204). Then, the system (20) graphically represent on the graphical display (60 and 68) the first (208) and second (210) objects within the virtual reality world (74) by selectively obscuring the first (208) and second (210) objects according to their relative position to the predetermined observation point.

Method of modeling player position and movement in a virtual reality system

A 360 degree scene is obtained by image generating devices. These image generating devices obtain a plurality of different video images, each of the image generating devices obtain a different video image of a different field of view of the scene. These different video images are sequentially recorded for use in a virtual reality system.

Method and apparatus for recording images for a virtual reality system

Video images are displayed from a sequence of frames or lines of video signals from a prerecorded source of frames 335 or lines, respectively. A direction that a user is facing is determined by a directional device 200 and a portion of the sequence of frames or lines are stored based on the direction determined by the directional device. The stored portion is read from memory and displayed for the user.

Method and apparatus for the display of video images

Video images are displayed from a sequence of frames or lines of video signals from a prerecorded source of frames or lines, respectively. A direction that a user is facing is determined by a directional device and a portion of the sequence of frames or lines are stored based on the direction determined by the directional device. The stored portion is read from memory and displayed for the user.

Method and apparatus for playback with a virtual reality system

A virtual reality system (20) wherein an operator perceives being in a virtual reality world (74) of virtual reality images (62) comprises a display circuit (36) including a display screen (60) for displaying image data as a virtual reality image (62). Memory circuit contains image data. Helmet device (28) attaches to the operator and contains display screen (60) for generating image data on display screen (60) in response to operator movement and control signals. Microprocessor (76) associates with memory circuit for extracting image data from memory circuit to control display circuit (36) to change virtual reality image (62) as though the operator were within and associated with the virtual reality image. Virtual reality system (20) provides a low-cost system for using a variety of virtual reality techniques. The system and method for modeling the relative position and movement of objects in the virtual reality world (74) includes representing graphically a first and second object in a virtual reality world (74) on a graphical display (60 and 68), determining a partitioning plane between the first and second objects, and determining a second partition plane between the first and second object in response to either of said first or second objects moving across the first partitioning plane. The method and system then graphically represent on the graphical display (60 and 68) movement of the first and second objects within the virtual reality world (74) by selectively obscuring the first and second objects according to their relative position to the predetermined observation point.

Daniel M. Donahue

Papers

Method of modeling player position and movement in a virtual reality system

Method and apparatus for recording images for a virtual reality system

Method and apparatus for the display of video images

Method and apparatus for playback with a virtual reality system

Low-cost system for modeling player position and movement in a virtual reality system