One embodiment is directed to a system for enabling two or more users to interact within a virtual world comprising virtual world data, comprising a computer network comprising one or more computing devices, the one or more computing devices comprising memory, processing circuitry, and software stored at least in part in the memory and executable by the processing circuitry to process at least a portion of the virtual world data; wherein at least a first portion of the virtual world data originates from a first user virtual world local to a first user, and wherein the computer network is operable to transmit the first portion to a user device for presentation to a second user, such that the second user may experience the first portion from the location of the second user, such that aspects of the first user virtual world are effectively passed to the second user.

System and method for augmented and virtual reality

One embodiment is directed to a user display device comprising a housing frame mountable on the head of the user, a lens mountable on the housing frame and a projection sub system coupled to the housing frame to determine a location of appearance of a display object in a field of view of the user based at least in part on at least one of a detection of a head movement of the user and a prediction of a head movement of the user, and to project the display object to the user based on the determined location of appearance of the display object.

Display system and method

A virtual image display device has a light guide device in which the half mirror layer (the semi-transmissive reflecting film) of the light guide member is formed on the partial area of the first junction surface, and the second junction surface of the light transmitting member is bonded to the first junction surface in at least the exceptional area. Therefore, it is possible to increase the bonding strength of the first junction surface and the second junction surface, namely the strength of the light guide device composed of the light guide member and the light transmitting member combined with each other even in the case in which the attachment force of the half mirror layer (the semi-transmissive reflecting film) with respect to the first junction surface is not sufficiently strong.

Virtual image display device

A method for three-dimensional (3D) sensing. The method includes obtaining a first two-dimensional (2D) skeleton of an object, obtaining a second 2D skeleton of the object different from the first 2D skeleton, and calculating a 3D skeleton of the object based on the first and second 2D skeletons.

Interactive input system and method

Methods and arrangements involving portable user devices such smartphones and wearable electronic devices are disclosed, as well as other devices and sensors distributed within an ambient environment. Some arrangements enable a user to perform an object recognition process in a computationally- and time-efficient manner. Other arrangements enable users and other entities to, either individually or cooperatively, register or enroll physical objects into one or more object registries on which an object recognition process can be performed. Still other arrangements enable users and other entities to, either individually or cooperatively, associate registered or enrolled objects with one or more items of metadata. A great variety of other features and arrangements are also detailed.

Methods for object recognition and related arrangements

Technology is described for providing realistic occlusion between a virtual object displayed by a head mounted, augmented reality display system and a real object visible to the user's eyes through the display. A spatial occlusion in a user field of view of the display is typically a three dimensional occlusion determined based on a three dimensional space mapping of real and virtual objects. An occlusion interface between a real object and a virtual object can be modeled at a level of detail determined based on criteria such as distance within the field of view, display size or position with respect to a point of gaze. Technology is also described for providing three dimensional audio occlusion based on an occlusion between a real object and a virtual object in the user environment.

Realistic occlusion for a head mounted augmented reality display

Technology is described for displaying a collision between objects by an augmented reality display device system. A collision between a real object and a virtual object is identified based on three dimensional space position data of the objects. At least one effect on at least one physical property of the real object is determined based on physical properties of the real object, like a change in surface shape, and physical interaction characteristics of the collision. Simulation image data is generated and displayed simulating the effect on the real object by the augmented reality display. Virtual objects under control of different executing applications can also interact with one another in collisions.

Displaying a collision between real and virtual objects

A system and method are disclosed relating to a pipeline for generating a computer model of a target user, including a hand model of the user's hands and fingers, captured by an image sensor in a NUI system The computer model represents a best estimate of the position and orientation of a user's hand or hands The generated hand model may be used by a gaming or other application to determine such things as user gestures and control actions

System for finger recognition and tracking

A system and method are disclosed relating to a pipeline for generating a computer model of a target user, including a hand model of the user's hands, captured by an image sensor in a NUI system. The computer model represents a best estimate of the position of a user's hand or hands and whether the hand or hand is in an open or closed state. The generated hand model may be used by a gaming or other application to determine such things as user gestures and control actions.

System for recognizing an open or closed hand

Two-handed interactions with a natural user interface are disclosed. For example, one embodiment provides a method comprising detecting via image data received by the computing device a context-setting input performed by a first hand of a user. and sending to a display a user interface positioned based on a virtual interaction coordinate system, the virtual coordinate system being positioned based upon a position of the first hand of the user. The method further includes detecting via image data received by the computing device an action input performed by a second hand of the user, the action input performed while the first hand of the user is performing the context-setting input, and sending to the display a response based on the context-setting input and an interaction between the action input and the virtual interaction coordinate system.

Kyungsuk David Lee

Papers

Realistic occlusion for a head mounted augmented reality display

Displaying a collision between real and virtual objects

System for finger recognition and tracking

System for recognizing an open or closed hand

Two-hand interaction with natural user interface