In 1997, Azuma published a survey on augmented reality (AR). Our goal is to complement, rather than replace, the original survey by presenting representative examples of the new advances. We refer one to the original survey for descriptions of potential applications (such as medical visualization, maintenance and repair of complex equipment, annotation, and path planning); summaries of AR system characteristics (such as the advantages and disadvantages of optical and video approaches to blending virtual and real, problems in display focus and contrast, and system portability); and an introduction to the crucial problem of registration, including sources of registration error and error-reduction strategies.

https://www.cc.gatech.edu/~blair/papers/ARsurveyCGA.pdf

Recent advances in augmented reality

A method includes receiving an image including an eye of a user of a computerized system and identifying, based the image of the eye, a direction of a gaze performed by the user. Based on the direction of the gaze, a region on a display coupled to the computerized system is identified, an operation is performed on content presented in the region.

Gaze-Based Display Control

A method, including receiving, by a computer executing a non-tactile three dimensional (3D) user interface, a set of multiple 3D coordinates representing a gesture by a hand positioned within a field of view of a sensing device coupled to the computer, the gesture including a first motion in a first direction along a selected axis in space, followed by a second motion in a second direction, opposite to the first direction, along the selected axis. Upon detecting completion of the gesture, the non-tactile 3D user interface is transitioned from a first state to a second state.

Three-dimensional user interface session control

An interaction system is described which uses a depth camera to capture a depth image of a physical object placed on, or in vicinity to, an interactive surface. The interaction system also uses a video camera to capture a video image of the physical object. The interaction system can then generate a 3D virtual object based on the depth image and video image. The interaction system then uses a 3D projector to project the 3D virtual object back onto the interactive surface, e.g., in a mirrored relationship to the physical object. A user may then capture and manipulate the 3D virtual object in any manner. Further, the user may construct a composite model based on smaller component 3D virtual objects. The interaction system uses a projective texturing technique to present a realistic-looking 3D virtual object on a surface having any geometry.

Providing an interactive experience using a 3d depth camera and a 3d projector

Architecture that combines multiple depth cameras and multiple projectors to cover a specified space (e.g., a room). The cameras and projectors are calibrated, allowing the development of a multi-dimensional (e.g., 3D) model of the objects in the space, as well as the ability to project graphics in a controlled fashion on the same objects. The architecture incorporates the depth data from all depth cameras, as well as color information, into a unified multi-dimensional model in combination with calibrated projectors. In order to provide visual continuity when transferring objects between different locations in the space, the user's body can provide a canvas on which to project this interaction. As the user moves body parts in the space, without any other object, the body parts can serve as temporary “screens” for “in-transit” data.

Projectors and depth cameras for deviceless augmented reality and interaction.

In mixed reality, occlusions and shadows are important to realize a natural fusion between the real and virtual worlds. In order to achieve this, it is necessary to acquire dense depth information of the real world from the observer's viewing position. The depth sensor must be attached to the see-through HMD of the observer because he/she moves around. The sensor should be small and light enough to be attached to the HMD and should be able to produce a reliable dense depth map at video rate. Unfortunately, however, no such depth sensors are available. We propose a client/server depth-sensing scheme to solve this problem. A server sensor located at a fixed position in the real world acquires the 3-D information of the world, and a client sensor attached to each observer produces the depth map from his/her viewing position using the 3-D information supplied from the server. Multiple clients can share the 3-D information of the server: we call it Share-Z. In this paper, the concept and merits of Share-Z are discussed. An experimental system developed to demonstrate the feasibility of Share-Z is also described.

Share-Z: client/server depth sensing for see-through head-mounted displays

A challenging issue for edge computing is how to correct meaningful information from sensor data while keeping the privacy of the data and individuals. One approach is given by the recent work [Choi et al. “Guaranteeing Local Differential Privacy on Ultra-Low-Power Systems” 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA)]. They point out that low resolution and fixed point characteristics of Ultra-Low-Power implementations may break privacy guarantees due to the low quality of noising. For overcoming this weakness, they introduce the techniques of resampling and thresholding. They also implemented in hardware to show the proposed method achieves both low overhead and high utility while keeping local differential privacy, with sensor/IoT benchmarks. Whereas, this research show some flaw in the existing work above and improve the method. We give the case in which infinite privacy loss still occurs because of low resolution even though the width of the output distribution is restricted, which was not investigated yet by the original paper of ISCA2018. Our major contribution is to propose an improvement to avoid privacy loss from low resolution and to enhance privacy protection in this circuit. Furthermore, we report the experimental results with software simulations, which guarantees the utility of our improving method. Finally, we discuss the power and limitation of our improved method with future challenging issues.

Kaito Taguchi

Papers

Share-Z: client/server depth sensing for see-through head-mounted displays

Towards the Design of Locally Differential Private Hardware System for Edge Computing