In 1997, Azuma published a survey on augmented reality (AR). Our goal is to complement, rather than replace, the original survey by presenting representative examples of the new advances. We refer one to the original survey for descriptions of potential applications (such as medical visualization, maintenance and repair of complex equipment, annotation, and path planning); summaries of AR system characteristics (such as the advantages and disadvantages of optical and video approaches to blending virtual and real, problems in display focus and contrast, and system portability); and an introduction to the crucial problem of registration, including sources of registration error and error-reduction strategies.

https://www.cc.gatech.edu/~blair/papers/ARsurveyCGA.pdf

Recent advances in augmented reality

https://www.cs.ucla.edu/~dt/papers/mia96/mia96.pdf

Deformable models in medical image analysis: a survey

A morphable model for the synthesis of 3D faces

We propose a new model for active contours based on a geometric partial differential equation. Our model is intrinsec, stable (satisfies the maximum principle) and permits a rigorous mathematical analysis. It enables us to extract smooth shapes (we cannot retrieve angles) and it can be adapted to find several contours simultaneously. Moreover, as a consequence of the stability, we can design robust algorithms which can be engineed with no parameters in applications. Numerical experiments are presented.

A geometric model for active contours in image processing

Automatic generation and detection of highly reliable fiducial markers under occlusion

https://www.cs.jhu.edu/~padoy/media/pubs/padoy-media2010-preprint.pdf

Statistical modeling and recognition of surgical workflow.

This paper presents a method for real-time augmentation of vascular network and tumors during minimally invasive liver surgery. Internal structures computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Compared to state-of-the-art methods, our method uses a real-time biomechanical model to compute a volumetric displacement field from partial three-dimensional liver surface motion. This permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Real-time augmentation results are presented on in vivo and phantom data and illustrate the benefits of such an approach for minimally invasive surgery.

/pdf/image-guided-simulation-of-heterogeneous-tissue-deformation-4q6pg3fjma.pdf

Image-guided simulation of heterogeneous tissue deformation for augmented reality during hepatic surgery

This paper addresses the problem of 3D surface reconstruction using image sequences. It has been shown that shape recovery from three or more occluding contours of the surface is possible given a known camera motion. Several algorithms, which have been recently proposed, allow such a reconstruction under the assumption of a linear camera motion. A new approach is presented which deals with the reconstruction problem directly from a discrete point of view. First, a theoretical study of the epipolar correspondence between occluding contours is achieved. A correct depth formulation is then derived from a local approximation of the surface up to order two. This allows the local shape to be estimated, given three consecutive contours, without any constraints on the camera motion. Experimental results are presented for both synthetic and real data.

/pdf/3d-surface-reconstruction-using-occluding-contours-14st1cc048.pdf

3D Surface Reconstruction Using Occluding Contours

We present a new approach for resolving occlusions in augmented reality. The main interest is that it does not require 3D reconstruction of the considered scene. Our idea is to use a contour based approach and to label each contour point as being "behind" or "in front of", depending on whether it is in front of or behind the virtual object. This labeling step only requires that the contours can be tracked from frame to frame. A proximity graph is then built in order to group the contours that belong to the same occluding object. Finally, we use some kind of active contours to accurately recover the mask of the occluding object.

Resolving occlusion in augmented reality: a contour based approach without 3D reconstruction

We address the registration problem for interactive AR applications. Such applications require a real-time registration process. Although the registration problem has received a lot of attention in the computer vision community, it's far from being solved. Ideally, an AR system should work in all environments without the need to prepare the scene ahead of time, and users should be able to walk anywhere they want. In the past, several AR systems have achieved accurate and fast tracking and registration, putting dots over objects and tracking the dots with a camera. We can also achieve registration by identifying features in the scene that we can carefully measure for real-world coordinates. However, such methods restrict the system's flexibility. Hence, we need to investigate registration methods that work in unprepared environments and reduce the need to know the objects' geometry in the scene. We propose an efficient solution to real-time camera tracking for scenes that contain planar structures. We can consider many types of scene with our method. We show that our system is reliable and we can use it for real-time applications. We also present results demonstrating real-time camera tracking on indoor and outdoor scenes.

Marie-Odile Berger

Papers

Statistical modeling and recognition of surgical workflow.

Image-guided simulation of heterogeneous tissue deformation for augmented reality during hepatic surgery

3D Surface Reconstruction Using Occluding Contours

Resolving occlusion in augmented reality: a contour based approach without 3D reconstruction

Pose estimation for planar structures