This paper presents a method of estimating camera pose in an unknown scene. While this has previously been attempted by adapting SLAM algorithms developed for robotic exploration, we propose a system specifically designed to track a hand-held camera in a small AR workspace. We propose to split tracking and mapping into two separate tasks, processed in parallel threads on a dual-core computer: one thread deals with the task of robustly tracking erratic hand-held motion, while the other produces a 3D map of point features from previously observed video frames. This allows the use of computationally expensive batch optimisation techniques not usually associated with real-time operation: The result is a system that produces detailed maps with thousands of landmarks which can be tracked at frame-rate, with an accuracy and robustness rivalling that of state-of-the-art model-based systems.

/pdf/parallel-tracking-and-mapping-for-small-ar-workspaces-4qm7cjaut2.pdf

Parallel Tracking and Mapping for Small AR Workspaces

In 1997, Azuma published a survey on augmented reality (AR). Our goal is to complement, rather than replace, the original survey by presenting representative examples of the new advances. We refer one to the original survey for descriptions of potential applications (such as medical visualization, maintenance and repair of complex equipment, annotation, and path planning); summaries of AR system characteristics (such as the advantages and disadvantages of optical and video approaches to blending virtual and real, problems in display focus and contrast, and system portability); and an introduction to the crucial problem of registration, including sources of registration error and error-reduction strategies.

https://www.cc.gatech.edu/~blair/papers/ARsurveyCGA.pdf

Recent advances in augmented reality

Many applications require tracking of complex 3D objects. These include visual servoing of robotic arms on specific target objects, Augmented Reality systems that require real-time registration of the object to be augmented, and head tracking systems that sophisticated interfaces can use. Computer Vision offers solutions that are cheap, practical and non-invasive.This survey reviews the different techniques and approaches that have been developed by industry and research. First, important mathematical tools are introduced: Camera representation, robust estimation and uncertainty estimation. Then a comprehensive study is given of the numerous approaches developed by the Augmented Reality and Robotics communities, beginning with those that are based on point or planar fiducial marks and moving on to those that avoid the need to engineer the environment by relying on natural features such as edges, texture or interest. Recent advances that avoid manual initialization and failures due to fast motion are also presented. The survery concludes with the different possible choices that should be made when implementing a 3D tracking system and a discussion of the future of vision-based 3D tracking.Because it encompasses many computer vision techniques from low-level vision to 3D geometry and includes a comprehensive study of the massive literature on the subject, this survey should be the handbook of the student, the researcher, or the engineer who wants to implement a 3D tracking system.

/pdf/monocular-model-based-3d-tracking-of-rigid-objects-a-survey-1tb409lav4.pdf

Monocular Model-Based 3D Tracking of Rigid Objects: A Survey

This survey summarizes almost 50 years of research and development in the field of Augmented Reality AR. From early research in the1960's until widespread availability by the 2010's there has been steady progress towards the goal of being able to seamlessly combine real and virtual worlds. We provide an overview of the common definitions of AR, and show how AR fits into taxonomies of other related technologies. A history of important milestones in Augmented Reality is followed by sections on the key enabling technologies of tracking, display and input devices. We also review design guidelines and provide some examples of successful AR applications. Finally, we conclude with a summary of directions for future work and a review of some of the areas that are currently being researched.

/pdf/a-survey-of-augmented-reality-2ce6xet7y5.pdf

A Survey of Augmented Reality

Augmented reality (AR) allows to seamlessly insert virtual objects in an image sequence. In order to accomplish this goal, it is important that synthetic elements are rendered and aligned in the scene in an accurate and visually acceptable way. The solution of this problem can be related to a pose estimation or, equivalently, a camera localization process. This paper aims at presenting a brief but almost self-contented introduction to the most important approaches dedicated to vision-based camera localization along with a survey of several extension proposed in the recent years. For most of the presented approaches, we also provide links to code of short examples. This should allow readers to easily bridge the gap between theoretical aspects and practical implementations.

/pdf/pose-estimation-for-augmented-reality-a-hands-on-survey-2liumlcgu0.pdf

Pose Estimation for Augmented Reality: A Hands-On Survey

We describe a markerless camera tracking system for augmented reality that operates in environments which contain one or more planes. This is a common special case, which we show significantly simplifies tracking. The result is a practical, reliable, vision-based tracker. Furthermore, the tracked plane imposes a natural reference frame, so that the alignment of the real and virtual coordinate systems is rather simpler than would be the case with a general structure-and-motion system. Multiple planes can be tracked, and additional data such as 2D point tracks are easily incorporated.

/pdf/markerless-tracking-using-planar-structures-in-the-scene-4lxemxpsgp.pdf

Markerless tracking using planar structures in the scene

We address the registration problem for interactive AR applications. Such applications require a real-time registration process. Although the registration problem has received a lot of attention in the computer vision community, it's far from being solved. Ideally, an AR system should work in all environments without the need to prepare the scene ahead of time, and users should be able to walk anywhere they want. In the past, several AR systems have achieved accurate and fast tracking and registration, putting dots over objects and tracking the dots with a camera. We can also achieve registration by identifying features in the scene that we can carefully measure for real-world coordinates. However, such methods restrict the system's flexibility. Hence, we need to investigate registration methods that work in unprepared environments and reduce the need to know the objects' geometry in the scene. We propose an efficient solution to real-time camera tracking for scenes that contain planar structures. We can consider many types of scene with our method. We show that our system is reliable and we can use it for real-time applications. We also present results demonstrating real-time camera tracking on indoor and outdoor scenes.

Pose estimation for planar structures

This paper addresses the registration problem for unprepared multi-planar scenes. An interactive process is proposed to obtain accurate results using only the texture information of planes. In particular, classical preparation steps (camera calibration, scene acquisition) are greatly simplified, since they are included in the on-line registration process. Results are shown on indoor and outdoor scenes. Videos are available at url http://www.loria.fr//spl tilde/gsimon/Ismar.

https://hal.inria.fr/inria-00099442/document

Reconstructing while registering: a novel approach for markerless augmented reality

This paper describes an ongoing research comparing two 3D astronomical tangible models: an Augmented Reality model versus a physical model. According to IBSE principles, learners should investigate and manipulate in order to become conscious of the origin of astronomical phenomena, construct scientific knowledge and change their misconceptions. In primary French schools, physical models are usually used. However, children do not take advantage of these models and form new synthetic models instead of scientific ones. We aim at providing an adapted pedagogical environment support. An Augmented Reality environment was designed for inquiry-based learning. This tangible AR model shows augmented views of the celestial bodies and supports the pupils' investigations using spatial visual guides and views from a terrestrial observer. The AR model not only exposes the phenomena as in several Virtual Environments, but also allows pupils to virtually move the celestial bodies and test "as for real" their hypotheses. Our results show that the AR environment is particularly suitable for astronomy learning compared to the physical one. Only AR users have developed scientific conceptions of the explored astronomical phenomena and learnings have been significantly improved. Furthermore, we present some arguments in order to support the assumption that the AR model assists the process of scaffolding and motivation dynamic by enhancing task controllability and by promoting collaborative learning.

/pdf/an-augmented-reality-environment-for-astronomy-learning-in-3yrb6ev4zz.pdf

An Augmented Reality Environment for Astronomy Learning in Elementary Grades: An Exploratory Study

A model registration system capable of tracking an object, the model of which is known, in an image sequence is presented. It integrates tracking, pose determination and updating of the visible features. The heart of our system is the pose computation method, which handles various features (points, lines and free-form curves) in a very robust way and is able to give a correct estimate of the pose even when tracking errors occur. The reliability of the system is shown on an augmented reality project.

https://hal.inria.fr/inria-00073454/document

Gilles Simon

Papers

Markerless tracking using planar structures in the scene

Pose estimation for planar structures

Reconstructing while registering: a novel approach for markerless augmented reality

An Augmented Reality Environment for Astronomy Learning in Elementary Grades: An Exploratory Study

A two-stage robust statistical method for temporal registration from features of various type