Mesurer en temps reel la pose d'une camera relativement a des reperes tridimensionnels identifies dans une image video est un, sinon le pilier fondamental de la realite augmentee. Nous proposons de resoudre ce probleme dans des environnements bâtis, a l'aide de la vision
par ordinateur. Nous montrons qu'un systeme de positionnement plus precis que le GPS, et par ailleurs plus stable, plus rapide et moins couteux en memoire que d'autres systemes de positionnement visuel introduits dans la litterature, peut etre
obtenu en faisant cooperer : approche probabiliste et geometrie aleatoire (detection a contrario des points de fuite de
l'image), apprentissage profond (proposition de boites contenant des facades, elaboration d'un descripteur de facades base sur un reseau de
neurones convolutifs), inference bayesienne (recalage par esperance-maximisation d'un modele geometrique et semantique compact
des facades identifiees) et selection de modele (analyse des mouvements de la camera par suivi de plans textures). Nous decrivons
de plus une methode de modelisation in situ, qui permet d'obtenir de maniere fiable, de par leur confrontation immediate a la realite, des modeles 3D utiles au calcul de pose tel que nous l'envisageons.

Positionnement visuel pour la réalité augmentée en environnement plan

This paper describes a method for online interactive building of piecewise planar environments for immediate use in augmented reality. This system combines user interaction from a camera–mouse and automated tracking/reconstruction methods to recover planar structures of the scene that are relevant for the augmentation task. An important contribution of our algorithm is that the process of tracking and reconstructing planar structures is decomposed into three steps—tracking, computation of the intersection lines of the planes, reconstruction—that can each be visually assessed by the user, making the interactive modeling procedure really robust and accurate with intuitive interaction. Videos illustrating our system both on synthetic and long real-size experiments are available at http://www.loria.fr/~gsimon/vc.

/pdf/interactive-building-and-augmentation-of-piecewise-planar-5sxj5vi1va.pdf

Interactive building and augmentation of piecewise planar environments using the intersection lines

In this paper, we propose a general purpose Augmented Reality AR system that allows to add easily 3D computer generated CG objects into real man-made environments. Our system goes to a very intuitive and easy in situ 3D structure recovery of planar piecewise scenes without using powerful hardware nor commodity sensors. The user simply has to move the camera translation of the focus is mandatory and take two different pictures of the scene and our approach obtains a rough planar piecewise representation of the environment suitable to conduct multi-planar tracking for visual model-based augmented reality and to augment it with virtual objects coherently. Polyhedral representations of scenes are very convenient for manmade environments indoor e.g., offices, rooms, classrooms and outdoor e.g., looking at facades, floor, hence we focus the potential applications of our system to augment simple rooms or urban scenes with virtual imagery.

Fast vision-based scene modeling for augmented reality in unprepared man-made environments

This work addresses two main problems: (i) localization of two cameras observing a 3D scene composed by planar structures; (ii) recovering of the original structure of the scene, i.e. the scene reconstruction and segmentation stages. Although there exist some work intending to deal with these problems, most of them are based on: epipolar geometry, non-linear optimization, or linear systems that do not incorporate geometrical consistency.

In this paper, we propose an iterative linear algorithm exploiting geometrical and algebraic constraints induced by rigidity and planarity in the scene. Instead of solving a complex multi-linear problem, we solve iteratively several linear problems: coplanar features segmentation, planar projective transferring, epipole computation, and all plane intersections. Linear methods allow our approach to be suitable for real-time localization and 3D reconstruction. Furthermore, our approach does not compute the fundamental matrix; therefore it does not face stability problems commonly associated with explicit epipolar geometry computation.

Automatic Camera Localization, Reconstruction and Segmentation of Multi-planar Scenes Using Two Views

This work addresses the problems of (i) self-calibration of a moving camera observing a 3D scene composed by planar structures and (ii) scene segmentation and reconstruction. Although there exist some works intending to deal with these problems, most of them are based on the estimation of the epipolar geometry, non-linear optimization, or linear systems that do not incorporate geometrical consistency and may produce undesirable side-effects. In this paper, we propose a novel iterative linear algorithm that exploits the geometrical and algebraic constraints induced by rigidity and planarity in the scene. Instead of solving a complex multi-linear problem, we solve iteratively several linear problems: coplanar features segmentation, planar projective transferring, epipole computation, and all the plane intersections. Linear methods allow our approach to be suitable for real-time localization and 3D reconstruction, e.g. for autonomous mobile robots applications. Furthermore, we avoid the explicit epipolar geometry computation and all the stability problems commonly associated with it.

Automatic Camera Calibration with Structure and Motion Recovery from Two Views of a Polyhedral Structure

This paper describes an integrated system for building a multiplanar model of the scene as the camera is localized on the fly. The core of this system is a robust and accurate procedure for detecting the intersection line between two planes. User cues are used to assist the system in the mapping tasks. Synthetic results and a long video demonstrate the relevance of the method.

/pdf/detection-of-the-intersection-lines-in-multiplanar-5ay8o0z430.pdf

M.-O. Berger

Papers

Detection of the intersection lines in multiplanar environments: Application to real-time estimation of the camera-scene geometry