Showing papers by "Luc Van Gool published in 2001"

3D MURALE: a multimedia system for archaeology

Abstract: This paper introduces the 3D Measurement and Virtual Reconstruction of Ancient Lost Worlds of Europe system (3D MURALE). It consists of a set of tools for recording, reconstructing, encoding, visualising and database searching/querying that operate on buildings, building parts, statues, statue parts, pottery, stratigraphy, terrain geometry and texture and material texture. The tools are loosely linked together by a common database on which they all have the facility to store and access data. The paper describes the overall architecture of the 3D MURALE system and then briefly describes the functionality of the tools provided by the project. The paper compares the multimedia studio architecture adopted in this project with other multimedia studio architectures.

Image-based 3D acquisition of archaeological heritage and applications

Abstract: In this paper an approach is presented that obtains virtual models from sequences of images. The system can deal with uncalibrated image sequences acquired with a hand-held camera. Based on tracked or matched features the relations between multiple views are computed. From this both the structure of the scene and the motion of the camera are retrieved. The ambiguity on the reconstruction is restricted from projective to metric through auto-calibration. A flexible multi-view stereo matching scheme is used to obtain a dense estimation of the surface geometry. From the computed data virtual models can be constructed or, inversely, virtual models can be included in the original images.

A compact model for viewpoint dependent texture synthesis. Discussion

Abstract: A texture synthesis method is presented that generates similar texture from an example image. It is based on the emulation of simple but rather carefully chosen image intensity statistics. The resulting texture models are compact and no longer require the example image from which they were derived. They make explicit some structural aspects of the textures and the modeling allows knitting together different textures with convincingly looking transition zones. As textures are seldom flat, it is important to also model 3D effects when textures change under changing viewpoint. The simulation of such changes is supported by the model, assuming examples for the different viewpoints are given.

Structure and motion from image sequences

Abstract: In this paper an approach is presented that obtains virtual models from sequences of images. The system can deal with uncalibrated image sequences acquired with a hand held camera. Based on tracked or matched features the relations between multiple views are computed. From this both the structure of the scene and the motion of the camera are retrieved. The ambiguity on the reconstruction is restricted from projective to metric through auto-calibration. A flexible multi-view stereo matching scheme is used to obtain a dense estimation of the surface geometry. From the computed data virtual models can be constructed or, inversely, virtual models can be included in the original images.

A guided tour to virtual Sagalassos

Abstract: The system presented in this paper allows to take a guided tour to a virtual reconstruction of the ancient site of Sagalassos. The site has been reconstructed by combining 3D models of the landscape and remains with CAD reconstructions of monuments based on archaeological hypothesis. The 3D models were obtained using advanced image-based modelling techniques developed over the last few years in computer vision. The visitor can communicate with the virtual guide through natural speech. The guide itself is represented by a 3D face model.

Tracking based structure and motion recovery for augmented video productions

Abstract: Augmented Reality (AR) can hardly be called uncharted territory. Much research in this area revealed solutions to the three most prominent challenges of AR: accurate camera state retrieval, resolving occlusions between real and virtual objects and extraction of environment illumination distribution. Solving these three challenges improves the illusion of virtual entities belonging to our reality. This paper demonstrates an elaborated framework that recovers accurate camera states from a video sequence based on feature tracking. Without prior calibration knowledge, it is able to create AR Video products with negligible/invisible jitter or drift of virtual entities starting from general input video sequences. Together with the referenced papers, this work describes a readily implementable and robust AR-System.

Local Features for Image Retrieval

Abstract: Global image features, although quite successful for a surprisingly large number of queries, are nevertheless restricted in their usefulness. They are less effective when e.g. looking for other images that contain the same object or simply the same scene but seen from a very different viewpoint. Relative amounts of colour and texture typically change dramatically in such cases. Hence, more local features are needed as well, that capture local image structure in a way that is robust against changes in viewpoint and illumination. We propose affinely invariant regions. These are extracted from single images in isolation. Their image shapes are viewpoint dependent in that they systematically cover the same physical part of surfaces. These ‘affinely invariant regions’ work in principle perfectly for planar shapes under pseudo-orthographic viewing conditions. Keeping them small helps to fulfill these conditions. Several ways of extracting such regions are discussed. Next, features are extracted from the colour patterns within the regions. These are invariant under the geometric and photometric changes that are considered. As a consequence, they can be directly compared between images. This is the basis for our simple retrieval algorithm.

3D Structure from Images — SMILE 2000

Abstract: This paper presents techniques and animations developed from 1991 to 2000 that use digital photographs of the real world to create 3D models, virtual camera moves, and realistic computer animations. In these projects, images are used to determine the structure, appearance, and lighting conditions of the scenes. Early work in recovering geometry (and generating novel views) from silhouettes and stereo correspondence are presented, which motivate Façade, an interactive photogrammetric modeling system that uses geometric primitives to model the scene. Subsequent work has been done to recover lighting and reflectance properties of real scenes, to illuminate synthetic objects with light captured from the real world, and to directly capture reflectance fields of real-world objects and people. The projects presented include The Chevette Project (1991), Immersion 94 (1994), Rouen Revisited (1996), The Campanile Movie (1997), Rendering with Natural Light (1998), Fiat Lux (1999), and the Light Stage (2000).

3D acquisition of archaeological heritage from images

Abstract: In this contribution an approach is proposed that can capture the 3D shape and appearance of objects, monuments or sites from photographs or video. The flexibility of the approach allows us to deal with uncalibrated hand-held camera images. In addition, through the use of advanced computer vision algorithms the process is largely automated. Both these factors make the approach ideally suited to applications in archaeology. Not only does it become feasible to obtain photo-realistic virtual reconstructions of monuments and sites, but also stratigraphy layers and separate building blocks can be reconstructed. These can then be used as detailed records of the excavations or allow virtual re-assemblage of monuments. Since the motion of the camera is also computed, it also becomes possible to augment video streams of ancient remains with virtual reconstruction. The proposed approach retrieves both the structure of a scene and the motion of the camera from an image sequence. In a first step features are extracted and matched over consecutive images. This step is followed by a structurefrom-motion algorithm that yields a sparse 3D reconstruction (i.e. the matched 3D features) and the path of the camera. These results are enhanced through auto-calibration and bundle adjustment. To allow a full surface reconstruction of the observed scene, the images are rectified so that a standard stereo algorithm can be used to determine dense disparity maps. By combining several of these maps, accurate depth maps are computed. These can then be integrated together to yield a dense 3D surface model. By making use of texture mapping photo-realistic models can be obtained.

Image-based 3D acquisition tool for architectural conservation

Abstract: In this paper an image-based 3D acquisition tool is presented. The tool has been developed with architectural conservation in mind. The proposed tool is developed as part of a project that aims at developing a technology that enables the operator to build up an accurate three dimensional model without too much repetitive work starting from photos of the objects and measured reference coordinates. This model can in a later phase be used as a core for a multimedia database, to represent designs for interventions, or for distribution to the wider public. The presented 3D acquisition tool is largely automated. A limited amount of interaction is used to indicate the reference points in the images to allow global localization in world coordinates. These points are also used to simplify the matching process so that wide baselines can be coped with. Depending on the number of reference points the camera needs to be pre-calibrated or not. From this point on the approach automatically finds additional matches and refines the calibration. A globally consistent estimate can be obtained through bundle adjustment. The next step consists of dense depth estimation based on stereo matching. From the computed information a dense textured 3D surface reconstruction of the recorded structure is computed. The tool also allows extracting 2D slices or pointing measurements. The results can be used both for measurements and for visualization purposes.

A Stereo Vision System for Support of Planetary Surface Exploration

Abstract: In this paper a system will be presented that was developed for ESA for the support of planetary exploration. The system that is sent to the planetary surface consists of a rover and a lander. The lander contains a stereo head equipped with a pan-tilt mechanism. This vision system is used both for modeling of the terrain and for localization of the rover. Both tasks are necessary for the navigation of the rover. Due to the stress that occurs during the flight a recalibration of the stereo vision system is required once it is deployed on the planet. Due to practical limitations it is infeasible to use a known calibration pattern for this purpose and therefore a new calibration procedure had to be developed that can work on images of the planetary environment. This automatic procedure recovers the relative orientation of the cameras and the pan-and tilt-axis, besides the exterior orientation for all the images. The same images are subsequently used to recover the 3D structure of the terrain. For this purpose a dense stereo matching algorithm is used that - after rectification - computes a disparity map. Finally, all the disparity maps are merged into a single digital terrain model. In this paper a simple and elegant procedure is proposed that achieves that goal. The fact that the same images can be used for both calibration and 3D reconstruction is important since in general the communication bandwidth is very limited. In addition to the use for navigation and path planning, the 3D model of the terrain is also used for Virtual Reality simulation of the mission, in which case the model is texture mapped with the original images. The system has been implemented and the first tests on the ESA planetary terrain testbed were successful.

Vision on Conservation: Virterf

Abstract: Although more and more computer-aided technologies are utilised or demonstrated on well-known monuments or archaeological sites, there is - at least for the professionals working in the field - still some lack of clarity about their application. The main idea of our research project is to construct a prototype of a tool that tries to bring new technologies closer to the daily practice. It incorporates recent developments in computer vision and reverse engineering, while at the same time tries to answer the practical concerns from civil servants, architects, topographers, art-historians,… who are responsible for the conservation of monuments or archaeological sites. The proposed system covers automatic correspondence analysis and point cloud manipulation to build up a textured 3D model. This model then acts as the central core of a multimedia data structure for the annotation, geometric and thematic interrogation and visualisation of the building or site being studied.

A probabilistic approach to roof patch extraction and reconstruction

Abstract: This paper investigates into the reconstruction of planar polygonal patches in a 3D scene from high resolution colour stereo imagery. 3D line segments, obtained from multiview correspondence analysis, are grouped into planes. In these planes, models for polygonal patches are then instantiated and verified using a Bayesian probabilistic formulation. Driven by the Maximum Expected Utility principle, the initial patch hypotheses are improved non-deterministically. Further patches are instantiated, verified and improved using a bootstrap strategy, until the complete reconstruction is found. Results for both, the intermediate reasoning steps and the complete reconstruction are given. A main application of the presented method is for automatic building reconstruction from high resolution aerial imagery in densely built up urban areas.

Grouping via the Matching of Repeated Patterns

Abstract: In this contribution, a novel and robust, geometry-based grouping strategy is proposed. Repeated, planar patterns in special relative positions are detected. The grouping is based on the idea of fixed structures. These are structures such as lines or points that remain fixed under the transformations mapping the patterns onto each other. As they define subgroups of the general group of projectivities, they significantly reduce the complexity of the problem. First, some initial matches are found by comparing local, affinely invariant regions. Then, possible fixed structure candidates are hypothesized using a cascaded Hough transform. In a further step, these candidates are verified. In this paper, we concentrate on planar homologies, i.e. subgroups that have a line of fixed points and a pencil of fixed lines.

Lip animation based on observed 3D speech dynamics

Abstract: We are all experts in the perception and interpretation of faces and their dynamics. This makes facial animation a particularly demanding area of graphics. Increasingly, computer vision is brought to bear and 3D models and their motions are learned from observations. The paper subscribes to this strand for the 3D modeling of human speech. The approach follows a kind of bootstrap procedure. First, 3D shape statistics are learned from faces with a few markers. A 3D reconstruction of a speaking face is produced for each video frame. A topological mask of the lower half of the face is fitted to the motion. The 3D shape statistics are extracted and pricipal components analysis (PCA) reduces the dimension of the maskspace. The final speech tracker can work without markers, as it is only allowed to roam this constrained space of masks. Upon the representation of the different visemes in this space, speech or text can be used as input for animation.

Composite textures: emulating building materials and vegetation for 3D models

Abstract: In building 3D site models for visualization and virtual walkthrough, most emphasis so far has been on creating the 3D shape models. Less emphasis has been on creating realistic textures, e.g. to simulate building materials or vegetation. Nevertheless, the appearance of object and landscape models will depend at least as much on their textures, as on the precision of their geometry. The paper proposes a texture synthesis technique for the simulation of building materials and vegetation types. As textures can often more easily be described as a composition of subtextures than as a single texture, a hierarchical extension of our basic texture synthesis is proposed. These "composite textures" are based on a kind of metatexture concept, i.e. the layout of the different subtextures is itself modeled as a texture, that can be generated automatically. Examples are shown for building materials with an intricate structure and for the automatic creation of landscape textures.

From a conservationist's point of view

Abstract: Although more and more computer-aided technologies are utilised or demonstrated on well-known monuments or archaeological sites, there is at least for the professionals working in the field still some lack of clarity about their application. The main idea of our research project is to construct a prototype of a tool that tries to bring new technologies closer to the daily practice. It incorporates recent developments in computer vision and reverse engineering, while at the same time tries to answer the practical concerns from civil servants, architects, topographers, art-historians... who are responsible for the conservation of monuments or archaeological sites. The proposed system covers automatic correspondence analysis and point cloud manipulation to build up a textured 3D model. This model then acts as the central core of a multimedia data structure for the annotation, geometric and thematic interrogation and visualisation of the building or site being studied.

Calibration, terrain reconstruction and path planning for a planetary exploration system

Abstract: In this paper, three important preparation tasks of a planetary exploration system are described, namely calibration, terrain reconstruction and path planning. The calibration is retrieved from the images of the planetary terrain (the same images from which the 3D measurements of the terrain are obtained). Once the system has been calibrated, the same images can be used to estimate a digital elevation map of the environment around the lander. These images are r st processed pairwise using a stereo algorithm yielding sub-pixel disparity maps. In order to perform the path planning a digital elevation map is required. An algorithm is described which can generate a regular digital elevation map at any desired resolution (interpolating if necessary) and easily takes occlusions into account. The results of the calibration and reconstruction are then used by the path planning module which is capable of computing the optimal path between points of interest of the terrain.

Virtual Models from Video and Vice-Versa

Abstract: In this paper an approach is presented that obtains virtual models from sequences of images. The system can deal with uncalibrated image sequences acquired with a hand-held camera. Based on tracked or matched features the relations between multiple views are computed. From this both the structure of the scene and the motion of the camera are retrieved. The ambiguity on the reconstruction is restricted from projective to metric through auto-calibration. A flexible multi-view stereo matching scheme is used to obtain a dense estimation of the surface geometry. From the computed data virtual models can be constructed or, inversely, virtual models can be included in the original images.

Augmented Reality using uncalibrated video sequences. Discussion

Abstract: Augmented Reality(AR) aims at merging the real and the virtual in order to enrich a real environment with virtual information. Augmentations range from simple text annotations accompanying real objects to virtual mimics of real-life objects inserted into a real environment. In the latter case the ultimate goal is to make it impossible to differentiate between real and virtual objects. Several problems need to be overcome before realizing this goal. Amongst them are the rigid registration of virtual objects into the real environment, the problem of mutual occlusion of real and virtual objects and the extraction of the illumination distribution of the real environment in order to render the virtual objects with this illumination model. This paper will unfold how we proceeded to implement an Augmented Reality System that registers virtual objects into a totally uncalibrated video sequence of a real environment that may contain some moving parts. The other problems of occlusion and illumination will not be discussed in this paper but are left as future research topics.

Using shape to correct for observed nonuniform color in automated egg grading

Abstract: We report on algorithmic aspects for the automated visual quality control for grading of brown eggs. Using RGB color images of four different views of every egg enabled to analyze the entire eggshell. The scene was illuminated using a set of white fluorescent tubes placed in a rectangular grid. After detection and approximation of the egg contour (ellipse fitted), the color was corrected to compensate for the elliptical shape of the eggs. A second order polynomial was fitted through points taken from subsequent horizontal lines inside the egg. Iteration was used to reject outliers (most likely points with visual defects). The shape- corrected intensity was calculated as the signed difference between polynomial and measured value, increased with the average egg intensity. Based on the corrected color, dirt regions like yolk, manure, blood, and red mite spots were segmented from the egg-background. Features based on color and shapes were calculated for every segmented region as the combined space and color moments of zeroth, first and second order. A classifier identified most of the defective eggs. Elimination of false rejects due to mirror reflection of the light tubes on some eggs (segmented because of the different color) is currently under investigation.© (2001) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Method for the production of a virtual reality environment

Abstract: The method for the presentation of an object in a virtual reality environment of a user is based upon the preparation of a concept data set with information on the properties of the objet, for example, a second user. The primary data of the object, containing video images are processed to give secondary data at the place of data recording by means of the concept data set. The secondary data thus obtained is transmitted to a virtual reality location. A virtual reality object, representing the object is created based on the secondary data, integrated in the virtual reality environment. An example of a concept data set is a position data set, which contains positional data of the virtual reality object in the virtual reality environment.