Showing papers on "3D reconstruction published in 2009"

Reconstructing building interiors from images

Abstract: This paper proposes a fully automated 3D reconstruction and visualization system for architectural scenes (interiors and exteriors). The reconstruction of indoor environments from photographs is particularly challenging due to texture-poor planar surfaces such as uniformly-painted walls. Our system first uses structure-from-motion, multi-view stereo, and a stereo algorithm specifically designed for Manhattan-world scenes (scenes consisting predominantly of piece-wise planar surfaces with dominant directions) to calibrate the cameras and to recover initial 3D geometry in the form of oriented points and depth maps. Next, the initial geometry is fused into a 3D model with a novel depth-map integration algorithm that, again, makes use of Manhattan-world assumptions and produces simplified 3D models. Finally, the system enables the exploration of reconstructed environments with an interactive, image-based 3D viewer. We demonstrate results on several challenging datasets, including a 3D reconstruction and image-based walk-through of an entire floor of a house, the first result of this kind from an automated computer vision system.

Light field superresolution

Abstract: Light field cameras have been recently shown to be very effective in applications such as digital refocusing and 3D reconstruction. In a single snapshot these cameras provide a sample of the light field of a scene by trading off spatial resolution with angular resolution. Current methods produce images at a resolution that is much lower than that of traditional imaging devices. However, by explicitly modeling the image formation process and incorporating priors such as Lambertianity and texture statistics, these types of images can be reconstructed at a higher resolution. We formulate this method in a variational Bayesian framework and perform the reconstruction of both the surface of the scene and the (superresolved) light field. The method is demonstrated on both synthetic and real images captured with our light-field camera prototype.

Multi-view image and ToF sensor fusion for dense 3D reconstruction

Abstract: Multi-view stereo methods frequently fail to properly reconstruct 3D scene geometry if visible texture is sparse or the scene exhibits difficult self-occlusions Time-of-Flight (ToF) depth sensors can provide 3D information regardless of texture but with only limited resolution and accuracy To find an optimal reconstruction, we propose an integrated multi-view sensor fusion approach that combines information from multiple color cameras and multiple ToF depth sensors First, multi-view ToF sensor measurements are combined to obtain a coarse but complete model Then, the initial model is refined by means of a probabilistic multi-view fusion framework, optimizing over an energy function that aggregates ToF depth sensor information with multi-view stereo and silhouette constraints We obtain high quality dense and detailed 3D models of scenes challenging for stereo alone, while simultaneously reducing complex noise of ToF sensors

Continuous Global Optimization in Multiview 3D Reconstruction

Abstract: In this article, we introduce a new global optimization method to the field of multiview 3D reconstruction. While global minimization has been proposed in a discrete formulation in form of the maxflow-mincut framework, we suggest the use of a continuous convex relaxation scheme. Specifically, we propose to cast the problem of 3D shape reconstruction as one of minimizing a spatially continuous convex functional. In qualitative and quantitative evaluation we demonstrate several advantages of the proposed continuous formulation over the discrete graph cut solution. Firstly, geometric properties such as weighted boundary length and surface area are represented in a numerically consistent manner: The continuous convex relaxation assures that the algorithm does not suffer from metrication errors in the sense that the reconstruction converges to the continuous solution as the spatial resolution is increased. Moreover, memory requirements are reduced, allowing for globally optimal reconstructions at higher resolutions. We study three different energy models for multiview reconstruction, which are based on a common variational template unifying regional volumetric terms and on-surface photoconsistency. The three models use data measurements at increasing levels of sophistication. While the first two approaches are based on a classical silhouette-based volume subdivision, the third one relies on stereo information to define regional costs. Furthermore, this scheme is exploited to compute a precise photoconsistency measure as opposed to the classical estimation. All three models are compared on standard data sets demonstrating their advantages and shortcomings. For the third one, which gives the most accurate results, a more exhaustive qualitative and quantitative evaluation is presented.

Efficient Polyhedral Modeling from Silhouettes

Abstract: Modeling from silhouettes is a popular and useful topic in computer vision. Many methods exist to compute the surface of the visual hull from silhouettes, but few address the problem of ensuring good topological properties of the surface, such as manifoldness. This article provides an efficient algorithm to compute such a surface in the form of a polyhedral mesh. It relies on a small number of geometric operations to compute a visual hull polyhedron in a single pass. Such simplicity enables the algorithm to combine the advantages of being fast, producing pixel-exact surfaces, and repeatably yield manifold and watertight polyhedra in general experimental conditions with real data, as verified with all datasets tested. The algorithm is fully described, its complexity analyzed and modeling results given.

Three-dimensional digital magnifier operation supporting system

Abstract: The simulation regarding the state change of the subject in a real space provides a system which represents impacts to three-dimensional computer graphics caused by changes of state of three-dimensional computer graphics composed and fixed to subject, and state of image taking space by simulation, surface polygon model and similar surface polygon model 1 is selected, according to shape pattern, from surface polygon model 2 measures, in a three-dimensional way, subject image existing in the same space, a tracking process is performed on the computer graphics, following to the relative position change of the position changes of the subject and the camera caused in real three-dimensional space, subjects in the visual field of the camera and virtual three-dimensional computer graphics image is unified and displayed by displaying computer graphics image having the same relative position change on the image.

Dense 3D reconstruction method using a single pattern for fast moving object

Abstract: Dense 3D reconstruction of extremely fast moving objects could contribute to various applications such as body structure analysis and accident avoidance and so on. The actual cases for scanning we assume are, for example, acquiring sequential shape at the moment when an object explodes, or observing fast rotating turbine's blades. In this paper, we propose such a technique based on a one-shot scanning method that reconstructs 3D shape from a single image where dense and simple pattern are projected onto an object. To realize dense 3D reconstruction from a single image, there are several issues to be solved; e.g. instability derived from using multiple colors, and difficulty on detecting dense pattern because of influence of object color and texture compression. This paper describes the solutions of the issues by combining two methods, that is (1) an efficient line detection technique based on de Bruijn sequence and belief propagation, and (2) an extension of shape from intersections of lines method. As a result, a scanning system that can capture an object in fast motion has been actually developed by using a high-speed camera. In the experiments, the proposed method successfully captured the sequence of dense shapes of an exploding balloon, and a breaking ceramic dish at 300–1000 fps.

3D reconstruction based on underwater video from ROV Kiel 6000 considering underwater imaging conditions

Abstract: This work presents a system for 3D reconstruction from underwater images or video. Aside from a camera in an underwater housing, no special equipment is required. However, if navigational data is available, it is utilized in the algorithm. The algorithm is in essence a classical structure from motion approach, which is adapted to account for the special imaging conditions. Hence, there is no need for the camera to follow a specialized trajectory. Adaptions to the underwater imaging environment include a special filtering of background and floating particles, which allows a robust estimation of the camera poses and a sparse set of 3D points. Based on the estimated camera track, dense image correspondence computation enables building a detailed 3D surface model. Once the 3D surface model is completed, the colors of the texture are corrected by a physical model for underwater light propagation, allowing to view the model without the effects of scattering and attenuation or to simulate the effects of water on light in a 3D viewer.

Large Area 3-D Reconstructions From Underwater Optical Surveys

Abstract: Robotic underwater vehicles are regularly performing vast optical surveys of the ocean floor. Scientists value these surveys since optical images offer high levels of detail and are easily interpreted by humans. Unfortunately, the coverage of a single image is limited by absorption and backscatter while what is generally desired is an overall view of the survey area. Recent works on underwater mosaics assume planar scenes and are applicable only to situations without much relief. We present a complete and validated system for processing optical images acquired from an underwater robotic vehicle to form a 3D reconstruction of the ocean floor. Our approach is designed for the most general conditions of wide-baseline imagery (low overlap and presence of significant 3D structure) and scales to hundreds or thousands of images. We only assume a calibrated camera system and a vehicle with uncertain and possibly drifting pose information (e.g., a compass, depth sensor, and a Doppler velocity log). Our approach is based on a combination of techniques from computer vision, photogrammetry, and robotics. We use a local to global approach to structure from motion, aided by the navigation sensors on the vehicle to generate 3D sub-maps. These sub-maps are then placed in a common reference frame that is refined by matching overlapping sub-maps. The final stage of processing is a bundle adjustment that provides the 3D structure, camera poses, and uncertainty estimates in a consistent reference frame. We present results with ground truth for structure as well as results from an oceanographic survey over a coral reef.

Continuous depth estimation for multi-view stereo

Abstract: Depth-map merging approaches have become more and more popular in multi-view stereo (MVS) because of their flexibility and superior performance. The quality of depth map used for merging is vital for accurate 3D reconstruction. While traditional depth map estimation has been performed in a discrete manner, we suggest the use of a continuous counterpart. In this paper, we first integrate silhouette information and epipolar constraint into the variational method for continuous depth map estimation. Then, several depth candidates are generated based on a multiple starting scales (MSS) framework. From these candidates, refined depth maps for each view are synthesized according to path-based NCC (normalized cross correlation) metric. Finally, the multiview depth maps are merged to produce 3D models. Our algorithm excels at detail capture and produces one of the most accurate results among the current algorithms for sparse MVS datasets according to the Middlebury benchmark. Additionally, our approach shows its outstanding robustness and accuracy in free-viewpoint video scenario.

Scanning from heating: 3D shape estimation of transparent objects from local surface heating.

Abstract: Today, with quality becoming increasingly important, each product requires three-dimensional in-line quality control. On the other hand, the 3D reconstruction of transparent objects is a very difficult problem in computer vision due to transparency and specularity of the surface. This paper proposes a new method, called Scanning From Heating (SFH), to determine the surface shape of transparent objects using laser surface heating and thermal imaging. Furthermore, the application to transparent glass is discussed and results on different surface shapes are presented.

3D imaging with axially distributed sensing.

Abstract: A new (to our knowledge) multiperspective 3D imaging architecture is proposed that uses imagers distributed along a common optical axis. In this axially distributed sensing method, either a single imager is translated along its optical axis or objects are moved parallel to the optical axis of a single imager. The 3D information collection capability of the proposed architecture is analyzed and a computational 3D reconstruction algorithm based on ray back-projection is proposed. It is shown analytically and experimentally that the collection capacity of this architecture is not uniform over the field of view. Experimental results are presented to verify the proposed approach. We believe this is the first report on 3D sensing and imaging with axially distributed sensing.

Stochastic Reconstruction of Particulate Media Using Simulated Annealing: Improving Pore Connectivity

Abstract: In this contribution the issue of the stochastic reconstruction of particulate media from 2D backscatter SEM images is addressed with particular reference to pore space connectivity. The reconstruction of porous bodies in 2D or 3D space was achieved by using a simulated annealing technique. Two constraints were found to be necessary for the successful reconstruction of well connected pore space: the two-point probability function, and the lineal-path function for the solid phase. Surprisingly, the most commonly used method of reconstruction (common method), consisting of a similar application of both the two-point probability function and the lineal-path function for the void phase, resulted in microstructures characterized by poor pore space connectivity, and by artificial patterns. Since it is desirable to employ the maximum possible number of microstructural descriptors (i.e. to use the lineal-path function for the void phase), we propose a new method of reconstruction. The influence of the lineal-path function for the void phase was suppressed during the initial stages of 2D reconstruction, thereby creating the possibility of obtaining microstructures whose two-point cluster functions match the experimentally measured functions. The effect of the lineal-path function for the void phase on the course of the reconstruction was adjusted by modifying two parameters of the reconstruction procedure. It was tacitly assumed that the parameters adjusted during 2D reconstruction had the same influence on the formation of 3D microstructures. Therefore, the experimental two-point cluster function, extracted from the 2D images, was only used indirectly during 3D reconstruction. 3D replicas obtained using our new method exhibited significantly better pore space connectivity and were more penetrable than porous bodies reconstructed using the common method.

Cone beam micro-CT system for small animal imaging and performance evaluation

Abstract: A prototype cone-beam micro-CT system for small animal imaging has been developed by our group recently, which consists of a microfocus X-ray source, a three-dimensional programmable stage with object holder, and a flat-panel X-ray detector. It has a large field of view (FOV), which can acquire the whole body imaging of a normal-size mouse in a single scan which usually takes about several minutes or tens of minutes. FDK method is adopted for 3D reconstruction with Graphics Processing Unit (GPU) acceleration. In order to reconstruct images with high spatial resolution and low artifacts, raw data preprocessing and geometry calibration are implemented before reconstruction. A method which utilizes a wire phantom to estimate the residual horizontal offset of the detector is proposed, and 1D point spread function is used to assess the performance of geometric calibration quantitatively. System spatial resolution, image uniformity and noise, and low contrast resolution have been studied. Mouse images with and without contrast agent are illuminated in this paper. Experimental results show that the system is suitable for small animal imaging and is adequate to provide high-resolution anatomic information for bioluminescence tomography to build a dual modality system.

3D reconstruction from image collections with a single known focal length

Abstract: In this paper we aim at reconstructing 3D scenes from images with unknown focal lengths downloaded from photosharing websites such as Flickr. First we provide a minimal solution to finding the relative pose between a completely calibrated camera and a camera with an unknown focal length given six point correspondences. We show that this problem has up to nine solutions in general and present two efficient solvers to the problem. They are based on Grobner basis, resp. on generalized eigenvalues, computation. We demonstrate by experiments with synthetic and real data that both solvers are correct, fast, numerically stable and work well even in some situations when the classical 6-point algorithm fails, e.g. when optical axes of the cameras are parallel or intersecting. Based on this solution we present a new efficient method for large-scale structure from motion from unordered data sets downloaded from the Internet. We show that this method can be effectively used to reconstruct 3D scenes from collection of images with very few (in principle single) images with known focal lengths 1.

Capturing 3D stretchable surfaces from single images in closed form

Abstract: We present a closed form solution to the problem of recovering the 3D shape of a nonrigid potentially stretchable surface from 3D-to-2D correspondences. In other words, we can reconstruct a surface from a single image without a priori knowledge of its deformations in that image. State of the art solutions to nonrigid 3D shape recovery rely on the fact that distances between neighboring surface points must be preserved and are therefore limited to inelastic surfaces. Here, we show that replacing the inextensibility constraints by shading ones removes this limitation while still allowing 3D reconstruction in closed-form. We demonstrate our method and compare it to an earlier one using both synthetic and real data.

Confocal Stereo

Abstract: We present confocal stereo, a new method for computing 3D shape by controlling the focus and aperture of a lens. The method is specifically designed for reconstructing scenes with high geometric complexity or fine-scale texture. To achieve this, we introduce the confocal constancy property, which states that as the lens aperture varies, the pixel intensity of a visible in-focus scene point will vary in a scene-independent way, that can be predicted by prior radiometric lens calibration. The only requirement is that incoming radiance within the cone subtended by the largest aperture is nearly constant. First, we develop a detailed lens model that factors out the distortions in high resolution SLR cameras (12MP or more) with large-aperture lenses (e.g., f1.2). This allows us to assemble an A×F aperture-focus image (AFI) for each pixel, that collects the undistorted measurements over all A apertures and F focus settings. In the AFI representation, confocal constancy reduces to color comparisons within regions of the AFI, and leads to focus metrics that can be evaluated separately for each pixel. We propose two such metrics and present initial reconstruction results for complex scenes, as well as for a scene with known ground-truth shape.

A Surface Registration Method for Quantification of Intraoperative Brain Deformations in Image-Guided Neurosurgery

Abstract: Intraoperative brain deformations decrease accuracy in image-guided neurosurgery. Approaches to quantify these deformations based on 3-D reconstruction of cortectomy surfaces have been described and have shown promising results regarding the extrapolation to the whole brain volume using additional prior knowledge or sparse volume modalities. Quantification of brain deformations from surface measurement requires the registration of surfaces at different times along the surgical procedure, with different challenges according to the patient and surgical step. In this paper, we propose a new flexible surface registration approach for any textured point cloud computed by stereoscopic or laser range approach. This method includes three terms: the first term is related to image intensities, the second to Euclidean distance, and the third to anatomical landmarks automatically extracted and continuously tracked in the 2-D video flow. Performance evaluation was performed on both phantom and clinical cases. The global method, including textured point cloud reconstruction, had accuracy within 2 mm, which is the usual rigid registration error of neuronavigation systems before deformations. Its main advantage is to consider all the available data, including the microscope video flow with higher temporal resolution than previously published methods.

3D reconstruction of a patient-specific surface model of the proximal femur from calibrated x-ray radiographs: a validation study.

Abstract: Twenty-three femurs (one plastic bone and twenty-two cadaver bones) with both nonpathologic and pathologic cases were considered to validate a statistical shape model based technique for three-dimensional (3D) reconstruction of a patient-specific surface model from calibrated x-ray radiographs. The 3D reconstruction technique is based on an iterative nonrigid registration of the features extracted from a statistically instantiated 3D surface model to those interactively identified from the radiographs. The surface models reconstructed from the radiographs were compared to the associated ground truths derived either from a 3D CT-scan reconstruction method or from a 3D laser-scan reconstruction method and an average error distance of 0.95 mm were found. Compared to the existing works, our approach has the advantage of seamlessly handling both nonpathologic and pathologic cases even when the statistical shape model that we used was constructed from surface models of nonpathologic bones.

Accurate 3D ground plane estimation from a single image

Abstract: Accurate localization of landmarks in the vicinity of a robot is a first step towards solving the SLAM problem. In this work, we propose algorithms to accurately estimate the 3D location of the landmarks from the robot only from a single image taken from its on board camera. Our approach differs from previous efforts in this domain in that it first reconstructs accurately the 3D environment from a single image, then it defines a coordinate system over the environment, and later it performs the desired localization with respect to this coordinate system using the environment's features. The ground plane from the given image is accurately estimated and this precedes segmentation of the image into ground and vertical regions. A Markov Random Field (MRF) based 3D reconstruction is performed to build an approximate depth map of the given image. This map is robust against texture variations due to shadows, terrain differences, etc. A texture segmentation algorithm is also applied to determine the ground plane accurately. Once the ground plane is estimated, we use the respective camera's intrinsic and extrinsic calibration information to calculate accurate 3D information about the features in the scene.

Structured Light Stereoscopic Imaging with Dynamic Pseudo-random Patterns

Abstract: Structured light stereoscopic imaging offers an efficient and affordable solution to 3D modeling of objects. The majority of structured light patterns that have been proposed either provide a limited resolution or are sensitive to the inherent texture on the surface of the object. This paper proposes an innovative imaging strategy that accomplishes 3D reconstruction of objects using a combination of spatial-neighboring and time-multiplexing structured light patterns encoded with uniquely defined pseudo-random color codes. The approach is extended with the concept of dynamic patterns that adaptively increases the reconstruction resolution. Original techniques are introduced to recover and validate pseudo-random codes from stereoscopic images, and to consistently map color and texture over the reconstructed surface map. Experimental results demonstrate the potential of the solution to create reconstructions with various densities of points and prove the robustness of the approach on objects with different surface properties.

Physical sketching: Reconstruction and analysis of 3D objects from freehand sketches

Abstract: When designing a 3D object, designers, engineers and teachers often begin investigating potential design tradeoffs by creating informal sketches. Ideally, these sketches-in combination with a variety of engineering analysis tools-would allow prediction of the object's physical properties, especially those that affect the critical early design process. We introduce a pen-based system that reconstructs 3D spatial geometry from a single 2D freehand-sketch consisting of straight and curved lines in interactive time. Several optimization-based approaches to this problem have been proposed, but these generally have difficulty converging to an acceptable solution because the dimensionality of the search space is large. The primary contribution of this paper is a new reconstruction algorithm for orthographic projections of 3D wireframes. The algorithm reconstructs the depths of each vertex by exploiting geometric regularities among the graph lines in a reduced solution space, then optimizes a cost function over this space to recover the vertex depths. A second optimization algorithm is used to infer the 3D geometry of curved strokes once the vertex depths have been recovered. The proposed approach can recover the geometry of several objects with approximately 50 curved strokes in near interactive time. We also present an iterative, Tablet-PC-based design system that uses the proposed reconstruction algorithm to recover 3D objects from 2D orthographic sketches. The system allows the reconstructed objects to be subjected to two types of physical analysis, the results of which are superimposed directly on the sketch: a fast, kinematic simulation, and a complete finite-element-based static analysis. The object can quickly be modified in place using the pen-based interface according to the results of the analysis to allow for iterative design work. We demonstrate the system in action on a variety of early-stage design analyses.

Virtual video camera: image-based viewpoint navigation through space and time

Abstract: We present an image-based rendering system to viewpoint-navigate through space and time of complex real-world, dynamic scenes. Our approach accepts unsynchronized, uncalibrated multi-video footage as input. Inexpensive, consumer-grade camcorders suffice to acquire arbitrary scenes, e.g., in the outdoors, without elaborate recording setup procedures. Instead of scene depth estimation, layer segmentation, or 3D reconstruction, our approach is based on dense image correspondences, treating view interpolation uniformly in space and time: spatial viewpoint navigation, slow motion, and freeze-and-rotate effects can all be created in the same fashion. Acquisition simplification, generalization to difficult scenes, and space-time symmetric interpolation amount to a widely applicable Virtual Video Camera system.

Image-based Modeling

Abstract: This book guides you in the journey of 3D modeling from the theory with elegant mathematics to applications with beautiful 3D model pictures. Written in a simple, straightforward, and concise manner, readers will learn the state of the art of 3D reconstruction and modeling. Professor Takeo Kanade, Carnegie Mellon University About this book: The computer vision and graphics communities use different terminologies for the same ideas. This books provides a translation, enabling graphics researchers to apply vision concepts, and vice-versa Independence of chapters allows readers to directly jump into a specific chapter of interest Compared to other texts, gives more succinct treatment overall, and focuses primarily on vision geometry Image-Based Modeling is for graduate students, researchers, and engineers working in the areas of computer vision, computer graphics, image processing, robotics, virtual reality, and photogrammetry. About the Author Long Quan is a Professor of the Department of Computer Science and Engineering at the Hong Kong University of Science and Technology. He received a Ph.D. degree in Computer Science from INRIA, and has been a CNRS researcher at INRIA. Professor Quan is a Fellow of the IEEE Computer Society.

3D Object Reconstruction from Uncalibrated Images using an Off-the-Shelf Camera

Abstract: Three-dimensional (3D) objects reconstruction using just bi-dimensional (2D) images has been a major research topic in Computer Vision. However, it is still a hard problem to address, when automation, speed and precision are required and/or the objects have complex shapes or image properties. In this paper, we compare two Active Computer Vision methods frequently used for the 3D reconstruction of objects from image sequences, acquired with a single off-the-shelf CCD camera: Structure From Motion (SFM) and Generalized Voxel Coloring (GVC). SFM recovers the 3D shape of an object based on the relative motion involved, while VC is a volumetric method that uses photo-consistency measures to build the required 3D model. Both methods considered do not impose any kind of restrictions on the relative motion involved.