Showing papers on "3D reconstruction published in 2005"

A geometric approach to shape from defocus

Abstract: We introduce a novel approach to shape from defocus, i.e., the problem of inferring the three-dimensional (3D) geometry of a scene from a collection of defocused images. Typically, in shape from defocus, the task of extracting geometry also requires deblurring the given images. A common approach to bypass this task relies on approximating the scene locally by a plane parallel to the image (the so-called equifocal assumption). We show that this approximation is indeed not necessary, as one can estimate 3D geometry while avoiding deblurring without strong assumptions on the scene. Solving the problem of shape from defocus requires modeling how light interacts with the optics before reaching the imaging surface. This interaction is described by the so-called point spread function (PSF). When the form of the PSF is known, we propose an optimal method to infer 3D geometry from defocused images that involves computing orthogonal operators which are regularized via functional singular value decomposition. When the form of the PSF is unknown, we propose a simple and efficient method that first learns a set of projection operators from blurred images and then uses these operators to estimate the 3D geometry of the scene from novel blurred images. Our experiments on both real and synthetic images show that the performance of the algorithm is relatively insensitive to the form of the PSF Our general approach is to minimize the Euclidean norm of the difference between the estimated images and the observed images. The method is geometric in that we reduce the minimization to performing projections onto linear subspaces, by using inner product structures on both infinite and finite-dimensional Hilbert spaces. Both proposed algorithms involve only simple matrix-vector multiplications which can be implemented in real-time.

EKF-based 3D SLAM for Structured Environment Reconstruction

Abstract: This paper presents the extension and experimental validation of the widely used EKF1-based SLAM2 algorithm to 3D space. It uses planar features extracted probabilistically from dense three-dimensional point clouds generated by a rotating 2D laser scanner. These features are represented in compliance with the Symmetries and Perturbation model (SPmodel) in a stochastic map. As the robot moves, this map is updated incrementally while its pose is tracked by using an Extended Kalman Filter. After showing how three-dimensional data can be generated, the probabilistic feature extraction method is described, capable of robustly extracting (infinite) planes from structured environments. The SLAM algorithm is then used to track a robot moving through an indoor environment and its capabilities in terms of 3D reconstruction are analyzed.

An algebraic approach to surface reconstruction from gradient fields

Abstract: Several important problems in computer vision such as shape from shading (SFS) and photometric stereo (PS) require reconstructing a surface from an estimated gradient field, which is usually non-integrable, i.e. have non-zero curl. We propose a purely algebraic approach to enforce integrability in discrete domain. We first show that enforcing integrability can be formulated as solving a single linear system Ax =b over the image. In general, this system is under-determined. We show conditions under which the system can be solved and a method to get to those conditions based on graph theory. The proposed approach is non-iterative, has the important property of local error confinement and can be applied to several problems. Results on SFS and PS demonstrate the applicability of our method.

Shape-From-Silhouette Across Time Part I: Theory and Algorithms

Abstract: Shape-From-Silhouette (SFS) is a shape reconstruction method which constructs a 3D shape estimate of an object using silhouette images of the object. The output of a SFS algorithm is known as the Visual Hull (VH). Traditionally SFS is either performed on static objects, or separately at each time instant in the case of videos of moving objects. In this paper we develop a theory of performing SFS across time: estimating the shape of a dynamic object (with unknown motion) by combining all of the silhouette images of the object over time. We first introduce a one dimensional element called a Bounding Edge to represent the Visual Hull. We then show that aligning two Visual Hulls using just their silhouettes is in general ambiguous and derive the geometric constraints (in terms of Bounding Edges) that govern the alignment. To break the alignment ambiguity, we combine stereo information with silhouette information and derive a Temporal SFS algorithm which consists of two steps: (1) estimate the motion of the objects over time (Visual Hull Alignment) and (2) combine the silhouette information using the estimated motion (Visual Hull Refinement). The algorithm is first developed for rigid objects and then extended to articulated objects. In the Part II of this paper we apply our temporal SFS algorithm to two human-related applications: (1) the acquisition of detailed human kinematic models and (2) marker-less motion tracking.

Metric 3D reconstruction and texture acquisition of surfaces of revolution from a single uncalibrated view

Abstract: Image analysis and computer vision can be effectively employed to recover the three-dimensional structure of imaged objects, together with their surface properties. In this paper, we address the problem of metric reconstruction and texture acquisition from a single uncalibrated view of a surface of revolution (SOR). Geometric constraints induced in the image by the symmetry properties of the SOR structure are exploited to perform self-calibration of a natural camera, 3D metric reconstruction, and texture acquisition. By exploiting the analogy with the geometry of single axis motion, we demonstrate that the imaged apparent contour and the visible segments of two imaged cross sections in a single SOR view provide enough information for these tasks. Original contributions of the paper are: single view self-calibration and reconstruction based on planar rectification, previously developed for planar surfaces, has been extended to deal also with the SOR class of curved surfaces; self-calibration is obtained by estimating both camera focal length (one parameter) and principal point (two parameters) from three independent linear constraints for the SOR fixed entities; the invariant-based description of the SOR scaling function has been extended from affine to perspective projection. The solution proposed exploits both the geometric and topological properties of the transformation that relates the apparent contour to the SOR scaling function. Therefore, with this method, a metric localization of the SOR occluded parts can be made, so as to cope with them correctly. For the reconstruction of textured SORs, texture acquisition is performed without requiring the estimation of external camera calibration parameters, but only using internal camera parameters obtained from self-calibration.

Three-dimensional cranio-facial reconstruction in forensic identification: latest progress and new tendencies in the 21st century.

Abstract: Three-dimensional (3D) cranio-facial reconstruction can be useful in the identification of an unknown body. The progress in computer science and the improvement of medical imaging technologies during recent years had significant repercussions on this domain. New facial soft tissue depth data for children and adults have been obtained using ultrasound, CT-scans and radiographies. New guidelines for facial feature properties such as nose projection, eye protrusion or mouth width, have been suggested, but also older theories and "rules of thumbs" have been critically evaluated based on digital technology. New fast, flexible and objective 3D reconstruction computer programs are in full development. The research on craniofacial reconstruction since the beginning of the 21st century is presented, highlighting computer-aided 3D facial reconstruction. Employing the newer technologies and permanently evaluating and (re)questioning the obtained results will hopefully lead to more accurate reconstructions.

Fully 3D Monte Carlo reconstruction in SPECT: a feasibility study.

Abstract: In single photon emission computed tomography (SPECT) with parallel hole collimation, image reconstruction is usually performed as a set of bidimensional (2D) analytical or iterative reconstructions. This approach ignores the tridimensional (3D) nature of scatter and detector response function that affects the detected signal. To deal with the 3D nature of the image formation process, iterative reconstruction can be used by considering a 3D projector modelling the 3D spread of photons. In this paper, we investigate the value of using accurate Monte Carlo simulations to determine the 3D projector used in a fully 3D Monte Carlo (F3DMC) reconstruction approach. Given the 3D projector modelling all physical effects affecting the imaging process, the reconstruction problem is solved using the maximum likelihood expectation maximization (MLEM) algorithm. To validate the concept, three data sets were simulated and F3DMC was compared with two other 3D reconstruction strategies using analytical corrections for attenuation, scatter and camera point spread function. Results suggest that F3DMC improves spatial resolution, relative and absolute quantitation and signal-to-noise ratio. The practical feasibility of the approach on real data sets is discussed.

3D face recognition using passive stereo vision

Abstract: This paper proposes a face recognition system that uses (i) passive stereo vision to capture three-dimensional (3D) facial information and (ii) 3D matching using a simple ICP (iterative closest point) algorithm. So far, the reported 3D face recognition techniques assume the use of active 3D measurement for 3D facial capture. However, active methods employ structured illumination (structure projection, phase shift, gray-code demodulation, etc.) or laser scanning, which is not desirable in many human recognition applications. A major problem of using passive stereo vision for 3D measurement is its low accuracy, and thus no passive methods for 3D face recognition have been reported previously. Addressing this problem, we have newly developed a high-accuracy 3D measurement system based on passive stereo vision, where phase-based image matching is employed for sub-pixel disparity estimation. This paper presents the first attempt to create a practical face recognition system based on fully passive 3D reconstruction.

Localization in urban environments: monocular vision compared to a differential GPS sensor

Abstract: In this paper we present a method for computing the localization of a mobile robot with reference to a learning video sequence. The robot is first guided on a path by a human, while the camera records a monocular learning sequence. Then a 3D reconstruction of the path and the environment is computed off line from the learning sequence. The 3D reconstruction is then used for computing the pose of the robot in real time (30 Hz) in autonomous navigation. Results from our localization method are compared to the ground truth measured with a differential GPS.

Computer-Graphic Facial Reconstruction

Abstract: Introduction To Facial Reconstruction, Classical Non Computer-Assisted Craniofacial Reconstruction, The Wisdom Of Bones: Facial Approximation On The Skull, Three-Dimensional Quantification Of Facial Shape, Automatic 3D Facial Reconstruction By Feature-Based Registration Of A Reference Head, Two-Dimensional Computer Generated Average Human Face Morphology And Facial Approximation, Predicting The Most Probable Facial Features Using Bayesian Networks, Mathematical Morphology And Computer Graphics, Face Reconstructions Using Flesh Deformation Modes, Digital 3D Reconstruction Of Skulls From Fragments Using SLT And CAD/CAM Tools, Forensic Facial Reconstruction Using Computer Modeling Software, Ceiling Recognition Limits Of Two-Dimensional Facial Approximations Constructed Using Averages, Utilization Of 3D Cephalometric Finite Elements Modeling For Measuring Human Facial Soft Tissue Thickness, Computer Aided Dental Identification: Developing Objective Criteria For Comparisons Of Oro-Facial Skeletal, Characteristics To Prove Human Identity, Two Methodologies Of Memory Research: Explanation-Testing And Reconstruction, Using Laser Scans To Study Face Perception, Investigation Of Ethnic Differences In Facial Morphology By Three-Dimensional Averaging, Estimation And Animation Of Faces Using Facial Motion Mapping And A 3D Face Database, Facial Image Identification System Based On 3D Physiognomic Data, A New Retrieval System Using A 3D Facial Image Database

Photometric stereo under perspective projection

Abstract: Photometric stereo is a fundamental approach in computer vision. At its core lies a set of image irradiance equations each taken with a different illumination. The vast majority of studies in this field have assumed orthography as the projection model. This paper re-examines the basic set of equations of photometric stereo, under an assumption of perspective projection. We show that the resulting system is linear (as is the case under the orthographic model; Nevertheless, the unknowns are different in the perspective case). We then suggest a simple reconstruction algorithm based on the perspective formulae, and compare it to its orthographic counterpart on synthetic as well as real images. This algorithm obtained lower error rates than the orthographic one in all of the error measures. These findings strengthen the hypothesis that a more realistic set of assumptions, the perspective one, improves reconstruction significantly.

Color segmentation-based stereo 3D reconstruction system and process

Abstract: A system and process for computing a 3D reconstruction of a scene from multiple images thereof, which is based on a color segmentation-based approach, is presented. First, each image is independently segmented. Second, an initial disparity space distribution (DSD) is computed for each segment, using the assumption that all pixels within a segment have the same disparity. Next, each segment's DSD is refined using neighboring segments and its projection into other images. The assumption that each segment has a single disparity is then relaxed during a disparity smoothing stage. The result is a disparity map for each image, which in turn can be used to compute a per pixel depth map if the reconstruction application calls for it.

Iterative reconstruction method for light emitting sources based on the diffusion equation

Abstract: Bioluminescent imaging (BLI) of luciferase-expressing cells in live small animals is a powerful technique for investigating tumor growth, metastasis, and specific biological molecular events Three-dimensional imaging would greatly enhance applications in biomedicine since light emitting cell populations could be unambiguously associated with specific organs or tissues Any imaging approach must account for the main optical properties of biological tissue because light emission from a distribution of sources at depth is strongly attenuated due to optical absorption and scattering in tissue Our image reconstruction method for interior sources is based on the deblurring expectation maximization method and takes into account both of these effects To determine the boundary of the object we use the standard iterative algorithm—maximum likelihood reconstruction method with an external source of diffuse light Depth-dependent corrections were included in the reconstruction procedure to obtain a quantitative measure of light intensity by using the diffusion equation for light transport in semi-infinite turbid media with extrapolated boundary conditions

3D reconstruction techniques made easy: know-how and pictures

Abstract: Three-dimensional reconstructions represent a visual-based tool for illustrating the basis of three-dimensional post-processing such as interpolation, ray-casting, segmentation, percentage classification, gradient calculation, shading and illumination. The knowledge of the optimal scanning and reconstruction parameters facilitates the use of three-dimensional reconstruction techniques in clinical practise. The aim of this article is to explain the principles of multidimensional image processing in a pictorial way and the advantages and limitations of the different possibilities of 3D visualisation.

A visual odometer without 3D reconstruction for aerial vehicles. Applications to building inspection

Abstract: This paper presents a vision-based method to estimate the * real motion of a single camera from views of a planar patch. Projective techniques allow to estimate camera motion from pixel space apparent motion without explicit 3-D reconstruction. In addition, the paper will present the HELINSPEC project, the framework where the proposed method has been tested, and will detail some applications in external building inspection that make use of the proposed techniques.

3D reconstruction from stereo images for interactions between real and virtual objects

Abstract: The 3D reconstruction algorithm in a stereo image pair for realizing mutual occlusion and interactions between the real and virtual world in an image synthesis is proposed. A two-stage algorithm, consisting of disparity estimation and regularization is used to locate a smooth and precise disparity vector. The hierarchical disparity estimation technique increases the efficiency and reliability of the estimation process, and edge-preserving disparity field regularization produces smooth disparity fields while preserving discontinuities that result from object boundaries. Depth information concerning the real scene is then recovered from the estimated disparity fields by stereo camera geometry. Simulation results show that the proposed algorithm provides accurate and spatially correlated disparity vector fields in various types of images, and the reconstructed 3D model produces a natural space in which the real world and virtual objects interact with each other as if they were in the same world.

3-D reconstruction of a dynamic environment with a fully calibrated background for traffic scenes

Abstract: Vision-based traffic surveillance systems are more and more employed for traffic monitoring, collection of statistical data and traffic control. We present an extension of such a system that additionally uses the captured image content for 3-D scene modeling and reconstruction. A basic goal of surveillance systems is to get a good coverage of the observed area with as few cameras as possible to keep the costs low. Therefore, the 3-D reconstruction has to be done from only a few original views with limited overlap and different lighting conditions. To cope with these specific restrictions we developed a model-based 3-D reconstruction scheme that exploits a priori knowledge about the scene. The system is fully calibrated offline by estimating camera parameters from measured 3-D-2-D correspondences. Then the scene is divided into static parts, which are modeled offline and dynamic parts, which are processed online. Therefore, we segment all views into moving objects and static background. The background is modeled as multitexture planes using the original camera textures. Moving objects are segmented and tracked in each view. All segmented views of a moving object are combined to a 3-D object, which is positioned and tracked in 3-D. Here we use predefined geometric primitives and map the original textures onto them. Finally the static and dynamic elements are combined to create the reconstructed 3-D scene, where the user can freely navigate, i.e., choose an arbitrary viewpoint and direction. Additionally, the system allows analyzing the 3-D properties of the scene and the moving objects.

Surface reconstruction via contour metamorphosis: an Eulerian approach with Lagrangian particle tracking

Abstract: We present a robust method for 3D reconstruction of closed surfaces from sparsely sampled parallel contours. A solution to this problem is especially important for medical segmentation, where manual contouring of 2D imaging scans is still extensively used. Our proposed method is based on a morphing process applied to neighboring contours that sweeps out a 3D surface. Our method is guaranteed to produce closed surfaces that exactly pass through the input contours, regardless of the topology of the reconstruction. Our general approach consecutively morphs between sets of input contours using an Eulerian formulation (i.e. fixed grid) augmented with Lagrangian particles (i.e. interface tracking). This is numerically accomplished by propagating the input contours as 2D level sets with carefully constructed continuous speed functions. Specifically this involves particle advection to estimate distances between the contours, monotonicity constrained spline interpolation to compute continuous speed functions without overshooting, and state-of-the-art numerical techniques for solving the level set equations. We demonstrate the robustness of our method on a variety of medical, topographic and synthetic data sets.

Progressive surface reconstruction from images using a local prior

Abstract: This paper introduces a new method for surface reconstruction from multiple calibrated images. The primary contribution of this work is the notion of local prior to combine the flexibility of the carving approach with the accuracy of graph-cut optimization. A progressive refinement scheme is used to recover the topology and reason the visibility of the object. Within each voxel, a detailed surface patch is optimally reconstructed using a graph-cut method. The advantage of this technique is its ability to handle complex shape similarly to level sets while enjoying a higher precision. Compared to carving techniques, the addressed problem is well-posed, and the produced surface does not suffer from aliasing. In addition, our approach seamlessly handles complete and partial reconstructions: If the scene is only partially visible, the process naturally produces an open surface; otherwise, if the scene is fully visible, it creates a complete shape. These properties are demonstrated on real image sequences

3D freehand ultrasound reconstruction based on probe trajectory

Abstract: 3D freehand ultrasound imaging is a very attractive technique in medical examinations and intra-operative stage for its cost and field of view capacities. This technique produces a set of non parallel B-scans which are irregularly distributed in the space. Reconstruction amounts to computing a regular lattice volume and is needed to apply conventional computer vision algorithms like registration. In this paper, a new 3D reconstruction method is presented, taking explicitly into account the probe trajectory. Experiments were conducted on different data sets with various probe motion types and indicate that this technique outperforms classical methods, especially on low acquisition frame rate.

3D Reconstruction of dental specimens from 2D histological images and μCT-Scans

Abstract: Direct comparison of experimental and theoretical results in biomechanical studies requires a careful reconstruction of specimen surfaces to achieve a satisfactory congruence for validation. In this paper a semi-automatic approach is described to reconstruct triangular boundary representations from images originating from, either histological sections or μCT-, CT- or MRI-data, respectively. In a user-guided first step, planar 2D contours were extracted for every material of interest, using image segmentation techniques. In a second step, standard 2D triangulation algorithms were used to derive high quality mesh representations of the underlying surfaces. This was accomplished by converting the 2D meshes into 3D meshes by a novel lifting procedure. The meshes can be imported as is into finite element programme packages such as Marc/Mentat or COSMOS/M. Accuracy and feasibility of the algorithm is demonstrated by reconstructing several specimens as examples and comparing simulated results with available meas...

Three-dimensional reconstruction based on images from spiral high-resolution computed tomography of the temporal bone: anatomy and clinical application.

Abstract: The aim of this study was to investigate the usefulness of a three-dimensional (3D) reconstruction of computed tomography (CT) images in determining the anatomy and topographic relationship between various important structures. Using 40 ears from 20 patients with various otological diseases, a 3D reconstruction based on the image data from spiral high-resolution CT was performed by segmentation, volume-rendering and surface-rendering algorithms on a personal computer. The 3D display of the middle and inner ear structures was demonstrated in detail. Computer-assisted measurements, many of which could not be easily measured in vivo, of the reconstructed structures provided accurate anatomic details that improved the surgeon's understanding of spatial relationships. A 3D reconstruction of temporal bone CT might be useful for education and increasing understanding of the anatomical structures of the temporal bone. However, it will be necessary to confirm the correlation between the 3D reconstructed images and histological sections through a validation study.

Uncalibrated multiple image stereo system with arbitrarily movable camera and projector for wide range scanning

Abstract: In this paper, we propose an uncalibrated, multi-image 3D reconstruction, using coded structured light. Normally, a conventional coded structured light system consists of a camera and a projector and needs precalibration before scanning. Since the camera and the projector have to be fixed after calibration, reconstruction of a wide area of the scene or reducing occlusions by multiple scanning are difficult and sometimes impossible. In the proposed method, multiple scanning while moving the camera or the projector is possible by applying the uncalibrated stereo method, thereby achieving a multi-image 3D reconstruction. As compared to the conventional coded structured light method, our system does not require calibration of extrinsic camera parameters, occlusions are reduced, and a wide area of the scene can be acquired. As compared to image-based multi-image reconstruction, the proposed system can obtain dense shape data with higher precision. As a result of these advantages, users can freely move either the cameras or projectors to scan a wide range of objects, but not if both the camera and the projector are moved at the same time.

Discontinuity preserving stereo with small baseline multi-flash illumination

Abstract: Currently, sharp discontinuities in depth and partial occlusions in multiview imaging systems pose serious challenges for many dense correspondence algorithms. However, it is important for 3D reconstruction methods to preserve depth edges as they correspond to important shape features like silhouettes which are critical for understanding the structure of a scene. In this paper, we show how active illumination algorithms can produce a rich set of feature maps that are useful in dense 3D reconstruction. We start by showing a method to compute a qualitative depth map from a single camera, which encodes object relative distances and can be used as a prior for stereo. In a multiview setup, we show that along with depth edges, binocular half-occluded pixels can also be explicitly and reliably labeled. To demonstrate the usefulness of these feature maps, we show how they can be used in two different algorithms for dense stereo correspondence. Our experimental results show that our enhanced stereo algorithms are able to extract high quality, discontinuity preserving correspondence maps from scenes that are extremely challenging for conventional stereo methods.

Method and device for medical image reconstruction

Abstract: The present invention relates to an image reconstruction device ( 12 ) for an X-ray apparatus and a method for local 3D reconstruction of an object area of an examination object ( 7 ) from 2D image data of several 2D X-ray images of the examination object ( 7 ) registered in chronological order with different known projection geometries using the X-ray apparatus. With the method a location in the object area under consideration is selected from one of the 2D X-ray images. The positions of the selected location are determined in at least some of the 2D X-ray images and a spatial motion of the selected location between the registrations of the 2D X-ray images is calculated from the positions obtained, taking the known projection geometries into consideration. The calculated motion is then annulled by modifying the 2D image data in the 2D X-ray images and a 3D image dataset of at least the object area is reconstructed from the modified 2D image data. The method and the image reconstruction device enable a 3D image of a moving locally bounded object area to be reconstructed in a simple way without motion artifacts.