Showing papers on "3D reconstruction published in 2004"

Reconstruction of 3D building models from aerial images and maps

Abstract: Automatic 3D building reconstruction has becoming increasingly important for a number of applications. The reconstruction of buildings using only aerial images as data source has been proven to be a very difficult problem. The complexity of the reconstruction can be greatly reduced by combining the aerial images with other data sources. In this paper, we describe a 3D building reconstruction method that integrates the aerial image analysis with information from large-scale 2D Geographic Information System (GIS) databases and domain knowledge. By combining the images with GIS data, the specific strengths of both the images (high resolution, accuracy, and large-information content) and the GIS data (relatively simple interpretation) are exploited.

Emerging Topics in Computer Vision

Abstract: The state-of-the art in computer vision: theory, applications, and programmingWhether you're a working engineer, developer, researcher, or student, this is your single authoritative source for today's key computer vision innovations. Gerard Medioni and Sing Bing Kang present advances in computer vision such as camera calibration, multi-view geometry, and face detection, and introduce important new topics such as vision for special effects and the tensor voting framework. They begin with the fundamentals, cover select applications in detail, and introduce two popular approaches to computer vision programming. Camera calibration using 3D objects, 2D planes, 1D lines, and self-calibration Extracting camera motion and scene structure from image sequences Robust regression for model fitting using M-estimators, RANSAC, and Hough transforms Image-based lighting for illuminating scenes and objects with real-world light images Content-based image retrieval, covering queries, representation, indexing, search, learning, and more Face detection, alignment, and recognition--with new solutions for key challenges Perceptual interfaces for integrating vision, speech, and haptic modalities Development with the Open Source Computer Vision Library (OpenCV) The new SAI framework and patterns for architecting computer vision applicationsDVD-ROMs INCLUDED (for PCs only)The accompanying DVD-ROMs (for PCs only) contain video lectures by each of this book's contributors, per their respective chapters. Each presentation varies from 30 to 60 minutes and is, accompanied by a slide presentation. The slides are also included on the discs in PDF format for your convenience.

High-Resolution, Real-time 3D Shape Acquisition

Abstract: In this paper we describe a high-resolution, real-time 3D shape acquisition system based on structured light techniques. This system uses a color pattern whose RGB channels are coded with either sinusoidal or trapezoidal fringe patterns. When projected by a modified DLP projector (color filters removed), this color pattern results in three grayscale patterns projected sequentially at a frequency of 240 Hz. A high-speed B/W CCD camera synchronized with the projector captures the three images, from which the 3D shape of the object is reconstructed. A color CCD camera is also used to capture images for texture mapping. The maximum 3D shape acquisition speed is 120 Hz (532 × 500 pixels), which is high enough for capturing the 3D shapes of moving objects. Two coding methods, sinusoidal phase-shifting method and trapezoidal phase-shifting method, were tested and results with good accuracy were obtained. The trapezoidal phase-shifting algorithm also makes real-time 3D reconstruction possible.

Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing

Abstract: In the computational three-dimensional (3D) volumetric reconstruction integral imaging (II) system, volume pixels of the scene are reconstructed by superimposing the inversely mapped elemental images through a computationally simulated optical reconstruction process according to ray optics. Close placement of a 3D object to the lenslet array in the pickup process may result in significant variation in intensity between the adjacent pixels of the reconstructed image, degrading the quality of the image. The intensity differences result from the different number of the superimposed elemental images used for reconstructing the corresponding pixels. In this paper, we propose improvements of the reconstructed image quality in two ways using 1) normalized computational 3D volumetric reconstruction II, and 2) hybrid moving lenslet array technique (MALT). To reduce the intensity irregularities between the pixels, we normalize the intensities of the reconstructed image pixels by the overlapping numbers of the inversely mapped elemental images. To capture the elemental image sets for the MALT process, a stationary 3D object pickup process is performed repeatedly at various locations of the pickup lenslet array's focal plane, which is perpendicular to the optical axis. With MALT, we are able to enhance the quality of the reconstructed images by increasing the sampling rate. We present experimental results of volume pixel reconstruction to test and verify the performance of the proposed reconstruction algorithm. We have shown that substantial improvement in the visual quality of the 3D reconstruction is obtained using the proposed technique.

Large area 3D reconstructions from underwater surveys

Abstract: Robotic underwater vehicles can perform vast optical surveys of the ocean floor. Scientists value these surveys since optical images offer high levels of information and are easily interpreted by humans. Unfortunately the coverage of a single image is limited by absorption and backscatter while what is needed is an overall view of the survey area. Recent work on underwater mosaics assume planar scenes and are applicable only to situations without much relief. We present a complete and validated system for processing optical images acquired from an underwater robotic vehicle to form a 3D reconstruction of the ocean floor. Our approach is designed for the most general conditions of wide-baseline imagery (low overlap and presence of significant 3D structure) and scales to hundreds of images. We only assume a calibrated camera system and a vehicle with uncertain and possibly drifting pose information (e.g. a compass, depth sensor and a Doppler velocity log). Our approach is based on a combination of techniques from computer vision, photogrammetry and robotics. We use a local to global approach to structure from motion, aided by the navigation sensors on the vehicle to generate 3D submaps. These submaps are then placed in a common reference frame that is refined by matching overlapping submaps. The final stage of processing is a bundle adjustment that provides the 3D structure, camera poses and uncertainty estimates in a consistent reference frame. We present results with ground-truth for structure as well as results from an oceanographic survey over a coral reef covering an area of approximately one hundred square meters.

Hand Motion from 3D Point Trajectories and a Smooth Surface Model

Abstract: A method is proposed to track the full hand motion from 3D points reconstructed using a stereoscopic set of cameras. This approach combines the advantages of methods that use 2D motion (e.g. optical flow), and those that use a 3D reconstruction at each time frame to capture the hand motion. Matching either contours or a 3D reconstruction against a 3D hand model is usually very difficult due to self-occlusions and the locally-cylindrical structure of each phalanx in the model, but our use of 3D point trajectories constrains the motion and overcomes these problems.

System and method for processing specimens and images for optical tomography

Abstract: A scanning method for scanning samples of biological cells using optical tomography includes preparing, acquiring, reconstructing and viewing three-dimensional images of cell samples. Concentration and enrichment of the cell sample follows. The cell sample is stained. Cells are isolated from the cell sample and purified. A cell/solvent mixture is injected into a gel by centrifugation. A cell/gel mixture is injected into a capillary tube until a cell appears centered in a field of view using a stopped-flow method. An optical imaging system, such as a fixed or variable motion optical tomography system acquires a projection image. The sample is rotated about a tube axis to generate additional projections. Once image acquisition is completed, the acquired image projections are corrected for errors. A computer or other equivalent processor is used to compute filtered backprojection information for 3D reconstruction.

Efficient reconstruction of large scattered geometric datasets using the partition of unity and radial basis functions

Abstract: We present a new scheme for the reconstruction of large geometric data. It is based on the well-known radial basis function model combined with an adaptive spatial and functional subdivision associated with a family of functions forming a partition of unity. This combination offers robust and efficient solution to a great variety of 2D and 3D reconstruction problems, such as the reconstruction of implicit curves or surfaces with attributes starting from unorganized point sets, image or mesh repairing, shape morphing or shape deformation, etc. After having presented the theoretical background, the paper mainly focuses on implementation details and issues, as well as on applications and experimental results.

Ultra-fast 3D filtered backprojection on commodity graphics hardware

Abstract: Recent efforts in cone-beam scanner technology have focused on developing interactive scanning capabilities, for example, to enable image-guided surgical interventions or real-time diagnosis with time-varying data. However, apart from a fast scanner these applications also require a fast reconstruction algorithm to match. The filtered backprojection algorithm devised by Feldkamp, Davis, and Kress is the most widely used algorithm for 3D reconstruction from cone-beam projections, and it is the algorithm with the lowest complexity. Yet, pure software implementations have difficulties to process the data at the speeds required for real-time scanning. One option is to utilize expensive and rare custom boards for this purpose. We describe an alternative solution, which is inexpensive, uses readily available PC graphics hardware boards, and provides the desired performance at the quality required.

Accuracy of Spherical Harmonic Approximations for Images of Lambertian Objects under Far and Near Lighting

Abstract: Various problems in Computer Vision become difficult due to a strong influence of lighting on the images of an object Recent work showed analytically that the set of all images of a convex, Lambertian object can be accurately approximated by the low-dimensional linear subspace constructed using spherical harmonic functions In this paper we present two major contributions: first, we extend previous analysis of spherical harmonic approximation to the case of arbitrary objects; second, we analyze its applicability for near light We begin by showing that under distant lighting, with uniform distribution of light sources, the average accuracy of spherical harmonic representation can be bound from below This bound holds for objects of arbitrary geometry and color, and for general illuminations (consisting of any number of light sources) We further examine the case when light is coming from above and provide an analytic expression for the accuracy obtained in this case Finally, we show that low-dimensional representations using spherical harmonics provide an accurate approximation also for fairly near light Our analysis assumes Lambertian reflectance and accounts for attached, but not for cast shadows We support this analysis by simulations and real experiments, including an example of a 3D shape reconstruction by photometric stereo under very close, unknown lighting

Calculating the 3d-pose of rigid-objects using active appearance models

Abstract: This paper presents two different algorithms for object tracking and pose estimation. Both methods are based on an appearance model technique called Active Appearance Model (AAM). The key idea of the first method is to utilize two instances of the AAM to track landmark points in a stereo pair of images and perform 3D reconstruction of the landmarks followed by 3 D pose estimation. The second method, the AAM matching algorithm is an extension of the original AAM that incorporates the full 6 DOF pose parameters as part of the minimization parameters. This extension allows for the estimation of the 3D pose of any object, without any restriction on its geometry. We compare both algorithms with a previously developed algorithm using a geometric-based approach [14]. The results show that the accuracy in pose estimation of our new appearance-based methods is better than using the geometric-based approach. Moreover, since appearance-based methods do not require customized feature extractions, the new methods present a more flexible alternative, especially in situations where extracting features is not simple due to cluttered background, complex and irregular features, etc.

Perspective shape-from-shading by fast marching

Abstract: Shape-from-shading (SfS) is a fundamental problem in computer vision. At its basis lies the image irradiance equation. Recently, the authors proposed to base the image irradiance equation on the assumption of perspective projection rather than the common orthographic one. The current paper presents a greatly-improved reconstruction method based on the perspective formulation. The proposed model is solved efficiently via a modification of the fast marching method of Kimmel and Sethian. We compare the two versions of the fast marching method (orthographic vs. perspective) on medical images. The perspective algorithm outperformed the orthographic one. This shows that the more realistic hypothesis of perspective projection improves reconstruction significantly. The comparison also demonstrates the usability of perspective SfS for real-life applications such as medical endoscopy.

Monocular 3D Reconstruction of Human Motion in Long Action Sequences

Abstract: A novel algorithm is presented for the 3D reconstruction of human action in long (>30 second) monocular image sequences. A sequence is represented by a small set of automatically found representative keyframes. The skeletal joint positions are manually located in each keyframe and mapped to all other frames in the sequence. For each keyframe a 3D key pose is created, and interpolation between these 3D body poses, together with the incorporation of limb length and symmetry constraints, provides a smooth initial approximation of the 3D motion. This is then fitted to the image data to generate a realistic 3D reconstruction. The degree of manual input required is controlled by the diversity of the sequence’s content. Sports’ footage is ideally suited to this approach as it frequently contains a limited number of repeated actions. Our method is demonstrated on a long (36 second) sequence of a woman playing tennis filmed with a non-stationary camera. This sequence required manual initialisation on <1.5% of the frames, and demonstrates that the system can deal with very rapid motion, severe self-occlusions, motion blur and clutter occurring over several concurrent frames. The monocular 3D reconstruction is verified by synthesising a view from the perspective of a ‘ground truth’ reference camera, and the result is seen to provide a qualitatively accurate 3D reconstruction of the motion.

3D Metric Reconstruction from Uncalibrated Omnidirectional Images

Abstract: We show that it is possible to obtain a very complete 3D metric reconstruction of the surrounding scene from two or more uncalibrated omnidirectional images. In particular, we demonstrate that omnidirectional images with angle of view above 180 can be reliably autocalibrated. We also show that wide angle images provide reliable information about their camera positions and orientations. We link together a method for simultaneous omnidirectional camera model and epipolar geometry estimation and a method for factorization-based 3D reconstruction in order to obtain metric reconstruction of unknown scene observed by uncalibrated omnidirectional images. The 3D reconstruction is done from automatically established image correspondences only. We demonstrate our method in experiments with Nikon FC–E8 and Sigma 8mm-f4-EX fish-eye lenses. Nevertheless, the proposed method can be used for a large class of non-perspective central omnidirectional cameras.

Multiple View Geometry in Computer Vision: N-View Geometry

Abstract: Outline This part is partly a recapitulation and partly new material. Chapter 17 is the recapitulation. We return to two- and three-view geometry but now within a more general framework which naturally extends to four- and n -views. The fundamental projective relations over multiple views arise from the intersection of lines (back-projected from points) and planes (back-projected from lines). These intersection properties are represented by the vanishing of determinants formed from the camera matrices of the views. The fundamental matrix, the trifocal tensor, and a new tensor for four views – the quadrifocal tensor – arise naturally from these determinants as the multiple view tensors for two, three, and four views respectively. The tensors are what remains when the 3D structure and non-essential part of the camera matrices are eliminated. The tensors stop at four views. These tensors are unique for each set of views, and generate relationships which are multi-linear in the coordinates of the image measurements. The tensors can be computed from sets of image correspondences, and subsequently a camera matrix for each view can be computed from the tensor. Finally, the 3D structure can be computed from the retrieved cameras and image correspondences. Chapter 18 covers the computation of a reconstruction from multiple views. In particular the important factorization algorithm is given for reconstruction from affine views. It is important because the algorithm is optimal, but is also non-iterative.

Three-Dimensional Reconstruction Based on Images from Spiral High-Resolution Computed Tomography of Temporal Bone：Anatomy and Clinical Application

Abstract: Background and Objectives：The intricate anatomy of the temporal bone has always been difficult to visualize. In this regard, he advantages of computer-assisted reconstruction of temporal bone based on image data from computed tomography (CT) are widely recognized. The goal of this study was to investigate the usefulness of three-dimensional (3D) reconstruction of computed tomography in determining the anatomy and topographic relationship of various important structures. Subjects and Method： For 40 ears of 20 patients with various otological diseases, 3D reconstruction based on image data from spiral high-resolution CT was performed by segmentation, volume-rendering and surface-rendering algorithm on a personal computer. The scanning was carried out in axial plane with technical factors of 140 kV, 100 mAs, 1 mm thickness, and 1 second scanning time. A software ( 4.0, CyberMed Inc, Korea) was used for image processing. Results：We were able to demonstrate the 3D display of the middle and inner ear structures. The computer-assisted measurement of reconstructed structures demonstrated the anatomic details comprehensively, which improved the surgeon’s understanding of their spatial relationship, and provided many details that could not be easily measured in vivo. Conclusion：The 3D reconstruction of temporal bone CT can be useful in demonstrating and thus understanding the anatomical structures of temporal bone. Also, its clinical applications are inestimable. But it is necessary to confirm the correlation between 3D reconstructed images and histologic sections through the validation study.

Region-Based Segmentation on Evolving Surfaces with Application to 3D Reconstruction of Shape and Piecewise Constant Radiance

Abstract: We consider the problem of estimating the shape and radiance of a scene from a calibrated set of images under the assumption that the scene is Lambertian and its radiance is piecewise constant. We model the radiance segmentation explicitly using smooth curves on the surface that bound regions of constant radiance. We pose the scene reconstruction problem in a variational framework, where the unknowns are the surface, the radiance values and the segmenting curves. We propose an iterative procedure to minimize a global cost functional that combines geometric priors on both the surface and the curves with a data fitness score. We carry out the numerical implementation in the level set framework.

Uncalibrated Euclidean 3-D reconstruction using an active vision system

Abstract: Uncalibrated reconstruction of a scene is desired in many practical applications of computer vision. However, using a single camera with unconstrained motion and unknown parameters, a true Euclidean three-dimensional (3-D) model of the scene cannot be reconstructed. In this paper, we present a method for true Euclidean 3-D reconstruction using an active vision system consisting of a pattern projector and a camera. When the intrinsic and extrinsic parameters of the camera are changed during the reconstruction, they can be self-calibrated and the real 3-D model of the scene can then be reconstructed. The parameters of the projector are precalibrated and are kept constant during the reconstruction process. This allows the configuration of the vision system to be varied during a reconstruction task, which increases its self-adaptability to the environment or scene structure in which it is to work.

Recovering shape and reflectance model of non-lambertian objects from multiple views

Abstract: This paper proposes an algorithm to simultaneously estimate both the 3D shape and parameters of a surface reflectance model from multiple views of an object made of a single material. The algorithm is based on a multiple view shape from shading method. A triangular mesh represents the shape of the object. The Phong reflectance model is used to model the surface reflectance. We iteratively find the shape and reflectance parameters that best fit all input images. Subdividing triangles in the mesh into smaller ones gradually refines the estimates of shape and reflectance model. The estimation takes into account both self-occlusion and self-shadowing. Analysis shows that the accuracy of reflectance estimation is limited by the triangle size in the shape model. We also propose to use Richardson extrapolation to overcome this and further refine the reflectance model estimate. The estimated 3D shape and reflectance model can be used to render the same object from different viewing directions and under different lighting conditions. Experimental results on both synthetic and real objects are given.

Turntable-based 3D object reconstruction

Abstract: In this paper, we present a system that can acquire graphical models from real objects. Given an image sequence of a complex shape object placed on a turntable, the presented algorithm generates automatically the 3D model. In contrast to previous approaches, the technique described here is only based on conies properties and uses the spatiotemporal aspect of the sequence of images. From the projective properties of the conies and using the camera calibration parameters the Euclidean 3D coordinates of a point are obtained from the geometric locus of the image points trajectories. An algorithm has been implemented to compute the 3D reconstruction automatically. Examples on both synthetic and real image sequences are presented

Visual-hull reconstruction from uncalibrated and unsynchronized video streams

Abstract: We present an approach for automatic reconstruction of a dynamic event using multiple video cameras recording from different viewpoints. Those cameras do not need to be calibrated or even synchronized. Our approach recovers all the necessary information by analyzing the motion of the silhouettes in the multiple video streams. The first step consists of computing the calibration and synchronization for pairs of cameras. We compute the temporal offset and epipolar geometry using an efficient RANSAC-based algorithm to search for the epipoles as well as for robustness. In the next stage the calibration and synchronization for the complete camera network is recovered and then refined through maximum likelihood estimation. Finally, a visual-hull algorithm is used to the recover the dynamic shape of the observed object. For unsynchronized video streams silhouettes are interpolated to deal with subframe temporal offsets. We demonstrate the validity of our approach by obtaining the calibration, synchronization and 3D reconstruction of a moving person from a set of 4 minute videos recorded from 4 widely separated video cameras.

On 3D mosaicing of rotationally symmetric ceramic fragments

Abstract: A major obstacle to the wider use of 3D object reconstruction and modeling is the extent of manual intervention needed. Such interventions are currently massive and exist throughout every phase of a 3D reconstruction project: collection of images, image management, establishment of sensor position and image orientation, extracting the geometric detail describing an object, merging geometric, texture and semantic data. This work aims to develop a solution for automated documentation of archaeological pottery, which also leads to a more complete 3D model out of multiple fragments. Generally the 3D reconstruction of arbitrary objects from their fragments can be regarded as a 3D puzzle. In order to solve it we identified the following main tasks: 3D data acquisition, orientation of the object, classification of the object and reconstruction. We demonstrate the method and give results on synthetic and real data.

Combining optical holograms with interactive computer graphics

Abstract: Holograms can reconstruct complete optical wavefronts, capturing images that have a three-dimensional appearance and can be observed from different perspectives. Museum exhibits often use optical hologram technology because it permits presentation of 3D objects with almost no loss in visual quality. Optical holograms are static, however, and lack interactivity. Combining 3D computer graphical elements with stereoscopic presentation techniques provides an alternative that allows interactivity.

3D model estimation from multiple images

Abstract: 3D reconstruction plays a very important role in computer vision. The determination of the 3D model from multiple images is of key importance. A 3D reconstruction algorithm is introduced, which is capable to determine the 3D model without any external intervention.

Realistic modeling and animation of human body based on scanned data

Abstract: In this paper we propose a novel method for building animation model of real human body from surface scanned data. The human model is represented by a triangular mesh and described as a layered geometric model. The model consists of two layers: the control skeleton generating body animation from motion capture data, and the simplified surface model providing an efficient representation of the skin surface shape. The skeleton is generated automatically from surface scanned data using the feature extraction, and then a point-to-line mapping is used to map the surface model onto the underlying skeleton. The resulting model enables real-time and smooth animation by manipulation of the skeleton while maintaining the surface detail. Compared with earlier approach, the principal advantages of our approach are the automated generation of body control skeletons from the scanned data for real-time animation, and the automatic mapping and animation of the captured human surface shape. The human model constructed in this work can be used for applications of ergonomic design, garment CAD, real-time simulating humans in virtual reality environment and so on.

Reconstruction of medical images by perspective shape-from-shading

Abstract: Shape-from-shading (SfS) is a fundamental problem in computer vision; it is based upon the image irradiance equation. Recently, the authors proposed to solve the image irradiance equation under the assumption of perspective projection rather than the common orthographic one. The solution was a modification of the fast marching method of Kimmel and Sethian. This paper presents an application of this novel perspective algorithm to reconstruction of medical images. We focus on gastrointestinal endoscopy and compare the two versions of the fast marching method (orthographic vs. perspective). The examples and comparison show that, unlike orthographic SfS, perspective SfS is robust and can be utilized for real-life applications.