View synthesis of scenes with multiple independently translating objects from uncalibrated views
13 Jan 2006-pp 460-469
TL;DR: A voxel-based volumetric scene reconstruction scheme is used to obtain a scene model and synthesize views of the entire scene using an affine coordinate system and experimental results are presented to validate the technique.
Abstract: We propose a technique for view synthesis of scenes with static objects as well as objects that translate independent of the camera motion. Assuming the availability of three vanishing points in general position in the given views, we set up an affine coordinate system in which the static and moving points are reconstructed and the translations of the dynamic objects are recovered. We then describe how to synthesize new views corresponding to a completely new camera specified in the affine space with new translations for the dynamic objects. As the extent of the synthesized scene is restricted by the availability of corresponding points, we use a voxel-based volumetric scene reconstruction scheme to obtain a scene model and synthesize views of the entire scene. We present experimental results to validate our technique.
References
More filters
TL;DR: An algebraic geometric approach to 3-D motion estimation and segmentation of multiple rigid-body motions from noise-free point correspondences in two perspective views that exploits the algebraic and geometric properties of the so-called multibody epipolar constraint and its associatedMultibody fundamental matrix.
Abstract: We present an algebraic geometric approach to 3-D motion estimation and segmentation of multiple rigid-body motions from noise-free point correspondences in two perspective views. Our approach exploits the algebraic and geometric properties of the so-called multibody epipolar constraint and its associated multibody fundamental matrix, which are natural generalizations of the epipolar constraint and of the fundamental matrix to multiple motions. We derive a rank constraint on a polynomial embedding of the correspondences, from which one can estimate the number of independent motions as well as linearly solve for the multibody fundamental matrix. We then show how to compute the epipolar lines from the first-order derivatives of the multibody epipolar constraint and the epipoles by solving a plane clustering problem using Generalized PCA (GPCA). Given the epipoles and epipolar lines, the estimation of individual fundamental matrices becomes a linear problem. The clustering of the feature points is then automatically obtained from either the epipoles and epipolar lines or from the individual fundamental matrices. Although our approach is mostly designed for noise-free correspondences, we also test its performance on synthetic and real data with moderate levels of noise.
170 citations
"View synthesis of scenes with multi..." refers background in this paper
...Since all objects are now static, a single fundamental matrix suffices, contrary to the treatment in [2]....
[...]
TL;DR: A visibility approach that uses all possible color information from the photographs during reconstruction, photo-consistency measures that are more robust and/or require less manual intervention, and a volumetric warping method for application of these reconstruction methods to large-scale scenes are described.
Abstract: In this paper, we present methods for 3D volumetric reconstruction of visual scenes photographed by multiple calibrated cameras placed at arbitrary viewpoints. Our goal is to generate a 3D model that can be rendered to synthesize new photo-realistic views of the scene. We improve upon existing voxel coloring/space carving approaches by introducing new ways to compute visibility and photo-consistency, as well as model infinitely large scenes. In particular, we describe a visibility approach that uses all possible color information from the photographs during reconstruction, photo-consistency measures that are more robust and/or require less manual intervention, and a volumetric warping method for application of these reconstruction methods to large-scale scenes.
130 citations
"View synthesis of scenes with multi..." refers background or methods in this paper
...Our approach is based upon the technique proposed in [1]....
[...]
...We have adapted the technique proposed in [1] to work with our affine reconstruction scheme as follows....
[...]
...While [1] proposes a number of photoconsistency tests, most of them are based on statistical quantities like standard deviation and colour histograms....
[...]
26 Jun 2000
TL;DR: This paper extends the recovery of structure and motion to image sequences with several independently moving objects, where Euclidean reconstruction becomes possible in the multibody case, when it was underconstrained for a static scene.
Abstract: This paper extends the recovery of structure and motion to image sequences with several independently moving objects. The motion, structure, and camera calibration are all a-priori unknown. The fundamental constraint that we introduce is that multiple motions must share the same camera parameters. Existing work on independent motions has not employed this constraint, and therefore has not gained over independent static-scene reconstructions.
We show how this constraint leads to several new results in structure and motion recovery, where Euclidean reconstruction becomes possible in the multibody case, when it was underconstrained for a static scene. We show how to combine motions of high-relief, low-relief and planar objects. Additionally we show that structure and motion can be recovered from just 4 points in the uncalibrated, fixed camera, case.
Experiments on real and synthetic imagery demonstrate the validity of the theory and the improvement in accuracy obtained using multibody analysis.
116 citations
"View synthesis of scenes with multi..." refers background in this paper
...In [6], an analysis of the constraints imposed by moving objects on the calibration parameters of the camera is presented....
[...]
26 Jun 2000
TL;DR: A 3 ×3 × 3 × 3 tensor Hijk and its dual Hijk which represent the 2D projective mapping of points across three projections (views) and it concatenates two homography matrices to represent the joint mapping across three views.
Abstract: We introduce a 3 × 3 × 3 tensor Hijk and its dual Hijk which represent the 2D projective mapping of points across three projections (views). The tensor Hijk is a generalization of the well known 2D collineation matrix (homography matrix) and it concatenates two homography matrices to represent the joint mapping across three views. The dual tensor Hijk concatenates two dual homography matrices (mappings of line space) and is responsible for representing the mapping associated with moving points along straight-line paths, i.e., Hijk can be recovered from line-of-sight measurements only.
82 citations
"View synthesis of scenes with multi..." refers background in this paper
...In [3] and [4] Homography Tensors are used for view synthesis of dynamic points....
[...]
13 Jun 2000
TL;DR: A new image sequence is created showing the scene from arbitrary viewing position and arbitrary time, made use of a newly discovered tool, the "dual Htensor" that connects together three views of a coplanar configuration of (unlabeled) static and moving points.
Abstract: We consider a scene, containing many objects moving with constant velocity along straight line paths, seen from three reference viewpoints at three different times. The scene may even consist only of moving objects with no static features. We wish to create a new image sequence showing the scene from arbitrary viewing position and arbitrary time. We make use of a newly discovered tool, the "dual Htensor" that connects together three views of a coplanar configuration of (unlabeled) static and moving points. The newly synthesized images use constant velocity in the world to achieve realistic and physically correct images.
63 citations
"View synthesis of scenes with multi..." refers background in this paper
...In [3] and [4] Homography Tensors are used for view synthesis of dynamic points....
[...]