Showing papers by "Shai Avidan published in 2000"

Layer extraction from multiple images containing reflections and transparency

Abstract: Many natural images contain reflections and transparency, i.e., they contain mixtures of reflected and transmitted light. When viewed from a moving camera, these appear as the superposition of component layer images moving relative to each other. The problem of multiple motion recovery has been previously studied by a number of researchers. However no one has yet demonstrated how to accurately recover the component images themselves. In this paper we develop an optimal approach to recovering layer images and their associated motions from an arbitrary number of composite images. We develop two different techniques for estimating the component layer images given known motion estimates. The first approach uses constrained least squares to recover the layer images. The second approach iteratively refines lower and upper bounds on the layer images using two novel compositing operations, namely minimum- and maximum-composites of aligned images. We combine these layer extraction techniques with a dominant motion estimator and a subsequent motion refinement stage. This results in a completely automated system that recovers transparent images and motions from a collection of input images.

Trajectory triangulation: 3D reconstruction of moving points from a monocular image sequence

Abstract: We consider the problem of reconstructing the 3D coordinates of a moving point seen from a monocular moving camera, i.e., to reconstruct moving objects from line-of-sight measurements only. The task is feasible only when some constraints are placed on the shape of the trajectory of the moving point. We coin the family of such tasks as "trajectory triangulation." We investigate the solutions for points moving along a straight-line and along conic-section trajectories, We show that if the point is moving along a straight line, then the parameters of the line (and, hence, the 3D position of the point at each time instant) can be uniquely recovered, and by linear methods, from at least five views. For the case of conic-shaped trajectory, we show that generally nine views are sufficient for a unique reconstruction of the moving point and fewer views when the conic is of a known type (like a circle in 3D Euclidean space for which seven views are sufficient). The paradigm of trajectory triangulation, in general, pushes the envelope of processing dynamic scenes forward. Thus static scenes become a particular case of a more general task of reconstructing scenes rich with moving objects (where an object could be a single point).

Integrating Local Affine into Global Projective Images in the Joint Image Space

Abstract: The fundamental matrix defines a nonlinear 3D variety in the joint image space of multiple projective (or "uncalibrated perspective") images. We show that, in the case of two images, this variety is a 4D cone whose vertex is the joint epipole (namely the 4D point obtained by stacking the two epipoles in the two images). Affine (or "para-perspective") projection approximates this nonlinear variety with a linear subspace, both in two views and in multiple views. We also show that the tangent to the projective joint image at any point on that image is obtained by using local affine projection approximations around the corresponding 3D point. We use these observations to develop a new approach for recovering multiview geometry by integrating multiple local affine joint images into the global projective joint image. Given multiple projective images, the tangents to the projective joint image are computed using local affine approximations for multiple image patches. The affine parameters from different patches are combined to obtain the epipolar geometry of pairs of projective images. We describe two algorithms for this purpose, including one that directly recovers the image epipoles without recovering the fundamental matrix as an intermediate step.

On the Reprojection of 3D and 2D Scenes Without Explicit Model Selection

Abstract: It is known that recovering projection matrices from planar configurations is ambiguous, thus, posing the problem of model selection -- is the scene planar (2D) or non-planar (3D)? For a 2D scene one would recover a homography matrix, whereas for a 3D scene one would recover the fundamental matrix or trifocal tensor. The task of model selection is especially problematic when the scene is neither 2D nor 3D -- for example a "thin" volume in space. In this paper we show that for certain tasks, such as reprojection, there is no need to select a model. The ambiguity that arises from a 2D scene is orthogonal to the reprojection process, thus if one desires to use multilinear matching constraints for transferring points along a sequence of views it is possible to do so under any situation of 2D, 3D or "thin" volumes.