Showing papers by "Stefano Soatto published in 2000"

Optimal Structure from Motion: Local Ambiguities and Global Estimates

Abstract: “Structure From Motion” (SFM) refers to the problem of estimating spatial properties of a three-dimensional scene from the motion of its projection onto a two-dimensional surface, such as the retina. We present an analysis of SFM which results in algorithms that are provably convergent and provably optimal with respect to a chosen norm. In particular, we cast SFM as the minimization of a high-dimensional quadratic cost function, and show how it is possible to reduce it to the minimization of a two-dimensional function whose stationary points are in one-to-one correspondence with those of the original cost function. As a consequence, we can plot the reduced cost function and characterize the configurations of structure and motion that result in local minima. As an example, we discuss two local minima that are associated with well-known visual illusions. Knowledge of the topology of the residual in the presence of such local minima allows us to formulate minimization algorithms that, in addition to provably converge to stationary points of the original cost function, can switch between different local extrema in order to converge to the global minimum, under suitable conditions. We also offer an experimental study of the distribution of the estimation error in the presence of noise in the measurements, and characterize the sensitivity of the algorithm using the structure of Fisher's Information matrix.

3-D Motion and Structure from 2-D Motion Causally Integrated over Time: Implementation

Abstract: The causal estimation of three-dimensional motion from a sequence of two-dimensional images can be posed as a nonlinear filtering problem. We describe the implementation of an algorithm whose uniform observability, minimal realization and stability have been proven analytically in [5]. We discuss a scheme for handling occlusions, drift in the scale factor and tuning of the filter. We also present an extension to partially calibrated camera models and prove its observability. We report the performance of our implementation on a few long sequences of real images. More importantly, however, we have made our real-time implementation - which runs on a personal computer - available to the public for first-hand testing.

Real-time 3D motion and structure of point features: a front-end system for vision-based control and interaction

Abstract: We present a system that consists of one camera connected to a personal computer that can (a) select and track a number of high-contrast point features on a sequence of images, (b) estimate their three-dimensional motion and position relative to an inertial reference frame, assuming rigidity, (c) handle occlusions that cause point-features to disappear as well as new features to appear The system can also (d) perform partial self-calibration and (e) check for consistency of the rigidity assumption, although these features are not implemented in the current release All of this is done automatically and in real-time (30 Hz) for 40-50 point features using commercial off-the-shelf hardware The system is based on an algorithm presented by Chiuso et al (2000), the properties of which have been analyzed by Chiuso and Soatto (2000) In particular, the algorithm is provably observable, provably minimal and provably stable- under suitable conditions The core of the system, consisting of C++ code ready to interface with a frame grabber as well as Matlab code for development, is available at http://eewustledu/-soatto/researchhtml We demonstrate the system by showing its use as (1) an ego-motion estimator, (2) an object tracker, and (3) an interactive input device, all without any modification of the system settings

Euclidean Reconstruction and Reprojection Up to Subgroups

Abstract: The necessary and sufficient conditions for being able to estimate scene structure, motion and camera calibration from a sequence of images are very rarely satisfied in practice. What exactly can be estimated in sequences of practical importance, when such conditions are not satisfied? In this paper we give a complete answer to this question. For every camera motion that fails to meet the conditions, we give explicit formulas for the ambiguities in the reconstructed scene, motion and calibration. Such a characterization is crucial both for designing robust estimation algorithms (that do not try to recover parameters that cannot be recovered), and for generating novel views of the scene by controlling the vantage point. To this end, we characterize explicitly all the vantage points that give rise to a valid Euclidean reprojection regardless of the ambiguity in the reconstruction. We also characterize vantage points that generate views that are altogether invariant to the ambiguity. All the results are presented using simple notation that involves no tensors nor complex projective geometry, and should be accessible with basic background in linear algebra.

Shape and Radiance Estimation from the Information-Divergence of Blurred Images

Abstract: We formulate the problem of reconstructing the shape and radiance of a scene as the minimization of the information divergence between blurred images, and propose an algorithm that is provably convergent and guarantees that the solution is admissible, in the sense of corresponding to a positive radiance and imaging kernel. The motivation for the use of information divergence comes from the work of Csiszar [5], while the fundamental elements of the proof of convergence come from work by Snyder et al. [14], extended to handle unknown imaging kernels (i.e. the shape of the scene).

Monte Carlo filtering on Lie groups

Abstract: We propose a nonlinear filter for estimating the trajectory of a random walk on a matrix Lie group with constant computational complexity. It is based on a finite-dimensional approximation of the conditional distribution of the state-given past measurements-via a set of fair samples, which are updated at each step and proven to be consistent with the updated conditional distribution. The algorithm proposed, like other Monte Carlo methods, can in principle track arbitrary distributions evolving on arbitrarily large state spaces. However, several issues concerning sample impoverishment need to be taken into account when designing practical working systems.

Stereoscopic shading: integrating multi-frame shape cues in a variational framework

Abstract: We address the problem of integrating multi-frame stereo and shading cues within the framework of optimization in the infinite-dimensional space of piecewise smooth surfaces. Cue integration then reduces to the determination of regions where prior assumptions on the reflectance of the surfaces can be enforced. By combining cues, our formulation allows defining a well-posed problem even when reconstruction from stereo or shading in isolation would be ill-posed. For a simplified model we prove the necessary conditions for optimality, and propose an iterative optimization algorithm, which we implement using ultra-narrowband level set methods.

A geometric approach to blind deconvolution with application to shape from defocus

Abstract: We propose a solution to the generic "bilinear calibration-estimation problem" when using a quadratic cost function and restricting to (locally) translation-invariant imaging models. We apply the solution to the problem of reconstructing the three-dimensional shape and radiance of a scene from a number of defocused images. Since the imaging process maps the continuum of three-dimensional space onto the discrete pixel grid, rather than discretizing the continuum we exploit the structure of maps between (finite-and infinite-dimensional) Hilbert spaces and arrive at a principled algorithm that does not involve any choice of basis or discretization. Rather, these are uniquely determined by the data, and exploited in a functional singular value decomposition in order to obtain a regularized solution.