While many algorithms for computing stereo correspondence have been proposed, there has been very little work on experimentally evaluating algorithm performance, especially using real (rather than synthetic) imagery. In this paper we propose an experimental comparison of several different stereo algorithms. We use real imagery, and explore two different methodologies, with different strengths and weaknesses. Our first methodology is based upon manual computation of dense ground truth. Here we make use of a two stereo pairs: one of these, from the University of Tsukuba, contains mostly fronto-parallel surfaces; while the other, which we built, is a simple scene with a slanted surface. Our second methodology uses the notion of prediction error, which is the ability of a disparity map to predict an (unseen) third image, taken from a known camera position with respect to the input pair. We present results for both correlation-style stereo algorithms and techniques based on global methods such as energy minimization. Our experiments suggest that the two methodologies give qualitatively consistent results. Source images and additional materials, such as the implementations of various algorithms, are available on the web from http://www.research.microsoft.com/ ̃szeliski/stereo.

Vision Algorithms: Theory and Practice

We improve and adjust usual weak calibration techniques to the case of stereo video microscopes: Harris detector using a simplex optimization method for feature points detection, a "cha" window based ZNSSD correlation for points matching. Images of a pattern made with a water drop covered with nickel fillings are used. The result is validated by constructing 3D view of a micromanipulation work field.

/pdf/performing-weak-calibration-at-the-microscale-application-to-1wfv1slrfe.pdf

Performing Weak Calibration at the Microscale, Application to Micromanipulation

This paper addresses the problem of characterizing the set of all images of a rigid set of m points and n lines observed by a weak perspective or paraperspective camera. By taking explicitly into account the Euclidean constraints associated with calibrated cameras, we show that the corresponding image space can be represented by a six-dimensional variety embedded in {\cal R}^{2(m+n)} and parameterized by the image positions of three reference points. The coefficients defining this parameterized image variety (or PIV for short) can be estimated from a sample of images of a scene via linear and non-linear least squares. The PIV provides an integrated framework for using both point and line features to synthesize new images from a set of pre-recorded pictures (image-based rendering). The proposed technique does not perform any explicit three-dimensional scene reconstruction but it supports hidden-surface elimination, texture mapping and interactive image synthesis at frame rate on ordinary PCs. It has been implemented and extensively tested on real data sets.

Image-Based Rendering Using Parameterized Image Varieties

This paper generalizes the parameterized image variety approach to image-based rendering proposed in so it can handle both points and lines in a unified setting. We show that the set of all images of a rigid set of m points and n lines observed by a weak perspective camera forms a six-dimensional variety embedded in R2(m+n). A parameterization of this variety by the image positions of three reference points is constructed via least squares techniques from point and line correspondences established across a sequence of images. It is used to synthesize new pictures without any explicit 3D model. Experiments with real image sequences are presented.

Point- and Line-Based Parameterized Image Varieties for Image-Based Rendering

This paper generalizes the parameterized image variety approach to image-based rendering proposed in [5] so it can handle both points and lines in a unified setting. We show that the set of all images of a rigid set of m points and n lines observed by a weak perspective camera forms a six-dimensional variety embedded in IR 2(m+n) . A parameterization of this variety by the image positions of three reference points is constructed via least squares techniques from point and line correspondences established across a sequence of images. It is used to synthesize new pictures without any explicit 3D model. Experiments with real image sequences are presented.

Point- and line-based parameterized image varieties for image-based rendering. Commentary

This thesis introduces two novel techniques for the analysis and synthesis of image sequences: a linear algorithm for weak calibration of a stereo rig from point correspondences, and an algorithm for image-based rendering without explicit three-dimensional reconstruction based on point and line correspondences. By recasting the epipolar constraint in a projective setting with an appropriate basis choice, we first show that Jepson''s and Heeger''s linear subspace algorithm for infinitesimal motion estimation can be generalized to the finite motion case. This yields a linear method for weak calibration. The algorithm has been implemented and tested on both real and synthetic images, and it is compared to other linear and non-linear approaches to weak calibration. We then show that the set of all images of a rigid scene taken by a Euclidean camera is a six-dimensional variety, and we introduce a parameterization (called parameterized image variety, or PIV in short) of this variety for weak perspective and paraperspective cameras in terms of the image positions of three reference points. This parameterization can be estimated via linear least-squares and non-linear least-squares with low-degree equations. We use parameterized image varieties of both point and line features to synthesize new images from a set of pre-recorded pictures without actual three-dimensional reconstruction (image-based rendering) in an integrated framework. The method has been implemented and extensively tested on real data sets. Finally, we show how to adapt recent advances in statistically-unbiased least-squares methods to our image-based rendering approach. The point-based PIV involves equations with bilinear or higher-order data dependencies and we show how to efficiently estimate its parameters by adapting Leedan''s and Meer''s technique for bilinear estimation problems.

Yakup Genc

Papers

Weak calibration and image-based rendering algorithms