Weak calibration and image-based rendering algorithms

TLDR: This thesis introduces two novel techniques for the analysis and synthesis of image sequences: a linear algorithm for weak calibration of a stereo rig from point correspondences, and an algorithm for image-based rendering without explicit three-dimensional reconstruction based on point and line correspondences.

Abstract: This thesis introduces two novel techniques for the analysis and synthesis of image sequences: a linear algorithm for weak calibration of a stereo rig from point correspondences, and an algorithm for image-based rendering without explicit three-dimensional reconstruction based on point and line correspondences. By recasting the epipolar constraint in a projective setting with an appropriate basis choice, we first show that Jepson''s and Heeger''s linear subspace algorithm for infinitesimal motion estimation can be generalized to the finite motion case. This yields a linear method for weak calibration. The algorithm has been implemented and tested on both real and synthetic images, and it is compared to other linear and non-linear approaches to weak calibration. We then show that the set of all images of a rigid scene taken by a Euclidean camera is a six-dimensional variety, and we introduce a parameterization (called parameterized image variety, or PIV in short) of this variety for weak perspective and paraperspective cameras in terms of the image positions of three reference points. This parameterization can be estimated via linear least-squares and non-linear least-squares with low-degree equations. We use parameterized image varieties of both point and line features to synthesize new images from a set of pre-recorded pictures without actual three-dimensional reconstruction (image-based rendering) in an integrated framework. The method has been implemented and extensively tested on real data sets. Finally, we show how to adapt recent advances in statistically-unbiased least-squares methods to our image-based rendering approach. The point-based PIV involves equations with bilinear or higher-order data dependencies and we show how to efficiently estimate its parameters by adapting Leedan''s and Meer''s technique for bilinear estimation problems.