The viewing graph represents a set of views that are related by pairwise relative geometries. In the context of Structure-from-Motion (SfM), the viewing graph is the input to the incremental or global estimation pipeline. Much effort has been put towards developing robust algorithms to overcome potentially inaccurate relative geometries in the viewing graph during SfM. In this paper, we take a fundamentally different approach to SfM and instead focus on improving the quality of the viewing graph before applying SfM. Our main contribution is a novel optimization that improves the quality of the relative geometries in the viewing graph by enforcing loop consistency constraints with the epipolar point transfer. We show that this optimization greatly improves the accuracy of relative poses in the viewing graph and removes the need for filtering steps or robust algorithms typically used in global SfM methods. In addition, the optimized viewing graph can be used to efficiently calibrate cameras at scale. We combine our viewing graph optimization and focal length calibration into a global SfM pipeline that is more efficient than existing approaches. To our knowledge, ours is the first global SfM pipeline capable of handling uncalibrated image sets.

/pdf/optimizing-the-viewing-graph-for-structure-from-motion-3yc4aoe9ru.pdf

Optimizing the Viewing Graph for Structure-from-Motion

In this paper we present four cases of minimal solutions for two-view relative pose estimation by exploiting the affine transformation between feature points and we demonstrate efficient solvers for these cases. It is shown, that under the planar motion assumption or with knowledge of a vertical direction, a single affine correspondence is sufficient to recover the relative camera pose. The four cases considered are two-view planar relative motion for calibrated cameras as a closed-form and a least-squares solution, a closed-form solution for unknown focal length and the case of a known vertical direction. These algorithms can be used efficiently for outlier detection within a RANSAC loop and for initial motion estimation. All the methods are evaluated on both synthetic data and real-world datasets from the KITTI benchmark. The experimental results demonstrate that our methods outperform comparable state-of-the-art methods in accuracy with the benefit of a reduced number of needed RANSAC iterations.

/pdf/minimal-solutions-for-relative-pose-with-a-single-affine-6i0ah9hhi8.pdf

Minimal Solutions for Relative Pose With a Single Affine Correspondence

The Hurwitz form of a variety is the discriminant that characterizes linear spaces of complementary dimension which intersect the variety in fewer than degree many points. We study computational aspects of the Hurwitz form, relate this to the dual variety and Chow form, and show why reduced degenerations are special on the Hurwitz polytope.

The Hurwitz Form of a Projective Variety

We present a new insight into the systematic generation of minimal solvers in computer vision, which leads to smaller and faster solvers. Many minimal problem formulations are coupled sets of linear and polynomial equations where image measurements enter the linear equations only. We show that it is useful to solve such systems by first eliminating all the unknowns that do not appear in the linear equations and then extending solutions to the rest of unknowns. This can be generalized to fully non-linear systems by linearization via lifting. We demonstrate that this approach leads to more efficient solvers in three problems of partially calibrated relative camera pose computation with unknown focal length and/or radial distortion. Our approach also generates new interesting constraints on the fundamental matrices of partially calibrated cameras, which were not known before.

/pdf/a-clever-elimination-strategy-for-efficient-minimal-solvers-fzibj2nndf.pdf

A Clever Elimination Strategy for Efficient Minimal Solvers

PART 1: THE ORIGINS AND DEVELOPMENT OF GEOMETRICAL KNOWLEDGE PART 2: ABSTRACT PROJECTIVE GEOMETRY

Algebraic Projective Geometry. By J. G. Semple and G. T. Kneebone. Pp. viii, 404. 35s. 1952. (Oxford University Press)

The Euclidean distance degree of a real variety is an important invariant arising in distance minimization problems. We show that the Euclidean distance degree of an orthogonally invariant matrix variety equals the Euclidean distance degree of its restriction to diagonal matrices. We illustrate how this result can greatly simplify calculations in concrete circumstances.

The euclidean distance degree of orthogonally invariant matrix varieties

This paper considers the foundational question of the existence of a fundamental (resp. essential) matrix given m point correspondences in two views. We present a complete answer for the existence of fundamental matrices for any value of m. We disprove the widely held beliefs that fundamental matrices always exist whenever $$m \le 7$$m≤7. At the same time, we prove that they exist unconditionally when $$m \le 5$$m≤5. Under a mild genericity condition, we show that an essential matrix always exists when $$m \le 4$$m≤4. We also characterize the six and seven point configurations in two views for which all matrices satisfying the epipolar constraint have rank at most one.

On the Existence of Epipolar Matrices

This paper considers the foundational question of the existence of a fundamental (resp. essential) matrix given $m$ point correspondences in two views. We present a complete answer for the existence of fundamental matrices for any value of $m$. Using examples we disprove the widely held beliefs that fundamental matrices always exist whenever $m \leq 7$. At the same time, we prove that they exist unconditionally when $m \leq 5$. Under a mild genericity condition, we show that an essential matrix always exists when $m \leq 4$. We also characterize the six and seven point configurations in two views for which all matrices satisfying the epipolar constraint have rank at most one.

Given a set of point correspondences in two images, the existence of a fundamental matrix is a necessary condition for the points to be the images of a 3-dimensional scene imaged with two pinhole cameras. If the camera calibration is known then one requires the existence of an essential matrix. 
We present an efficient algorithm, using exact linear algebra, for testing the existence of a fundamental matrix. The input is any number of point correspondences. For essential matrices, we characterize the solvability of the Demazure polynomials. In both scenarios, we determine which linear subspaces intersect a fixed set defined by non-linear polynomials. The conditions we derive are polynomials stated purely in terms of image coordinates. They represent a new class of two-view invariants, free of fundamental (resp.~essential)~matrices.

Certifying the Existence of Epipolar Matrices

Minimizing the Euclidean distance to a set arises frequently in applications. When the set is algebraic, a measure of complexity of this optimization problem is its number of critical points. In this paper we provide a general framework to compute and count the real smooth critical points of a data matrix on an orthogonally invariant set of matrices. The technique relies on “transfer principles” that allow calculations to be done in the space of singular values of the matrices in the orthogonally invariant set. The calculations often simplify greatly and yield transparent formulas. We illustrate the method on several examples and compare our results to the recently introduced notion of Euclidean distance degree of an algebraic variety.

Hon-Leung Lee

Papers

The euclidean distance degree of orthogonally invariant matrix varieties

On the Existence of Epipolar Matrices

On the Existence of Epipolar Matrices

Certifying the Existence of Epipolar Matrices

Counting real critical points of the distance to orthogonally invariant matrix sets