scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Using Sparse Elimination for Solving Minimal Problems in Computer Vision

01 Oct 2017-pp 76-84
TL;DR: A new algorithm is proposed for selecting the basis that is in general more compact than the basis obtained with a state-of-the-art algorithm making PEP a more viable option for solving polynomial equations.
Abstract: Finding a closed form solution to a system of polynomial equations is a common problem in computer vision as well as in many other areas of engineering and science. Grob-ner basis techniques are often employed to provide the solution, but implementing an efficient Grobner basis solver to a given problem requires strong expertise in algebraic geometry. One can also convert the equations to a polynomial eigenvalue problem (PEP) and solve it using linear algebra, which is a more accessible approach for those who are not so familiar with algebraic geometry. In previous works PEP has been successfully applied for solving some relative pose problems in computer vision, but its wider exploitation is limited by the problem of finding a compact monomial basis. In this paper, we propose a new algorithm for selecting the basis that is in general more compact than the basis obtained with a state-of-the-art algorithm making PEP a more viable option for solving polynomial equations. Another contribution is that we present two minimal problems for camera self-calibration based on homography, and demonstrate experimentally using synthetic and real data that our algorithm can provide a numerically stable solution to the camera focal length from two homographies of unknown planar scene.

Summary (2 min read)

1. Introduction

  • Many camera pose estimation and calibration problems boil down to solving a system of polynomial equations.
  • Fitzgibbon [8] augmented the fundamental matrix estimation to include one term of radial lens distortion, and solved them from 9 point correspondences.
  • One drawback of this approach is that when the polynomial degrees are high, it often suffers from numerical inaccuracies.

2. Polynomial eigenvalue problems

  • ,Cl are m × m square matrices that contain the coefficients of the polynomials.
  • Since most of the mathematical software libraries and packages can solve this problem it becomes easy to find all the roots for λ.
  • Notice that this procedure also increases the number of monomials in v and hence the dimensions of the coefficient matrices.
  • Therefore, they proposed a small modification to the resultant-based approach that gives a higher number of polynomial equations that increases the chances to get a linearly independent set of basis monomials.
  • Therefore, this approach is feasible only for small systems of equations and low polynomial degrees.

3. Determining basis monomials

  • Most of the polynomial equations encountered in computer vision are sparse, and therefore classical multivariate resultants are not well-suited for generating the basis monomials.
  • Next the authors discuss about finding the basis monomials for the polynomial eigenvalue problem, i.e., the elements of v in (1).
  • The main disadvantage of the resultant-based approach for solving the polynomial equations is that it requires computing the determinant of a matrix which often has high dimensions.
  • Due to the relaxed requirements, the authors can try to find a smaller set of basis monomials than (7) defined for the sparse resultant.
  • It should be noticed that the mixed volume is computed for the original system (3).

4. Planar self-calibration

  • To demonstrate the applicability of their algorithm, the authors present two minimal problems for solving the camera focal length from two homographies corresponding to three images where the patterns are unknown, which makes this a self-calibration problem.
  • Because there are now three unknowns only three constraints are needed.
  • The authors obtain 135 putative bases B (62 valid), and from those they select the minimal one which has more than or equal to ceil(70/4) = 18 monomials and produces 18 or more equations.
  • Hence, besides the actual 70 roots the authors get 12 spurious roots that can be found, for example, by substituting the solutions obtained to the original equations.
  • Interestingly, the authors cannot solve the focal length of the first camera λ0 in this case, because the variety is no longer zero-dimensional, and a single reference image does not provide enough constraints for λ0, but it can still be used to solve λ1.

5. Experiments

  • The authors show experimentally that the selfcalibration methods presented in Section 4 give numerically stable results both with synthetic and real data.
  • The authors randomly picked 4 points from the dataset and computed the homographies.
  • The random sampling for images and points is again repeated 30, 000 times to get statistically reliable results.
  • The relative errors are plotted in Fig. 3 (a) both for the noiseless case and for σ = 0.5 using the simulated data.

6. Conclusions

  • The authors have proposed a new algorithm for selecting the monomial basis for polynomial eigenvalue problems based on sparse elimination that has been previously used for constructing sparse resultants.
  • The authors approach has two important advantages over the sparse resultants: 1) the solution is provided by eigenvalues instead of the roots of a high-order determinant, and 2) the cofficient matrices do not need to be of full rank unlike sparse resultants that simplifies the algorithm and often leads to a more compact basis.
  • In contrast to the modified resultant-based method [13] their algorithm can exploit the sparsity of the polynomials that is a common property in real-world problems.
  • As a result, the monomial basis becomes smaller, and it is the same only in the limiting case where the polynomials are dense.
  • The authors also presented two new minimal problems for camera self-calibration, and demonstrated that their algorithm can provide numerically more stable results than the modified resultant-based method.

Did you find this useful? Give us your feedback

Content maybe subject to copyright    Report

Using Sparse Elimination for Solving Minimal Problems in Computer Vision
Janne Heikkilä
Center for Machine Vision and Signal Analysis
University of Oulu, Finland
janne.heikkila@oulu.fi
Abstract
Finding a closed form solution to a system of polynomial
equations is a common problem in computer vision as well
as in many other areas of engineering and science. Gröb-
ner basis techniques are often employed to provide the so-
lution, but implementing an efficient Gröbner basis solver
to a given problem requires strong expertise in algebraic
geometry. One can also convert the equations to a poly-
nomial eigenvalue problem (PEP) and solve it using linear
algebra, which is a more accessible approach for those who
are not so familiar with algebraic geometry. In previous
works PEP has been successfully applied for solving some
relative pose problems in computer vision, but its wider ex-
ploitation is limited by the problem of finding a compact
monomial basis. In this paper, we propose a new algorithm
for selecting the basis that is in general more compact than
the basis obtained with a state-of-the-art algorithm making
PEP a more viable option for solving polynomial equations.
Another contribution is that we present two minimal prob-
lems for camera self-calibration based on homography, and
demonstrate experimentally using synthetic and real data
that our algorithm can provide a numerically stable solu-
tion to the camera focal length from two homographies of
unknown planar scene.
1. Introduction
Many camera pose estimation and calibration problems boil
down to solving a system of polynomial equations. These
are often so-called minimal problems, where the camera
parameters are computed from a minimal number of con-
straints so that there are essentially as many unknowns as
equations, but the relationship between the unknown vari-
ables and the measurements follows a polynomial model
that makes the dependence nonlinear and difficult to solve
by means of linear algebra. Such minimal problems in-
clude for example, the classical P3P (Perspective-Three-
Point) problem for a calibrated camera where an image of
three points with known distances is sufficient to compute
the camera pose, but it requires solving a system of three
quadratic equations in three variables [
9]. Another clas-
sical example is the five-point problem that allows find-
ing the relative pose between two views from an unknown
scene using five point correspondences. Nistér [
17] con-
verted the resulting system of polynomial equations to a
tenth degree univariate polynomial that can be efficiently
solved using standard numerical techniques. The relative
pose problem has been modified in various studies to in-
corporate also unknown camera parameters that enable the
use of uncalibrated cameras and makes it a self-calibration
problem. To mention few of them, Stewenius et al. [
20]
used six point correspondences to solve the relative pose
together with the focal length. Fitzgibbon [
8] augmented
the fundamental matrix estimation to include one term of
radial lens distortion, and solved them from 9 point corre-
spondences. Kukelova and Pajdla [
15] used an additional
constraint to solve the same problem from 8 point corre-
spondences, and Jiang et al. [
11] added still one constraint
and they were able to solve the problem from 7 point corre-
spondences. A comprehensive list of minimal problems in
computer vision and related papers can be found in [
18].
Planar objects are commonly used for estimating the
camera pose and intrinsic parameters. Well-known Zhang’s
calibration method [23] provides a closed form solution to
the calibration problem from images of a known planar tar-
get. Also, OpenCV and Matlab include tools to perform cal-
ibration with a similar setup. Despite of the extensive num-
ber of minimal problems introduced in recent years, it is
surprising that homography has not been much considered
in this context, and there are only few related works. Min-
imal solutions to panorama stitching in [
1] and [2] assume
that the camera centers coincide which reduces the motion
to pure rotation. Methods for decomposing a homography
into rotation, translation, and surface normal parameters
have been proposed e.g. in [
7] and [24]. Saurer et al. [19]
consider a minimal solution to a 3-point plus a common di-
rection relative pose problem using homography. Recently,
Kukelova et al. [
14] have presented two algorithms for esti-
mating the homography between two cameras with different
radial distortions. However, none of these works address the
76

problem of solving the camera focal length from images of
an unknown planar target, which is the homography-based
minimal problem presented in this paper.
The most common approach for solving minimal prob-
lems in computer vision and corresponding systems of poly-
nomial equations is to use Gröbner basis techniques. One
drawback of this approach is that when the polynomial de-
grees are high, it often suffers from numerical inaccuracies.
To address this problem, for example, Byröd et al. [
3] have
proposed a generalization of the Gröbner basis method for
improving the numerical stability. Another limitation of this
approach is that implementing a Gröbner basis solver for a
given problem requires expertise in algebraic geometry be-
cause the solver needs to be handcrafted in practice to make
it efficient. Because of the complicated theory this approach
is often beyond the reach of non-experts. An alternative ap-
proach that is also commonly used for solving polynomial
equations is multipolynomial resultant, which provides an
efficient tool for eliminating variables from multiple equa-
tions, and solving the remaining variable as a root of a uni-
variate polynomial [
4]. However, there are some limita-
tions that make resultants less useful for engineering ap-
plications. For classical multipolynomial resultants such as
the Macaulay resultant, most of the polynomial coefficients
need to be non-zero, the roots distinct and there should be
no solutions at infinity. This problem can often be avoided
by using a sparse resultant [
6] that works also for polynomi-
als with several zero coefficients. Another disadvantage is
that after elimination the remaining univariate polynomial
is a determinant of a matrix that often has high dimensions.
Because a determinant of an N × N matrix has N! terms,
finding the roots of the remaining polynomial can easily be-
come computationally infeasible or unstable.
In addition to the Gröbner basis techniques and resul-
tants, systems of polynomial equations can be often solved
using eigenvalues and eigenvectors. One approach is to
convert the classical multipolynomial resultant to a stan-
dard eigenvalue problem [
5],[4] which however works only
with dense polynomials. In [
8] the minimal problem of
computing the radial distortion coefficient was expressed
as a quadratic polynomial eigenvalue problem and later it
was extended in [
16] to include an additional constraint.
A resultant-based algorithm for transforming a system of
polynomial equations to a polynomial eigenvalue problem
(PEP) was proposed in [
13] that enabled solving several
minimal relative pose problems using linear algebra. How-
ever, this algorithm has an inherent property of leading to
unnecessarily high dimensional vector spaces and spurious
roots that make the algorithm numerically unstable when
solving sparse systems of polynomials with high degrees.
To overcome the problems related to the algorithm pre-
sented in [13], we propose a new algorithm based on sparse
elimination theory that provides more stable solutions to
sparse systems that are the most typical cases in practical
applications. In addition, we demonstrate the applicablility
of our algorithm in two new minimal problems that have
higher number of solutions than typical relative pose prob-
lems previously presented in the literature.
2. Polynomial eigenvalue problems
Polynomial eigenvalue problem (PEP) is an extension of the
standard eigenvalue problem (C λI)v = 0 to a system of
polynomials represented with the matrix equation:
(C
0
+ C
1
λ + C
2
λ
2
+ · · · + C
l
λ
l
)v = 0, (1)
where l is the highest degree of the polynomials in the vari-
able λ that we want to solve, v is a vector of monomials in
other variables than λ, and C
0
, . . . , C
l
are m × m square
matrices that contain the coefficients of the polynomials.
This equation can be converted to a generalized eigenvalue
problem
Au = λBu, (2)
where
A =
0 I 0 · · · 0
0 0 I · · · 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
C
0
C
1
C
2
· · · C
l1
,
B =
I 0 0 · · · 0
0 I 0 · · · 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 0 · · · C
l
, u =
v
λv
.
.
.
λ
l1
v
.
Since most of the mathematical software libraries and pack-
ages can solve this problem it becomes easy to find all the
roots for λ. The eigenvector u contains the solutions of the
monomials that exist in the polynomials, and therefore, one
can extract the roots of the remaining variables by comput-
ing suitable ratios between individual elements of u.
The most difficult part of converting the system of poly-
nomials to PEP is to determine the monomials in v and
consequently the matrices C
0
, . . . , C
l
. Given n polynomi-
als one can easily construct n × m matrices and the cor-
responding v satisfying (
1), but usually n < m which re-
sults in an underdetermined system of linear equations that
cannot be solved. The trick emplyed in [
13] is to gener-
ate new equations by multiplying the initial equations with
monomials produced by their algorithm. Some of these new
equations may be linearly independent from the initial equa-
tions, which then enables constructing a fully determined
system. Notice that this procedure also increases the num-
ber of monomials in v and hence the dimensions of the co-
efficient matrices. In [
13] they used the classical Macaulay
resultant formulation for creating the set of basis monomials
77

in v. Because the Macaulay resultant is designed for dense
homogeneous polynomials, it is not guaranteed to produce
a basis that is linearly independent. Therefore, they pro-
posed a small modification to the resultant-based approach
that gives a higher number of polynomial equations that in-
creases the chances to get a linearly independent set of ba-
sis monomials. However, for larger systems of polynomials
the basis generated by this method becomes huge, because
the Macaulay resultant is based on the assumption that the
number of the solutions obtained is maximal for the given
degrees of the polynomials. According to Bézout’s theo-
rem [
4] the maximum number of solutions is d
1
· d
2
· · · d
n
,
where d
i
is the degree of the polynomial f
i
. The set of ba-
sis monomials contain all the monomials with total degree
of d =
P
(d
i
1) + 1. One can easily see that the num-
ber of monomials increases exponentially. Therefore, this
approach is feasible only for small systems of equations
and low polynomial degrees. Next, we present a method
based on sparse elimination that exploits the sparsity of the
general polynomials, and produces smaller monomial bases
and coefficient matrices enabling solutions to problems that
have been previously intractable.
3. Determining basis monomials
Most of the polynomial equations encountered in computer
vision are sparse, and therefore classical multivariate resul-
tants are not well-suited for generating the basis monomi-
als. Sparse elimination theory [
21],[5] has been developed
to deal with general polynomials that have many zero co-
efficients. The benefit of the sparsity is that the resultants
obtained have much smaller dimensions than the classical
resultants. Therefore, instead of selecting all the monomials
of a certain total degree we can get a significantly smaller
set of monomials by using the tools provided by the sparse
elimination theory.
Let x = {x
1
, x
2
, . . . , x
n
} be a set of unknown variables
that we want to solve from n multivariate polynomials
f
1
(x) = f
2
(x) = · · · = f
n
(x) = 0 (3)
defined by
f
i
(x) =
s
i
X
j=1
c
ij
x
a
ij
, (4)
where x
a
ij
= x
α
ij1
1
x
α
ij2
2
· · · x
α
ijn
n
are the monomi-
als corresponding to the non-zero coefficients c
ij
. Let
A
i
= {a
i1
, . . . , a
is
i
} Z
n
+
denote the exponent vectors of
all the monomials in f
i
that is also called the support of f
i
.
Next we introduce few concepts from algebraic geometry
[
4] that are needed to formulate our method.
Definition 1: The Newton polytope of f
i
is the convex hull
of the support A
i
denoted by P
i
= Conv(A
i
) R
n
. The
volume of P
i
is denoted by Vol
n
(P
i
).
Notice that in the low dimensional cases when
n = 1, 2 or 3, the Newton polytope represents a line,
polygon or polyhedron, respectively. Clearly, the way
how the volume Vol
n
(P
i
) is computed depends on n. For
example, Vol
1
(P
i
) is the length of the line, and Vol
2
(P
i
) is
the area of the polygon.
Definition 2: The Minkowski sum of two convex polytopes
P
i
and P
j
is the convex polytope
P
ij
= P
i
+ P
j
= {p
i
+ p
j
|p
i
P
i
, p
j
P
j
} R
n
.
Using the Minkowski sum (also known as dilation) we can
aggregate the Newton polytopes of individual polynomials
f
i
to form combined supports. It is also needed for defining
the mixed volume.
Definition 3: Given convex polytopes P
1
, . . . , P
n
R
n
there is a real-valued function called mixed volume that can
be computed as
MV
n
(P
1
, . . . , P
n
) =
n
X
k=1
(1)
nk
X
I⊂{1,...,n}
|I|=k
Vol
n
X
iI
P
i
!
.
(5)
In high-dimensional cases computing the mixed volume us-
ing (
5) can be time consuming. There are faster algorithms
that use a so called mixed subdivision of the Minkowski
sum, and one can also find their software implementations
from the Internet, but in the cases discussed in this paper
n 4 and using (
5) is still tractable. The following theorem
is the reason why we introduced the mixed volume.
Theorem 1 (Bernstein’s Theorem): Given the polynomi-
als f
1
, . . . f
n
over C with finitely many common zeroes in
(C
)
n
, where C
= C \ {0}, let P
i
be the Newton polytope
of f
i
in R
n
. Then the number of solutions of the f
i
in (C
)
n
is bounded above by the mixed volume MV
n
(P
1
, . . . , P
n
).
For generic choices of the coefficients c
ij
the number of
common solutions is exactly MV
n
(P
1
, . . . , P
n
).
The proof of the theorem can be found from [
4]. Bern-
stein’s theorem is an important result of the sparse elimina-
tion theory that gives us a tool for calculating the maximum
number of the roots in advance without knowing the numer-
ical values of the coefficients. What we need is only the ex-
ponent vectors a
ij
of the monomials, i.e., supports A
i
. This
also determines the minimum size of the monomial basis as
we will see later.
Next we discuss about finding the basis monomials for
the polynomial eigenvalue problem, i.e., the elements of
v in (
1). Sparse elimination provides the tools for con-
structing sparse resultants that generalize the classical mul-
tivariate resultant. While the degree of the classical mul-
tivariate resultant is determined by Bézout’s theorem (i.e.
d
1
·d
2
· · · d
n
), the degree of the sparse resultant comes from
Bernstein’s theorem which is the mixed volume. These two
types of resultants coincide only when all Newton polytopes
are n-simplices scaled by the total degree of the respec-
tive polynomials [
4]. Otherwise the degree of the sparse
78

resultant is smaller, which also means that the matrix con-
structed from the coefficients c
ij
has smaller dimensions.
Furthermore, it is necessary that the matrix has full rank
and its determinant vanishes only when the equations have
a common solution. It often happens that the multivariate
resultant is rank-deficient if the polynomials have zero co-
efficients, and thus it fails to provide a solution. In order
to have a full rank, it becomes necessary to select the basis
monomials of the sparse resultant carefully using, e.g., the
Lift-Prune algorithm proposed by Emiris & Canny [
6]. The
main disadvantage of the resultant-based approach for solv-
ing the polynomial equations is that it requires computing
the determinant of a matrix which often has high dimen-
sions. Because the determinant of an N × N matrix has N !
terms, solving the unknowns from the resultant often be-
comes computationally infeasible even for relatively small
problems. For example, if the dimension of the coefficient
matrix for the sparse resultant is 10 × 10, the resultant is a
factor of an expression that has more than 3.6 million terms.
In such cases finding the solution via PEP is much more ef-
ficient.
A sparse resultant to a system of n equations is com-
puted in n 1 variables, which means that one of the vari-
ables of our original problem (
3) needs to be hidden to the
coefficient field. The resultant obtained is then a univariate
polynomial of the hidden variable which can be solved by
finding the roots of this polynomial. Without loss of gener-
ality we can decide to solve the first variable x
1
that is then
treated as a coefficient in the polynomials
f
i
(
˜
x) =
X
s
i
j=1
c
ij
˜
x
a
ij
, (6)
where c
ij
=
P
α
ij1
c
ij
x
α
ij1
1
,
˜
x = {x
2
, . . . , x
n
} and
a
ij
= (α
ij2
, . . . , α
ijn
) A
i
are n 1 dimensional sup-
port vectors with |A
i
| = s
i
. The first step is to create the
Newton polytopes P
1
, . . . , P
n
R
n1
corresponding to
the modified system (
6), and compute the Minkowski sum
P = P
1
+ · · · + P
n
. The set of basis monomials S for the
sparse resultant is then obtained from
S = Z
n1
(P + d), (7)
where Z
n1
defines a square lattice with integer points, and
d R
n1
is a small translation vector that displaces P
slightly so that the lattice points lie in the interiors of the
convex polytope [
6, 4]. In practice, the elements of d can be
randomly selected from {−ǫ, 0, ǫ} where ǫ Q is a small
rational number.
Sparse resultants need to be of full rank in order to have
a non-zero determinant. In our case, the only strict require-
ment is that C
0
, . . . , C
l
in (1) must be square matrices. No-
tice that this is a looser condition than in [
13] where they
assumed that either C
l
or C
0
must be of full rank and in-
vertible. Here this not necessary, but in some cases rank-
deficiency may lead to numerical instability with the eigen-
value solver. Because the PEP in (
1) is defined for one un-
known variable λ which is then computed as an eigenvalue
of (
2) we need to choose this variable from x
1
, . . . , x
n
. This
is exactly the same situation as with the sparse resultant, and
therefore, we decide again without loss of generality that
λ x
1
, and we hide x
1
to the coefficient field which then
results in the modified system (
6).
Due to the relaxed requirements, we can try to find
a smaller set of basis monomials than (
7) defined for
the sparse resultant. The lower bound is determined by
Bernstein’s theorem which gives the maximum number of
the common roots for the polynomials denoted by r
MV
n
(P
1
, . . . , P
n
). It should be noticed that the mixed vol-
ume is computed for the original system (
3). The eigen-
vector u in (
2) has the same dimension as the maximum
number of unique eigenvalues i.e. possible roots of the sys-
tem. The length of u is clearly lm which gives us the bound
m
r
l
. (8)
Hence, it is sufficient to find a set of support vectors B for
the basis monomials where |B| r/l.
Algorithm
1 summarizes the procedure for constructing
B based on the previous discussion. It generates several
putative sets of support vectors for the basis and selects
the smallest set among these candidates. It also returns a
set T = {T
1
, . . . , T
n
} where T
i
6= are subsets of vec-
tors that can be used to construct the coefficient matrices
C
0
, . . . , C
l
. These vectors are first converted to n sets of
monomials M
i
= {
˜
x
t
| t T
i
}, and the monomials are
multiplied with the original equations f
i
which then results
in n sets of new equations E
i
= {
˜
x
t
f
i
(x)|
˜
x
t
M
i
}.
These equations are converted to a matrix form (
1), which
then directly gives us the coefficient matrices C
0
, . . . , C
l
.
The total number of new equations
P
i
|E
i
| is greater or
equal to the number of the basis monomials, which means
that the coefficient matrices have at least as many rows as
columns. If there are more rows than columns, one can
choose m rows that minimize the condition number, and
discard the remaining rows. It may also happen that the
most compact basis does not work, and in that case one
could try the next candidate produced by the algorithm.
There are often monomials (or vectors) in B that do not
contribute to the equations. Such monomials may cause in-
stability to the eigenvalue solver, and they need to be re-
moved. In [
13] they call them “parasitic” zero eigenval-
ues, and they propose to convert the generalized eigenvalue
problem to a standard eigenvalue problem so that one can
easily identify and remove these monomials as they corre-
spond to zero columns of the matrix to be decomposed. The
drawback is that either C
0
or C
l
need to be of full rank,
which then causes extra constraints to selection of the basis
monomials. Hence, we propose here a simple strategy for
finding these zero monomials. First, we need to specialize
the coefficient matrices with some random numerical val-
79

Algorithm 1 Generate basis monomials
Input: A
1
, . . . , A
n
, r, l
Output: B, T
1: Create Newton polytopes P
i
Conv(A
i
) R
n1
for
i = 1, . . . , n, and a unit (n 1)-simplex P
0
R
n1
.
2: Create a list of index sets:
K [{k
0
, . . . , k
i
} | i = 0, . . . , n ; k
0
, . . . , k
i
{0, . . . , n} ; k
j+1
> k
j
].
3: Create a list of displacement vectors:
[(δ
1
, . . . , δ
n1
) | δ
1
, . . . , δ
n1
{−ǫ, 0, ǫ}].
4: Initialize B , T and N .
5: for I in K do
6: Compute Minkowski sum Q
P
kI
P
k
.
7: for d in do
8: Create a putative basis B Z
n1
(Q + d).
9: if |B|
r
l
AND |B| < N then
10: Find the sets of vectors:
T
i
{t|t Z
n1
+
, A
i
+ t B}
for i = 1, . . . , n.
11: if
P
i
|T
i
| |B| AND min(|T
i
|) > 0 then
12: B B, T {T
i
}
i=1,...,n
, and
N |B|.
13: end if
14: end if
15: end for
16: end for
ues as c
ij
. Using these values we construct the matrices A
and B in (
2), and compute the singular value decomposition
B = USV
. Next we perform a unitary transformation
A
= U
AV, (9)
and find the zero columns of A
. These columns correspond
to the zero monomials and they can be removed from A and
B. The rows with the same indices are also removed so that
the matrices will remain square. This procedure might need
to be repeated few times to find all zero monomials. How-
ever, one should notice that the elimination is performed
offline when designing the solver, and there is no need to do
it runtime once the zero polynomials have been identified.
4. Planar self-calibration
A standard approach for geometric camera calibration is to
use a known checker board pattern printed on a planar sur-
face. To demonstrate the applicability of our algorithm, we
present two minimal problems for solving the camera focal
length from two homographies corresponding to three im-
ages where the patterns are unknown, which makes this a
self-calibration problem. We consider the following cases:
1) a constant focal length and 2) two different focal lengths.
The resulting polynomials can be converted to PEPs using
Algorithm
1 and solved efficiently, for example, with Mat-
lab or some other software package or library capable of
computing generalized eigenvalues.
Let two 3D vectors a and b span a plane so that they are
both orthogonal and of equal length fulfilling the constraints
a
b = 0 and a
a b
b = 0. (10)
If |a| = |b| = 1 the normal vector of the plane is defined
by n = a × b. In order to express the vectors a and b in
terms of the normal vector n we can choose
a = n × e and b = n × a, (11)
where e is a unit vector not parallel to n. For simplic-
ity, we select e = [1, 0, 0]
. We use parametrization
n = [n
x
, n
y
, 1]
/
q
n
2
x
+ n
2
y
+ 1, and we can now express
a and b in variables n
x
and n
y
. Further assuming that a and
b are represented in the camera coordinate frame of the ref-
erence view we can convert them to the image coordinates
using
ˆ
a
0
= K
0
a and
ˆ
b
0
= K
0
b, (12)
where K
0
is the intrinsic camera matrix for the reference
camera. Because one can often assume with reasonable ac-
curacy that the principal point of the camera is in the center
of the image, the pixel aspect ratio is 1, and lens distortion
is negligible, we limit ourselves to the case where we have
only one intrinsic parameter, the focal length λ
0
, that leads
to the camera matrix K
0
= diag(λ
0
, λ
0
, 1).
The mapping from the reference image to the i
th
image is
described by the homography H
i
. After back-projecting to
the 3D space the corresponding vectors are obtained from
a
i
= K
1
i
H
i
K
0
a and b
i
= K
1
i
H
i
K
0
b, (13)
where K
i
= d i ag(λ
i
, λ
i
, 1) and λ
i
is the focal length of
the i
th
camera. Because orthogonality and equality in the
length should hold in each frame we have the following self-
calibration constraints expressed by two polynomials
f
1,i
(λ
0
, λ
i
, n
x
, n
y
) = a
i
b
i
= 0 (14)
f
2,i
(λ
0
, λ
i
, n
x
, n
y
) = a
i
a
i
b
i
b
i
= 0. (15)
Herrera et al. [
10] used similar constraints for planar
self-calibration but they solved the camera parameters with
non-linear minimization. The solver was initialized by as-
suming that the reference view is fronto-parallel when it be-
comes easy to compute an initial value for the focal length.
In this paper, we do not make such assumptions, and no
initialization is needed. There are now 3 + k unknowns
λ
0
, . . . , λ
k
, n
x
and n
y
, and two equations which means
that we cannot solve the problem from a single homogra-
phy, but we need at least two homographies (i = 1, 2) that
lead to four equations with four unknowns. This is true also
in general, because one homography provides only 8 con-
straints to calibration. Five constraints are needed for the
camera pose (3 for rotation and 2 for translation up to scale),
the normal of the plane n requires two constraints and one
80

Citations
More filters
Journal ArticleDOI
TL;DR: It is shown that knowing the value of the inscribed angle between the two 3D rays poses additional constraints on the relative orientation, and using the latter enables the solution of the relative pose problem with as few as 3 correspondences across the two images.
Abstract: Corners are popular features for relative pose computation with 2D-2D point correspondences. Stable corners may be formed by two 3D rays sharing a common starting point. We call such elements ray-point-ray (RPR) structures. Besides a local invariant keypoint given by the lines’ intersection, their reprojection also defines a corner orientation and an inscribed angle in the image plane. The present paper investigates such RPR features, and aims at answering the fundamental question of what additional constraints can be formed from correspondences between RPR features in two views. In particular, we show that knowing the value of the inscribed angle between the two 3D rays poses additional constraints on the relative orientation. Using the latter enables the solution of the relative pose problem with as few as 3 correspondences across the two images. We provide a detailed analysis of all minimal cases distinguishing between 90-degree RPR-structures and structures with an arbitrary, known inscribed angle. We furthermore investigate the special cases of a known directional correspondence and planar motion, the latter being solvable with only a single RPR correspondence. We complete the exposition by outlining an image processing technique for robust RPR-feature extraction. Our results suggest high practicality in man-made environments, where 90-degree RPR-structures naturally occur.

25 citations


Cites methods from "Using Sparse Elimination for Solvin..."

  • ...Successful applications of the resultant method in computer vision can be found in [36], [37]....

    [...]

Proceedings ArticleDOI
14 Jun 2020
TL;DR: The proposed algorithms can cope with coplanar points, which is a degenerate configuration for these 6- and 7-point counterparts, and derive new 4- and 5-point algorithms for these two cases, respectively.
Abstract: We propose minimal solutions to relative pose estimation problem from two views sharing a common direction with unknown focal length. This is relevant for cameras equipped with an IMU (inertial measurement unit), e.g., smart phones, tablets. Similar to the 6-point algorithm for two cameras with unknown but equal focal lengths and 7-point algorithm for two cameras with different and unknown focal lengths, we derive new 4- and 5-point algorithms for these two cases, respectively. The proposed algorithms can cope with coplanar points, which is a degenerate configuration for these 6- and 7-point counterparts. We present a detailed analysis and comparisons with the state of the art. Experimental results on both synthetic data and real images from a smart phone demonstrate the usefulness of the proposed algorithms.

19 citations


Cites methods from "Using Sparse Elimination for Solvin..."

  • ...Polynomial eigenvalue methods have been successfully used for many minimal problems in computer vision, such as the 9-point one-parameter radial distortion problem [11], the 5- and 6-point relative pose problems [21], the 6point one unknown focal length problem [4], and the selfcalibration problems [17]....

    [...]

Proceedings ArticleDOI
14 Jun 2020
TL;DR: This paper studies an alternative algebraic method for solving systems of polynomial equations, i.e., the sparse resultant-based method and proposes a novel approach to convert the resultant constraint to an eigenvalue problem, which can significantly improve the efficiency and stability of existing resultant- based solvers.
Abstract: Many computer vision applications require robust and efficient estimation of camera geometry. The robust estimation is usually based on solving camera geometry problems from a minimal number of input data measurements, i.e. solving minimal problems in a RANSAC framework. Minimal problems often result in complex systems of polynomial equations. Many state-of-the-art efficient polynomial solvers to these problems are based on Grobner basis and the action-matrix method that has been automatized and highly optimized in recent years. In this paper we study an alternative algebraic method for solving systems of polynomial equations, i.e., the sparse resultant-based method and propose a novel approach to convert the resultant constraint to an eigenvalue problem. This technique can significantly improve the efficiency and stability of existing resultant-based solvers. We applied our new resultant-based method to a large variety of computer vision problems and show that for most of the considered problems, the new method leads to solvers that are the same size as the the best available Grobner basis solvers and of similar accuracy. For some problems the new sparse-resultant based method leads to even smaller and more stable solvers than the state-of-the-art Grobner basis solvers. Our new method can be fully automatized and incorporated into existing tools for automatic generation of efficient polynomial solvers and as such it represents a competitive alternative to popular Grobner basis methods for minimal problems in computer vision.

16 citations


Cites background or methods from "Using Sparse Elimination for Solvin..."

  • ...The most promising results in this direction were proposed by Emiris [12] and Heikkilä [18], where methods based on sparse resultants were proposed and applied to camera geometry problems....

    [...]

  • ...The most promising results in this direction were proposed by Emiris [12] and Heikkilä [18], where methods based on sparse resultants were proposed and applied to camera geometry problems....

    [...]

  • ...The augmented polynomial system is solved by hiding λ and reducing a constraint similar to (2) into a regular eigenvalue problem that leads to smaller solvers than [12, 18]....

    [...]

  • ...Our algorithm is inspired by the ideas explored in [18, 12], but thanks to the special form of added equation and by solving the resultant as a small eigenvalue problem, in contrast to a polynomial eigenvalue problem in [18], the new approach achieves significant improvements over [18, 12] in terms of efficiency of the generated solvers....

    [...]

  • ...The algorithm by Heikkilä [18] basically computes the Minkowski sum of the Newton polytopes of a subset of input polynomials, Q = ΣiNP(fi(x))....

    [...]

Posted Content
TL;DR: This paper studies an interesting alternative resultant-based method for solving sparse systems of polynomial equations by hiding one variable, which results in a larger eigenvalue problem than the action matrix and extra variable resultant- based methods; however, it does not need to compute an inverse or elimination of large matrices that may be numerically unstable.
Abstract: Many computer vision applications require robust and efficient estimation of camera geometry. The robust estimation is usually based on solving camera geometry problems from a minimal number of input data measurements, i.e., solving minimal problems, in a RANSAC-style framework. Minimal problems often result in complex systems of polynomial equations. The existing state-of-the-art methods for solving such systems are either based on Grobner bases and the action matrix method, which have been extensively studied and optimized in the recent years or recently proposed approach based on a sparse resultant computation using an extra variable. In this paper, we study an interesting alternative sparse resultant-based method for solving sparse systems of polynomial equations by hiding one variable. This approach results in a larger eigenvalue problem than the action matrix and extra variable sparse resultant-based methods; however, it does not need to compute an inverse or elimination of large matrices that may be numerically unstable. The proposed approach includes several improvements to the standard sparse resultant algorithms, which significantly improves the efficiency and stability of the hidden variable resultant-based solvers as we demonstrate on several interesting computer vision problems. We show that for the studied problems, our sparse resultant based approach leads to more stable solvers than the state-of-the-art Grobner bases-based solvers as well as existing sparse resultant-based solvers, especially in close to critical configurations. Our new method can be fully automated and incorporated into existing tools for the automatic generation of efficient minimal solvers.

7 citations


Cites background or methods from "Using Sparse Elimination for Solvin..."

  • ...based algorithms [24], [26] that can generate smaller and stable solvers for general polynomial systems....

    [...]

  • ...Additionally, we propose several improvements to the previously published sparse resultant method [24]....

    [...]

  • ...Existing resultant-based solvers are mostly handcrafted and tailored to a particular problem, are not exploiting a sparsity of the systems [8] or can not be directly applied to general minimal problems [24]....

    [...]

  • ...Another sparse hidden variable resultant based approach has been proposed by Heikkilä in [24]....

    [...]

  • ...Next we list shortcomings of the previous methods [24]–[26]:...

    [...]

Dissertation
01 Jan 2018
TL;DR: New convex relaxations for rank-based optimization which avoid drawbacks of previous approaches and provide tighter relaxations are presented.
Abstract: Robust fitting of geometric models is a core problem in computer vision. The most common approach is to use a hypothesize-and-test framework, such as RANSAC. In these frameworks the model is estimated from as few measurements as possible, which minimizes the risk of selecting corrupted measurements. These estimation problems are called minimal problems, and they can often be formulated as systems of polynomial equations. In this thesis we present new methods for building so-called minimal solvers or polynomial solvers, which are specialized code for solving such systems. On several minimal problems we improve on the state-of-the-art both with respect to numerical stability and execution time.In many computer vision problems low rank matrices naturally occur. The rank can serve as a measure of model complexity and typically a low rank is desired. Optimization problems containing rank penalties or constraints are in general difficult. Recently convex relaxations, such as the nuclear norm, have been used to make these problems tractable. In this thesis we present new convex relaxations for rank-based optimization which avoid drawbacks of previous approaches and provide tighter relaxations. We evaluate our methods on a number of real and synthetic datasets and show state-of-the-art results. (Less)

6 citations


Cites methods from "Using Sparse Elimination for Solvin..."

  • ...This was recently extended by Heikkila [86] using techniques for constructing sparse resultants [205, 56]....

    [...]

References
More filters
Journal ArticleDOI
TL;DR: A generalization of the Gröbner basis method for polynomial equation solving, which improves overall numerical stability and is shown how the action matrix can be computed in the general setting of an arbitrary linear basis for ℂ[x]/I.
Abstract: This paper presents several new results on techniques for solving systems of polynomial equations in computer vision. Grobner basis techniques for equation solving have been applied successfully to several geometric computer vision problems. However, in many cases these methods are plagued by numerical problems. In this paper we derive a generalization of the Grobner basis method for polynomial equation solving, which improves overall numerical stability. We show how the action matrix can be computed in the general setting of an arbitrary linear basis for ?[x]/I. In particular, two improvements on the stability of the computations are made by studying how the linear basis for ?[x]/I should be selected. The first of these strategies utilizes QR factorization with column pivoting and the second is based on singular value decomposition (SVD). Moreover, it is shown how to improve stability further by an adaptive scheme for truncation of the Grobner basis. These new techniques are studied on some of the latest reported uses of Grobner basis methods in computer vision and we demonstrate dramatically improved numerical stability making it possible to solve a larger class of problems than previously possible.

88 citations


"Using Sparse Elimination for Solvin..." refers methods in this paper

  • ...[3] have proposed a generalization of the Gröbner basis method for improving the numerical stability....

    [...]

Proceedings ArticleDOI
17 Jun 2007
TL;DR: This paper provides a solution to the problem of estimating radial distortion and epipolar geometry from eight correspondences in two images, and shows how to construct the action matrix without computing complete Grobner basis, which provides an efficient and robust solver.
Abstract: Epipolar geometry and relative camera pose computation are examples of tasks which can be formulated as minimal problems and solved from a minimal number of image points. Finding the solution leads to solving systems of algebraic equations. Often, these systems are not trivial and therefore special algorithms have to be designed to achieve numerical robustness and computational efficiency. In this paper we provide a solution to the problem of estimating radial distortion and epipolar geometry from eight correspondences in two images. Unlike previous algorithms, which were able to solve the problem from nine correspondences only, we enforce the determinant of the fundamental matrix be zero. This leads to a system of eight quadratic and one cubic equation in nine variables. We simplify this system by eliminating six of these variables. Then, we solve the system by finding eigenvectors of an action matrix of a suitably chosen polynomial. We show how to construct the action matrix without computing complete Grobner basis, which provides an efficient and robust solver. The quality of the solver is demonstrated on synthetic and real data.

84 citations


"Using Sparse Elimination for Solvin..." refers background in this paper

  • ...Kukelova and Pajdla [15] used an additional constraint to solve the same problem from 8 point correspondences, and Jiang et al....

    [...]

  • ...Kukelova and Pajdla [15] used an additional constraint to solve the same problem from 8 point correspondences, and Jiang et al. [11] added still one constraint and they were able to solve the problem from 7 point correspondences....

    [...]

Journal ArticleDOI
TL;DR: This work provides a characterization of problems that can be efficiently solved as polynomial eigenvalue problems (PEPs) and presents a resultant-based method for transforming a system of poynomial equations to a polynometric eigen value problem.
Abstract: We present a method for solving systems of polynomial equations appearing in computer vision. This method is based on polynomial eigenvalue solvers and is more straightforward and easier to implement than the state-of-the-art Grobner basis method since eigenvalue problems are well studied, easy to understand, and efficient and robust algorithms for solving these problems are available. We provide a characterization of problems that can be efficiently solved as polynomial eigenvalue problems (PEPs) and present a resultant-based method for transforming a system of polynomial equations to a polynomial eigenvalue problem. We propose techniques that can be used to reduce the size of the computed polynomial eigenvalue problems. To show the applicability of the proposed polynomial eigenvalue method, we present the polynomial eigenvalue solutions to several important minimal relative pose problems.

82 citations


"Using Sparse Elimination for Solvin..." refers background or methods in this paper

  • ...In contrast to the modified resultant-based method [13] our algorithm can exploit the sparsity of the polynomials that is a common property in real-world problems....

    [...]

  • ...Notice that this is a looser condition than in [13] where they assumed that either Cl or C0 must be of full rank and invertible....

    [...]

  • ...We compared our method to the modified resultant-based method [13] which generates ( n+d−1...

    [...]

  • ...The trick emplyed in [13] is to generate new equations by multiplying the initial equations with monomials produced by their algorithm....

    [...]

  • ...In [13] they used the classical Macaulay resultant formulation for creating the set of basis monomials...

    [...]

Proceedings ArticleDOI
17 Jun 2007
TL;DR: New 2 and 3 point solutions for the homography induced by a rotation with 1 and 2 unknown focal length parameters are presented and shown to outperform the standard 4 point linear homography solution in terms of accuracy of focal length estimation and image based projection errors.
Abstract: This paper presents minimal solutions for the geometric parameters of a camera rotating about its optical centre. In particular we present new 2 and 3 point solutions for the homography induced by a rotation with 1 and 2 unknown focal length parameters. Using tests on real data, we show that these algorithms outperform the standard 4 point linear homography solution in terms of accuracy of focal length estimation and image based projection errors.

81 citations


"Using Sparse Elimination for Solvin..." refers background in this paper

  • ...Minimal solutions to panorama stitching in [1] and [2] assume that the camera centers coincide which reduces the motion to pure rotation....

    [...]

Frequently Asked Questions (18)
Q1. What was used to compute the homographies?

SURF keypoints and descriptors were used to find correspondences between the image pairs, and RANSAC was employed to compute the homographies. 

In this paper, the authors propose a new algorithm for selecting the basis that is in general more compact than the basis obtained with a state-of-the-art algorithm making PEP a more viable option for solving polynomial equations. Another contribution is that the authors present two minimal problems for camera self-calibration based on homography, and demonstrate experimentally using synthetic and real data that their algorithm can provide a numerically stable solution to the camera focal length from two homographies of unknown planar scene. 

The solver was initialized by assuming that the reference view is fronto-parallel when it becomes easy to compute an initial value for the focal length. 

Five constraints are needed for the camera pose (3 for rotation and 2 for translation up to scale), the normal of the plane n requires two constraints and oneis needed for the perspective scaling factor. 

A resultant-based algorithm for transforming a system of polynomial equations to a polynomial eigenvalue problem (PEP) was proposed in [13] that enabled solving several minimal relative pose problems using linear algebra. 

The main disadvantage of the resultant-based approach for solving the polynomial equations is that it requires computing the determinant of a matrix which often has high dimensions. 

Because the determinant of an N ×N matrix has N ! terms, solving the unknowns from the resultant often becomes computationally infeasible even for relatively small problems. 

1. Since d = 8 and n = 3 in the first calibration problem (equal focal length) the authors get 45 basis monomial which is more than twice the number of the monomials produced by their method. 

In addition to the Gröbner basis techniques and resultants, systems of polynomial equations can be often solved using eigenvalues and eigenvectors. 

Polynomial eigenvalue problem (PEP) is an extension of the standard eigenvalue problem (C−λI)v = 0 to a system of polynomials represented with the matrix equation:(C0 +C1λ+C2λ 2 + · · ·+Clλ l)v = 0, (1)where l is the highest degree of the polynomials in the variable λ that the authors want to solve, v is a vector of monomials in other variables than λ, and C0, . . . 

The total number of new equations ∑i |Ei| is greater or equal to the number of the basis monomials, which means that the coefficient matrices have at least as many rows as columns. 

One approach is to convert the classical multipolynomial resultant to a standard eigenvalue problem [5],[4] which however works only with dense polynomials. 

There are now 3 + k unknowns λ0, . . . , λk, nx and ny , and two equations which means that the authors cannot solve the problem from a single homography, but the authors need at least two homographies (i = 1, 2) that lead to four equations with four unknowns. 

Instead the authors selected the equations randomly and used their SVD based elimination scheme to remove 11 zero monomials that helped to improve the stability of the method. 

An alternative approach that is also commonly used for solving polynomial equations is multipolynomial resultant, which provides an efficient tool for eliminating variables from multiple equations, and solving the remaining variable as a root of a univariate polynomial [4]. 

Another limitation of this approach is that implementing a Gröbner basis solver for a given problem requires expertise in algebraic geometry because the solver needs to be handcrafted in practice to make it efficient. 

One possible reason for this is that in the second problem the solver uses 4 constraints in contrast to the 3 constraints of the first problem. 

The authors did not implement the modified resultant-based method, because in this case d = 17 and n = 4 that would give us 1140 monomials which is almost 6 times more than with their method, and it is clear that the results would be inferior.