scispace - formally typeset

Proceedings ArticleDOI

Using Sparse Elimination for Solving Minimal Problems in Computer Vision

01 Oct 2017-pp 76-84

TL;DR: A new algorithm is proposed for selecting the basis that is in general more compact than the basis obtained with a state-of-the-art algorithm making PEP a more viable option for solving polynomial equations.
Abstract: Finding a closed form solution to a system of polynomial equations is a common problem in computer vision as well as in many other areas of engineering and science. Grob-ner basis techniques are often employed to provide the solution, but implementing an efficient Grobner basis solver to a given problem requires strong expertise in algebraic geometry. One can also convert the equations to a polynomial eigenvalue problem (PEP) and solve it using linear algebra, which is a more accessible approach for those who are not so familiar with algebraic geometry. In previous works PEP has been successfully applied for solving some relative pose problems in computer vision, but its wider exploitation is limited by the problem of finding a compact monomial basis. In this paper, we propose a new algorithm for selecting the basis that is in general more compact than the basis obtained with a state-of-the-art algorithm making PEP a more viable option for solving polynomial equations. Another contribution is that we present two minimal problems for camera self-calibration based on homography, and demonstrate experimentally using synthetic and real data that our algorithm can provide a numerically stable solution to the camera focal length from two homographies of unknown planar scene.
Topics: Gröbner basis (61%), System of polynomial equations (59%), Polynomial (59%), Monomial basis (57%), Solver (56%)

Content maybe subject to copyright    Report

Using Sparse Elimination for Solving Minimal Problems in Computer Vision
Janne Heikkilä
Center for Machine Vision and Signal Analysis
University of Oulu, Finland
janne.heikkila@oulu.fi
Abstract
Finding a closed form solution to a system of polynomial
equations is a common problem in computer vision as well
as in many other areas of engineering and science. Gröb-
ner basis techniques are often employed to provide the so-
lution, but implementing an efficient Gröbner basis solver
to a given problem requires strong expertise in algebraic
geometry. One can also convert the equations to a poly-
nomial eigenvalue problem (PEP) and solve it using linear
algebra, which is a more accessible approach for those who
are not so familiar with algebraic geometry. In previous
works PEP has been successfully applied for solving some
relative pose problems in computer vision, but its wider ex-
ploitation is limited by the problem of finding a compact
monomial basis. In this paper, we propose a new algorithm
for selecting the basis that is in general more compact than
the basis obtained with a state-of-the-art algorithm making
PEP a more viable option for solving polynomial equations.
Another contribution is that we present two minimal prob-
lems for camera self-calibration based on homography, and
demonstrate experimentally using synthetic and real data
that our algorithm can provide a numerically stable solu-
tion to the camera focal length from two homographies of
unknown planar scene.
1. Introduction
Many camera pose estimation and calibration problems boil
down to solving a system of polynomial equations. These
are often so-called minimal problems, where the camera
parameters are computed from a minimal number of con-
straints so that there are essentially as many unknowns as
equations, but the relationship between the unknown vari-
ables and the measurements follows a polynomial model
that makes the dependence nonlinear and difficult to solve
by means of linear algebra. Such minimal problems in-
clude for example, the classical P3P (Perspective-Three-
Point) problem for a calibrated camera where an image of
three points with known distances is sufficient to compute
the camera pose, but it requires solving a system of three
quadratic equations in three variables [
9]. Another clas-
sical example is the five-point problem that allows find-
ing the relative pose between two views from an unknown
scene using five point correspondences. Nistér [
17] con-
verted the resulting system of polynomial equations to a
tenth degree univariate polynomial that can be efficiently
solved using standard numerical techniques. The relative
pose problem has been modified in various studies to in-
corporate also unknown camera parameters that enable the
use of uncalibrated cameras and makes it a self-calibration
problem. To mention few of them, Stewenius et al. [
20]
used six point correspondences to solve the relative pose
together with the focal length. Fitzgibbon [
8] augmented
the fundamental matrix estimation to include one term of
radial lens distortion, and solved them from 9 point corre-
spondences. Kukelova and Pajdla [
15] used an additional
constraint to solve the same problem from 8 point corre-
spondences, and Jiang et al. [
11] added still one constraint
and they were able to solve the problem from 7 point corre-
spondences. A comprehensive list of minimal problems in
computer vision and related papers can be found in [
18].
Planar objects are commonly used for estimating the
camera pose and intrinsic parameters. Well-known Zhang’s
calibration method [23] provides a closed form solution to
the calibration problem from images of a known planar tar-
get. Also, OpenCV and Matlab include tools to perform cal-
ibration with a similar setup. Despite of the extensive num-
ber of minimal problems introduced in recent years, it is
surprising that homography has not been much considered
in this context, and there are only few related works. Min-
imal solutions to panorama stitching in [
1] and [2] assume
that the camera centers coincide which reduces the motion
to pure rotation. Methods for decomposing a homography
into rotation, translation, and surface normal parameters
have been proposed e.g. in [
7] and [24]. Saurer et al. [19]
consider a minimal solution to a 3-point plus a common di-
rection relative pose problem using homography. Recently,
Kukelova et al. [
14] have presented two algorithms for esti-
mating the homography between two cameras with different
radial distortions. However, none of these works address the
76

problem of solving the camera focal length from images of
an unknown planar target, which is the homography-based
minimal problem presented in this paper.
The most common approach for solving minimal prob-
lems in computer vision and corresponding systems of poly-
nomial equations is to use Gröbner basis techniques. One
drawback of this approach is that when the polynomial de-
grees are high, it often suffers from numerical inaccuracies.
To address this problem, for example, Byröd et al. [
3] have
proposed a generalization of the Gröbner basis method for
improving the numerical stability. Another limitation of this
approach is that implementing a Gröbner basis solver for a
given problem requires expertise in algebraic geometry be-
cause the solver needs to be handcrafted in practice to make
it efficient. Because of the complicated theory this approach
is often beyond the reach of non-experts. An alternative ap-
proach that is also commonly used for solving polynomial
equations is multipolynomial resultant, which provides an
efficient tool for eliminating variables from multiple equa-
tions, and solving the remaining variable as a root of a uni-
variate polynomial [
4]. However, there are some limita-
tions that make resultants less useful for engineering ap-
plications. For classical multipolynomial resultants such as
the Macaulay resultant, most of the polynomial coefficients
need to be non-zero, the roots distinct and there should be
no solutions at infinity. This problem can often be avoided
by using a sparse resultant [
6] that works also for polynomi-
als with several zero coefficients. Another disadvantage is
that after elimination the remaining univariate polynomial
is a determinant of a matrix that often has high dimensions.
Because a determinant of an N × N matrix has N! terms,
finding the roots of the remaining polynomial can easily be-
come computationally infeasible or unstable.
In addition to the Gröbner basis techniques and resul-
tants, systems of polynomial equations can be often solved
using eigenvalues and eigenvectors. One approach is to
convert the classical multipolynomial resultant to a stan-
dard eigenvalue problem [
5],[4] which however works only
with dense polynomials. In [
8] the minimal problem of
computing the radial distortion coefficient was expressed
as a quadratic polynomial eigenvalue problem and later it
was extended in [
16] to include an additional constraint.
A resultant-based algorithm for transforming a system of
polynomial equations to a polynomial eigenvalue problem
(PEP) was proposed in [
13] that enabled solving several
minimal relative pose problems using linear algebra. How-
ever, this algorithm has an inherent property of leading to
unnecessarily high dimensional vector spaces and spurious
roots that make the algorithm numerically unstable when
solving sparse systems of polynomials with high degrees.
To overcome the problems related to the algorithm pre-
sented in [13], we propose a new algorithm based on sparse
elimination theory that provides more stable solutions to
sparse systems that are the most typical cases in practical
applications. In addition, we demonstrate the applicablility
of our algorithm in two new minimal problems that have
higher number of solutions than typical relative pose prob-
lems previously presented in the literature.
2. Polynomial eigenvalue problems
Polynomial eigenvalue problem (PEP) is an extension of the
standard eigenvalue problem (C λI)v = 0 to a system of
polynomials represented with the matrix equation:
(C
0
+ C
1
λ + C
2
λ
2
+ · · · + C
l
λ
l
)v = 0, (1)
where l is the highest degree of the polynomials in the vari-
able λ that we want to solve, v is a vector of monomials in
other variables than λ, and C
0
, . . . , C
l
are m × m square
matrices that contain the coefficients of the polynomials.
This equation can be converted to a generalized eigenvalue
problem
Au = λBu, (2)
where
A =
0 I 0 · · · 0
0 0 I · · · 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
C
0
C
1
C
2
· · · C
l1
,
B =
I 0 0 · · · 0
0 I 0 · · · 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 0 · · · C
l
, u =
v
λv
.
.
.
λ
l1
v
.
Since most of the mathematical software libraries and pack-
ages can solve this problem it becomes easy to find all the
roots for λ. The eigenvector u contains the solutions of the
monomials that exist in the polynomials, and therefore, one
can extract the roots of the remaining variables by comput-
ing suitable ratios between individual elements of u.
The most difficult part of converting the system of poly-
nomials to PEP is to determine the monomials in v and
consequently the matrices C
0
, . . . , C
l
. Given n polynomi-
als one can easily construct n × m matrices and the cor-
responding v satisfying (
1), but usually n < m which re-
sults in an underdetermined system of linear equations that
cannot be solved. The trick emplyed in [
13] is to gener-
ate new equations by multiplying the initial equations with
monomials produced by their algorithm. Some of these new
equations may be linearly independent from the initial equa-
tions, which then enables constructing a fully determined
system. Notice that this procedure also increases the num-
ber of monomials in v and hence the dimensions of the co-
efficient matrices. In [
13] they used the classical Macaulay
resultant formulation for creating the set of basis monomials
77

in v. Because the Macaulay resultant is designed for dense
homogeneous polynomials, it is not guaranteed to produce
a basis that is linearly independent. Therefore, they pro-
posed a small modification to the resultant-based approach
that gives a higher number of polynomial equations that in-
creases the chances to get a linearly independent set of ba-
sis monomials. However, for larger systems of polynomials
the basis generated by this method becomes huge, because
the Macaulay resultant is based on the assumption that the
number of the solutions obtained is maximal for the given
degrees of the polynomials. According to Bézout’s theo-
rem [
4] the maximum number of solutions is d
1
· d
2
· · · d
n
,
where d
i
is the degree of the polynomial f
i
. The set of ba-
sis monomials contain all the monomials with total degree
of d =
P
(d
i
1) + 1. One can easily see that the num-
ber of monomials increases exponentially. Therefore, this
approach is feasible only for small systems of equations
and low polynomial degrees. Next, we present a method
based on sparse elimination that exploits the sparsity of the
general polynomials, and produces smaller monomial bases
and coefficient matrices enabling solutions to problems that
have been previously intractable.
3. Determining basis monomials
Most of the polynomial equations encountered in computer
vision are sparse, and therefore classical multivariate resul-
tants are not well-suited for generating the basis monomi-
als. Sparse elimination theory [
21],[5] has been developed
to deal with general polynomials that have many zero co-
efficients. The benefit of the sparsity is that the resultants
obtained have much smaller dimensions than the classical
resultants. Therefore, instead of selecting all the monomials
of a certain total degree we can get a significantly smaller
set of monomials by using the tools provided by the sparse
elimination theory.
Let x = {x
1
, x
2
, . . . , x
n
} be a set of unknown variables
that we want to solve from n multivariate polynomials
f
1
(x) = f
2
(x) = · · · = f
n
(x) = 0 (3)
defined by
f
i
(x) =
s
i
X
j=1
c
ij
x
a
ij
, (4)
where x
a
ij
= x
α
ij1
1
x
α
ij2
2
· · · x
α
ijn
n
are the monomi-
als corresponding to the non-zero coefficients c
ij
. Let
A
i
= {a
i1
, . . . , a
is
i
} Z
n
+
denote the exponent vectors of
all the monomials in f
i
that is also called the support of f
i
.
Next we introduce few concepts from algebraic geometry
[
4] that are needed to formulate our method.
Definition 1: The Newton polytope of f
i
is the convex hull
of the support A
i
denoted by P
i
= Conv(A
i
) R
n
. The
volume of P
i
is denoted by Vol
n
(P
i
).
Notice that in the low dimensional cases when
n = 1, 2 or 3, the Newton polytope represents a line,
polygon or polyhedron, respectively. Clearly, the way
how the volume Vol
n
(P
i
) is computed depends on n. For
example, Vol
1
(P
i
) is the length of the line, and Vol
2
(P
i
) is
the area of the polygon.
Definition 2: The Minkowski sum of two convex polytopes
P
i
and P
j
is the convex polytope
P
ij
= P
i
+ P
j
= {p
i
+ p
j
|p
i
P
i
, p
j
P
j
} R
n
.
Using the Minkowski sum (also known as dilation) we can
aggregate the Newton polytopes of individual polynomials
f
i
to form combined supports. It is also needed for defining
the mixed volume.
Definition 3: Given convex polytopes P
1
, . . . , P
n
R
n
there is a real-valued function called mixed volume that can
be computed as
MV
n
(P
1
, . . . , P
n
) =
n
X
k=1
(1)
nk
X
I⊂{1,...,n}
|I|=k
Vol
n
X
iI
P
i
!
.
(5)
In high-dimensional cases computing the mixed volume us-
ing (
5) can be time consuming. There are faster algorithms
that use a so called mixed subdivision of the Minkowski
sum, and one can also find their software implementations
from the Internet, but in the cases discussed in this paper
n 4 and using (
5) is still tractable. The following theorem
is the reason why we introduced the mixed volume.
Theorem 1 (Bernstein’s Theorem): Given the polynomi-
als f
1
, . . . f
n
over C with finitely many common zeroes in
(C
)
n
, where C
= C \ {0}, let P
i
be the Newton polytope
of f
i
in R
n
. Then the number of solutions of the f
i
in (C
)
n
is bounded above by the mixed volume MV
n
(P
1
, . . . , P
n
).
For generic choices of the coefficients c
ij
the number of
common solutions is exactly MV
n
(P
1
, . . . , P
n
).
The proof of the theorem can be found from [
4]. Bern-
stein’s theorem is an important result of the sparse elimina-
tion theory that gives us a tool for calculating the maximum
number of the roots in advance without knowing the numer-
ical values of the coefficients. What we need is only the ex-
ponent vectors a
ij
of the monomials, i.e., supports A
i
. This
also determines the minimum size of the monomial basis as
we will see later.
Next we discuss about finding the basis monomials for
the polynomial eigenvalue problem, i.e., the elements of
v in (
1). Sparse elimination provides the tools for con-
structing sparse resultants that generalize the classical mul-
tivariate resultant. While the degree of the classical mul-
tivariate resultant is determined by Bézout’s theorem (i.e.
d
1
·d
2
· · · d
n
), the degree of the sparse resultant comes from
Bernstein’s theorem which is the mixed volume. These two
types of resultants coincide only when all Newton polytopes
are n-simplices scaled by the total degree of the respec-
tive polynomials [
4]. Otherwise the degree of the sparse
78

resultant is smaller, which also means that the matrix con-
structed from the coefficients c
ij
has smaller dimensions.
Furthermore, it is necessary that the matrix has full rank
and its determinant vanishes only when the equations have
a common solution. It often happens that the multivariate
resultant is rank-deficient if the polynomials have zero co-
efficients, and thus it fails to provide a solution. In order
to have a full rank, it becomes necessary to select the basis
monomials of the sparse resultant carefully using, e.g., the
Lift-Prune algorithm proposed by Emiris & Canny [
6]. The
main disadvantage of the resultant-based approach for solv-
ing the polynomial equations is that it requires computing
the determinant of a matrix which often has high dimen-
sions. Because the determinant of an N × N matrix has N !
terms, solving the unknowns from the resultant often be-
comes computationally infeasible even for relatively small
problems. For example, if the dimension of the coefficient
matrix for the sparse resultant is 10 × 10, the resultant is a
factor of an expression that has more than 3.6 million terms.
In such cases finding the solution via PEP is much more ef-
ficient.
A sparse resultant to a system of n equations is com-
puted in n 1 variables, which means that one of the vari-
ables of our original problem (
3) needs to be hidden to the
coefficient field. The resultant obtained is then a univariate
polynomial of the hidden variable which can be solved by
finding the roots of this polynomial. Without loss of gener-
ality we can decide to solve the first variable x
1
that is then
treated as a coefficient in the polynomials
f
i
(
˜
x) =
X
s
i
j=1
c
ij
˜
x
a
ij
, (6)
where c
ij
=
P
α
ij1
c
ij
x
α
ij1
1
,
˜
x = {x
2
, . . . , x
n
} and
a
ij
= (α
ij2
, . . . , α
ijn
) A
i
are n 1 dimensional sup-
port vectors with |A
i
| = s
i
. The first step is to create the
Newton polytopes P
1
, . . . , P
n
R
n1
corresponding to
the modified system (
6), and compute the Minkowski sum
P = P
1
+ · · · + P
n
. The set of basis monomials S for the
sparse resultant is then obtained from
S = Z
n1
(P + d), (7)
where Z
n1
defines a square lattice with integer points, and
d R
n1
is a small translation vector that displaces P
slightly so that the lattice points lie in the interiors of the
convex polytope [
6, 4]. In practice, the elements of d can be
randomly selected from {−ǫ, 0, ǫ} where ǫ Q is a small
rational number.
Sparse resultants need to be of full rank in order to have
a non-zero determinant. In our case, the only strict require-
ment is that C
0
, . . . , C
l
in (1) must be square matrices. No-
tice that this is a looser condition than in [
13] where they
assumed that either C
l
or C
0
must be of full rank and in-
vertible. Here this not necessary, but in some cases rank-
deficiency may lead to numerical instability with the eigen-
value solver. Because the PEP in (
1) is defined for one un-
known variable λ which is then computed as an eigenvalue
of (
2) we need to choose this variable from x
1
, . . . , x
n
. This
is exactly the same situation as with the sparse resultant, and
therefore, we decide again without loss of generality that
λ x
1
, and we hide x
1
to the coefficient field which then
results in the modified system (
6).
Due to the relaxed requirements, we can try to find
a smaller set of basis monomials than (
7) defined for
the sparse resultant. The lower bound is determined by
Bernstein’s theorem which gives the maximum number of
the common roots for the polynomials denoted by r
MV
n
(P
1
, . . . , P
n
). It should be noticed that the mixed vol-
ume is computed for the original system (
3). The eigen-
vector u in (
2) has the same dimension as the maximum
number of unique eigenvalues i.e. possible roots of the sys-
tem. The length of u is clearly lm which gives us the bound
m
r
l
. (8)
Hence, it is sufficient to find a set of support vectors B for
the basis monomials where |B| r/l.
Algorithm
1 summarizes the procedure for constructing
B based on the previous discussion. It generates several
putative sets of support vectors for the basis and selects
the smallest set among these candidates. It also returns a
set T = {T
1
, . . . , T
n
} where T
i
6= are subsets of vec-
tors that can be used to construct the coefficient matrices
C
0
, . . . , C
l
. These vectors are first converted to n sets of
monomials M
i
= {
˜
x
t
| t T
i
}, and the monomials are
multiplied with the original equations f
i
which then results
in n sets of new equations E
i
= {
˜
x
t
f
i
(x)|
˜
x
t
M
i
}.
These equations are converted to a matrix form (
1), which
then directly gives us the coefficient matrices C
0
, . . . , C
l
.
The total number of new equations
P
i
|E
i
| is greater or
equal to the number of the basis monomials, which means
that the coefficient matrices have at least as many rows as
columns. If there are more rows than columns, one can
choose m rows that minimize the condition number, and
discard the remaining rows. It may also happen that the
most compact basis does not work, and in that case one
could try the next candidate produced by the algorithm.
There are often monomials (or vectors) in B that do not
contribute to the equations. Such monomials may cause in-
stability to the eigenvalue solver, and they need to be re-
moved. In [
13] they call them “parasitic” zero eigenval-
ues, and they propose to convert the generalized eigenvalue
problem to a standard eigenvalue problem so that one can
easily identify and remove these monomials as they corre-
spond to zero columns of the matrix to be decomposed. The
drawback is that either C
0
or C
l
need to be of full rank,
which then causes extra constraints to selection of the basis
monomials. Hence, we propose here a simple strategy for
finding these zero monomials. First, we need to specialize
the coefficient matrices with some random numerical val-
79

Algorithm 1 Generate basis monomials
Input: A
1
, . . . , A
n
, r, l
Output: B, T
1: Create Newton polytopes P
i
Conv(A
i
) R
n1
for
i = 1, . . . , n, and a unit (n 1)-simplex P
0
R
n1
.
2: Create a list of index sets:
K [{k
0
, . . . , k
i
} | i = 0, . . . , n ; k
0
, . . . , k
i
{0, . . . , n} ; k
j+1
> k
j
].
3: Create a list of displacement vectors:
[(δ
1
, . . . , δ
n1
) | δ
1
, . . . , δ
n1
{−ǫ, 0, ǫ}].
4: Initialize B , T and N .
5: for I in K do
6: Compute Minkowski sum Q
P
kI
P
k
.
7: for d in do
8: Create a putative basis B Z
n1
(Q + d).
9: if |B|
r
l
AND |B| < N then
10: Find the sets of vectors:
T
i
{t|t Z
n1
+
, A
i
+ t B}
for i = 1, . . . , n.
11: if
P
i
|T
i
| |B| AND min(|T
i
|) > 0 then
12: B B, T {T
i
}
i=1,...,n
, and
N |B|.
13: end if
14: end if
15: end for
16: end for
ues as c
ij
. Using these values we construct the matrices A
and B in (
2), and compute the singular value decomposition
B = USV
. Next we perform a unitary transformation
A
= U
AV, (9)
and find the zero columns of A
. These columns correspond
to the zero monomials and they can be removed from A and
B. The rows with the same indices are also removed so that
the matrices will remain square. This procedure might need
to be repeated few times to find all zero monomials. How-
ever, one should notice that the elimination is performed
offline when designing the solver, and there is no need to do
it runtime once the zero polynomials have been identified.
4. Planar self-calibration
A standard approach for geometric camera calibration is to
use a known checker board pattern printed on a planar sur-
face. To demonstrate the applicability of our algorithm, we
present two minimal problems for solving the camera focal
length from two homographies corresponding to three im-
ages where the patterns are unknown, which makes this a
self-calibration problem. We consider the following cases:
1) a constant focal length and 2) two different focal lengths.
The resulting polynomials can be converted to PEPs using
Algorithm
1 and solved efficiently, for example, with Mat-
lab or some other software package or library capable of
computing generalized eigenvalues.
Let two 3D vectors a and b span a plane so that they are
both orthogonal and of equal length fulfilling the constraints
a
b = 0 and a
a b
b = 0. (10)
If |a| = |b| = 1 the normal vector of the plane is defined
by n = a × b. In order to express the vectors a and b in
terms of the normal vector n we can choose
a = n × e and b = n × a, (11)
where e is a unit vector not parallel to n. For simplic-
ity, we select e = [1, 0, 0]
. We use parametrization
n = [n
x
, n
y
, 1]
/
q
n
2
x
+ n
2
y
+ 1, and we can now express
a and b in variables n
x
and n
y
. Further assuming that a and
b are represented in the camera coordinate frame of the ref-
erence view we can convert them to the image coordinates
using
ˆ
a
0
= K
0
a and
ˆ
b
0
= K
0
b, (12)
where K
0
is the intrinsic camera matrix for the reference
camera. Because one can often assume with reasonable ac-
curacy that the principal point of the camera is in the center
of the image, the pixel aspect ratio is 1, and lens distortion
is negligible, we limit ourselves to the case where we have
only one intrinsic parameter, the focal length λ
0
, that leads
to the camera matrix K
0
= diag(λ
0
, λ
0
, 1).
The mapping from the reference image to the i
th
image is
described by the homography H
i
. After back-projecting to
the 3D space the corresponding vectors are obtained from
a
i
= K
1
i
H
i
K
0
a and b
i
= K
1
i
H
i
K
0
b, (13)
where K
i
= d i ag(λ
i
, λ
i
, 1) and λ
i
is the focal length of
the i
th
camera. Because orthogonality and equality in the
length should hold in each frame we have the following self-
calibration constraints expressed by two polynomials
f
1,i
(λ
0
, λ
i
, n
x
, n
y
) = a
i
b
i
= 0 (14)
f
2,i
(λ
0
, λ
i
, n
x
, n
y
) = a
i
a
i
b
i
b
i
= 0. (15)
Herrera et al. [
10] used similar constraints for planar
self-calibration but they solved the camera parameters with
non-linear minimization. The solver was initialized by as-
suming that the reference view is fronto-parallel when it be-
comes easy to compute an initial value for the focal length.
In this paper, we do not make such assumptions, and no
initialization is needed. There are now 3 + k unknowns
λ
0
, . . . , λ
k
, n
x
and n
y
, and two equations which means
that we cannot solve the problem from a single homogra-
phy, but we need at least two homographies (i = 1, 2) that
lead to four equations with four unknowns. This is true also
in general, because one homography provides only 8 con-
straints to calibration. Five constraints are needed for the
camera pose (3 for rotation and 2 for translation up to scale),
the normal of the plane n requires two constraints and one
80

Citations
More filters

Journal ArticleDOI
Ji Zhao, Laurent Kneip1, Yijia He2, Jiayi Ma3Institutions (3)
TL;DR: It is shown that knowing the value of the inscribed angle between the two 3D rays poses additional constraints on the relative orientation, and using the latter enables the solution of the relative pose problem with as few as 3 correspondences across the two images.
Abstract: Corners are popular features for relative pose computation with 2D-2D point correspondences. Stable corners may be formed by two 3D rays sharing a common starting point. We call such elements ray-point-ray (RPR) structures. Besides a local invariant keypoint given by the lines’ intersection, their reprojection also defines a corner orientation and an inscribed angle in the image plane. The present paper investigates such RPR features, and aims at answering the fundamental question of what additional constraints can be formed from correspondences between RPR features in two views. In particular, we show that knowing the value of the inscribed angle between the two 3D rays poses additional constraints on the relative orientation. Using the latter enables the solution of the relative pose problem with as few as 3 correspondences across the two images. We provide a detailed analysis of all minimal cases distinguishing between 90-degree RPR-structures and structures with an arbitrary, known inscribed angle. We furthermore investigate the special cases of a known directional correspondence and planar motion, the latter being solvable with only a single RPR correspondence. We complete the exposition by outlining an image processing technique for robust RPR-feature extraction. Our results suggest high practicality in man-made environments, where 90-degree RPR-structures naturally occur.

15 citations


Cites methods from "Using Sparse Elimination for Solvin..."

  • ...Successful applications of the resultant method in computer vision can be found in [36], [37]....

    [...]


Proceedings ArticleDOI
Yaqing Ding1, Jian Yang1, Jean Ponce2, Hui Kong1Institutions (2)
14 Jun 2020
TL;DR: The proposed algorithms can cope with coplanar points, which is a degenerate configuration for these 6- and 7-point counterparts, and derive new 4- and 5-point algorithms for these two cases, respectively.
Abstract: We propose minimal solutions to relative pose estimation problem from two views sharing a common direction with unknown focal length. This is relevant for cameras equipped with an IMU (inertial measurement unit), e.g., smart phones, tablets. Similar to the 6-point algorithm for two cameras with unknown but equal focal lengths and 7-point algorithm for two cameras with different and unknown focal lengths, we derive new 4- and 5-point algorithms for these two cases, respectively. The proposed algorithms can cope with coplanar points, which is a degenerate configuration for these 6- and 7-point counterparts. We present a detailed analysis and comparisons with the state of the art. Experimental results on both synthetic data and real images from a smart phone demonstrate the usefulness of the proposed algorithms.

7 citations


Cites methods from "Using Sparse Elimination for Solvin..."

  • ...Polynomial eigenvalue methods have been successfully used for many minimal problems in computer vision, such as the 9-point one-parameter radial distortion problem [11], the 5- and 6-point relative pose problems [21], the 6point one unknown focal length problem [4], and the selfcalibration problems [17]....

    [...]


Dissertation
01 Jan 2018
TL;DR: New convex relaxations for rank-based optimization which avoid drawbacks of previous approaches and provide tighter relaxations are presented.
Abstract: Robust fitting of geometric models is a core problem in computer vision. The most common approach is to use a hypothesize-and-test framework, such as RANSAC. In these frameworks the model is estimated from as few measurements as possible, which minimizes the risk of selecting corrupted measurements. These estimation problems are called minimal problems, and they can often be formulated as systems of polynomial equations. In this thesis we present new methods for building so-called minimal solvers or polynomial solvers, which are specialized code for solving such systems. On several minimal problems we improve on the state-of-the-art both with respect to numerical stability and execution time.In many computer vision problems low rank matrices naturally occur. The rank can serve as a measure of model complexity and typically a low rank is desired. Optimization problems containing rank penalties or constraints are in general difficult. Recently convex relaxations, such as the nuclear norm, have been used to make these problems tractable. In this thesis we present new convex relaxations for rank-based optimization which avoid drawbacks of previous approaches and provide tighter relaxations. We evaluate our methods on a number of real and synthetic datasets and show state-of-the-art results. (Less)

6 citations


Cites methods from "Using Sparse Elimination for Solvin..."

  • ...This was recently extended by Heikkila [86] using techniques for constructing sparse resultants [205, 56]....

    [...]


Proceedings ArticleDOI
14 Jun 2020
TL;DR: This paper studies an alternative algebraic method for solving systems of polynomial equations, i.e., the sparse resultant-based method and proposes a novel approach to convert the resultant constraint to an eigenvalue problem, which can significantly improve the efficiency and stability of existing resultant- based solvers.
Abstract: Many computer vision applications require robust and efficient estimation of camera geometry. The robust estimation is usually based on solving camera geometry problems from a minimal number of input data measurements, i.e. solving minimal problems in a RANSAC framework. Minimal problems often result in complex systems of polynomial equations. Many state-of-the-art efficient polynomial solvers to these problems are based on Grobner basis and the action-matrix method that has been automatized and highly optimized in recent years. In this paper we study an alternative algebraic method for solving systems of polynomial equations, i.e., the sparse resultant-based method and propose a novel approach to convert the resultant constraint to an eigenvalue problem. This technique can significantly improve the efficiency and stability of existing resultant-based solvers. We applied our new resultant-based method to a large variety of computer vision problems and show that for most of the considered problems, the new method leads to solvers that are the same size as the the best available Grobner basis solvers and of similar accuracy. For some problems the new sparse-resultant based method leads to even smaller and more stable solvers than the state-of-the-art Grobner basis solvers. Our new method can be fully automatized and incorporated into existing tools for automatic generation of efficient polynomial solvers and as such it represents a competitive alternative to popular Grobner basis methods for minimal problems in computer vision.

5 citations


Cites background or methods from "Using Sparse Elimination for Solvin..."

  • ...The most promising results in this direction were proposed by Emiris [12] and Heikkilä [18], where methods based on sparse resultants were proposed and applied to camera geometry problems....

    [...]

  • ...The most promising results in this direction were proposed by Emiris [12] and Heikkilä [18], where methods based on sparse resultants were proposed and applied to camera geometry problems....

    [...]

  • ...The augmented polynomial system is solved by hiding λ and reducing a constraint similar to (2) into a regular eigenvalue problem that leads to smaller solvers than [12, 18]....

    [...]

  • ...Our algorithm is inspired by the ideas explored in [18, 12], but thanks to the special form of added equation and by solving the resultant as a small eigenvalue problem, in contrast to a polynomial eigenvalue problem in [18], the new approach achieves significant improvements over [18, 12] in terms of efficiency of the generated solvers....

    [...]

  • ...The algorithm by Heikkilä [18] basically computes the Minkowski sum of the Newton polytopes of a subset of input polynomials, Q = ΣiNP(fi(x))....

    [...]


Book ChapterDOI
Danda Pani Paudel1, Luc Van Gool1Institutions (1)
08 Sep 2018
TL;DR: This paper addresses the problem of robustly autocalibrating a moving camera with constant intrinsics by using the Branch-and-Bound (BnB) search paradigm to maximize the consensus of the polynomials.
Abstract: This paper addresses the problem of robustly autocalibrating a moving camera with constant intrinsics. The proposed calibration method uses the Branch-and-Bound (BnB) search paradigm to maximize the consensus of the polynomials. These polynomials are parameterized by the entries of, either the Dual Image of Absolute Conic (DIAC) or the Plane-at-Infinity (PaI). During the BnB search, we exploit the theory of sampling algebraic varieties, to test the positivity of any polynomial within a parameter’s interval, i.e. outliers with certainty. The search process explores the space of exact parameters (i.e. the entries of DIAC or PaI), benefits from the solution of a local method, and converges to the solution satisfied by the largest number of polynomials. Given many polynomials on the sought parameters (with possibly overwhelmingly many from outlier measurements), their consensus for calibration is searched for two cases: simplified Kruppa’s equations and Modulus constraints, expressed in DIAC and PaI, resp. Our approach yields outstanding results in terms of robustness and optimality.

5 citations


References
More filters

Journal ArticleDOI
ZhenQiu Zhang1Institutions (1)
TL;DR: A flexible technique to easily calibrate a camera that only requires the camera to observe a planar pattern shown at a few (at least two) different orientations is proposed and advances 3D computer vision one more step from laboratory environments to real world use.
Abstract: We propose a flexible technique to easily calibrate a camera. It only requires the camera to observe a planar pattern shown at a few (at least two) different orientations. Either the camera or the planar pattern can be freely moved. The motion need not be known. Radial lens distortion is modeled. The proposed procedure consists of a closed-form solution, followed by a nonlinear refinement based on the maximum likelihood criterion. Both computer simulation and real data have been used to test the proposed technique and very good results have been obtained. Compared with classical techniques which use expensive equipment such as two or three orthogonal planes, the proposed technique is easy to use and flexible. It advances 3D computer vision one more step from laboratory environments to real world use.

11,465 citations


"Using Sparse Elimination for Solvin..." refers methods in this paper

  • ...Well-known Zhang’s calibration method [23] provides a closed form solution to the calibration problem from images of a known planar target....

    [...]


Journal ArticleDOI
David Nister1Institutions (1)
TL;DR: The algorithm is used in a robust hypothesize-and-test framework to estimate structure and motion in real-time with low delay and is the first algorithm well-suited for numerical implementation that also corresponds to the inherent complexity of the problem.
Abstract: An efficient algorithmic solution to the classical five-point relative pose problem is presented. The problem is to find the possible solutions for relative camera pose between two calibrated views given five corresponding points. The algorithm consists of computing the coefficients of a tenth degree polynomial in closed form and, subsequently, finding its roots. It is the first algorithm well-suited for numerical implementation that also corresponds to the inherent complexity of the problem. We investigate the numerical precision of the algorithm. We also study its performance under noise in minimal as well as overdetermined cases. The performance is compared to that of the well-known 8 and 7-point methods and a 6-point scheme. The algorithm is used in a robust hypothesize-and-test framework to estimate structure and motion in real-time with low delay. The real-time system uses solely visual input and has been demonstrated at major conferences.

1,834 citations


"Using Sparse Elimination for Solvin..." refers methods in this paper

  • ...Nistér [17] converted the resulting system of polynomial equations to a tenth degree univariate polynomial that can be efficiently solved using standard numerical techniques....

    [...]


Journal ArticleDOI
Olivier Faugeras1, Francis Lustman1Institutions (1)
TL;DR: It is shown that when the environment is piecewise linear, it provides a powerful constraint on the kind of matches that exist between two images of the scene when the camera motion is unknown, and that this constraint can be recovered from an estimate of the matrix of this collineation.
Abstract: We show in this article that when the environment is piecewise linear, it provides a powerful constraint on the kind of matches that exist between two images of the scene when the camera motion is unknown. For points and lines located in the same plane, the correspondence between the two cameras is a collineation. We show that the unknowns (the camera motion and the plane equation) can be recovered, in general, from an estimate of the matrix of this collineation. The two-fold ambiguity that remains can be removed by looking at a second plane, by taking a third view of the same plane, or by using a priori knowledge about the geometry of the plane being looked at. We then show how to combine the estimation of the matrix of collineation and the obtaining of point and line matches between the two images, by a strategy of Hypothesis Prediction and Testing guided by a Kalman filter. We finally show how our approach can be used to calibrate a system of cameras.

545 citations


Proceedings ArticleDOI
Andrew Fitzgibbon1Institutions (1)
01 Dec 2001
TL;DR: This paper shows how linear estimation of the fundamental matrix from two-view point correspondences may be augmented to include one term of radial lens distortion, by expressing fundamental matrix estimation as a quadratic eigenvalue problem (QEP), for which efficient algorithms are well known.
Abstract: A problem in uncalibrated stereo reconstruction is that cameras which deviate from the pinhole model have to be pre-calibrated in order to correct for nonlinear lens distortion. If they are not, and point correspondence is attempted using the uncorrected images, the matching constraints provided by the fundamental matrix must be set so loose that point matching is significantly hampered. This paper shows how linear estimation of the fundamental matrix from two-view point correspondences may be augmented to include one term of radial lens distortion. This is achieved by (1) changing from the standard radial-lens model to another which (as we show) has equivalent power, but which takes a simpler form in homogeneous coordinates, and (2) expressing fundamental matrix estimation as a quadratic eigenvalue problem (QEP), for which efficient algorithms are well known. I derive the new estimator, and compare its performance against bundle-adjusted calibration-grid data. The new estimator is fast enough to be included in a RANSAC-based matching loop, and we show cases of matching being rendered possible by its use. I show how the same lens can be calibrated in a natural scene where the lack of straight lines precludes most previous techniques. The modification when the multi-view relation is a planar homography or trifocal tensor is described.

535 citations


"Using Sparse Elimination for Solvin..." refers background or methods in this paper

  • ...In [8] the minimal problem of computing the radial distortion coefficient was expressed as a quadratic polynomial eigenvalue problem and later it was extended in [16] to include an additional constraint....

    [...]

  • ...Fitzgibbon [8] augmented the fundamental matrix estimation to include one term of radial lens distortion, and solved them from 9 point correspondences....

    [...]


Proceedings ArticleDOI
01 Jun 1991
TL;DR: It is shown that even in cases where the solution is not near the geometric unstable region considerable care must be exercised in the calculation, and an analytical method is presented which produces a numerically stable calculation.
Abstract: The major direct solutions to the three-point perspective pose estimation problems are reviewed from a unified perspective. The numerical stability of these three-point perspective solutions are discussed. It is shown that even in cases where the solution is not near the geometric unstable region considerable care must be exercised in the calculation. Depending on the order of the substitutions utilized, the relative error can change over a thousand to one. This difference is due entirely to the way the calculations are performed and not to any geometric structural instability of any problem instance. An analytical method is presented which produces a numerically stable calculation. >

335 citations


"Using Sparse Elimination for Solvin..." refers background in this paper

  • ...Such minimal problems include for example, the classical P3P (Perspective-ThreePoint) problem for a calibrated camera where an image of three points with known distances is sufficient to compute the camera pose, but it requires solving a system of three quadratic equations in three variables [9]....

    [...]


Performance
Metrics
No. of citations received by the Paper in previous years
YearCitations
20211
20204
20191
20182