scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

A Sparse Resultant Based Method for Efficient Minimal Solvers

TL;DR: This paper studies an alternative algebraic method for solving systems of polynomial equations, i.e., the sparse resultant-based method and proposes a novel approach to convert the resultant constraint to an eigenvalue problem, which can significantly improve the efficiency and stability of existing resultant- based solvers.
Abstract: Many computer vision applications require robust and efficient estimation of camera geometry. The robust estimation is usually based on solving camera geometry problems from a minimal number of input data measurements, i.e. solving minimal problems in a RANSAC framework. Minimal problems often result in complex systems of polynomial equations. Many state-of-the-art efficient polynomial solvers to these problems are based on Grobner basis and the action-matrix method that has been automatized and highly optimized in recent years. In this paper we study an alternative algebraic method for solving systems of polynomial equations, i.e., the sparse resultant-based method and propose a novel approach to convert the resultant constraint to an eigenvalue problem. This technique can significantly improve the efficiency and stability of existing resultant-based solvers. We applied our new resultant-based method to a large variety of computer vision problems and show that for most of the considered problems, the new method leads to solvers that are the same size as the the best available Grobner basis solvers and of similar accuracy. For some problems the new sparse-resultant based method leads to even smaller and more stable solvers than the state-of-the-art Grobner basis solvers. Our new method can be fully automatized and incorporated into existing tools for automatic generation of efficient polynomial solvers and as such it represents a competitive alternative to popular Grobner basis methods for minimal problems in computer vision.

Summary (1 min read)

The meso level of analysis

  • The first aspect I shall refer to is meta-theoretical in nature, and has to do with the balance between the generality and the specificity of claims raised along the way of any academic inquiry.
  • Such middle ground is reached by way of a well-gauged combination of fundamental theorization and empirical analysis attentive to the specificities of the phenomena under consideration.
  • The title of the motion picture is The Silent Chaos (Spanò 2013b), and its director, Antonio Spanò, comments: been changed after an unexpected encounter with some deaf guys in Butembo.
  • In other words, that politics, religion, science, and the like may be said to be media dependent does not rule out the fact that the media themselves are embedded in the former too, which suggests a relation of mutual dependency.
  • That is the case, for example, of Debrix and Barder’s work on violence, horror and spaces of exception (2013).

Contributor details

  • Carlos M. Roos is currently pursuing doctoral research in Communication Sciences at Ghent University, Belgium, and in Philosophy of Art at Leiden University, the Netherlands.
  • He teaches at the Department of Media Communications of Webster University Leiden.
  • His research interests include metaphysics of art, critical theory and comparative philosophy.
  • Contact: E-mail: c.m.roos.munoz@umail.leidenuniv.nl Carlos M. Roos has asserted his right under the Copyright, Designs and Patents Act, 1988, to be identified as the author of this work in the format that was submitted to Intellect Ltd.

Did you find this useful? Give us your feedback

Figures (6)

Content maybe subject to copyright    Report

A sparse resultant based method for efficient minimal solvers
Snehal Bhayani
Center for Machine Vision and Signal Analysis
University of Oulu, Finland
snehal.bhayani@oulu.fi
Zuzana Kukelova
Center for Machine Perception
Czech Technical University, Prague
kukelova@cmp.felk.cvut.cz
Janne Heikkil
¨
a
Center for Machine Vision and Signal Analysis
University of Oulu, Finland
janne.heikkila@oulu.fi
Abstract
Many computer vision applications require robust and effi-
cient estimation of camera geometry. The robust estimation
is usually based on solving camera geometry problems from
a minimal number of input data measurements, i.e. solving
minimal problems in a RANSAC framework. Minimal prob-
lems often result in complex systems of polynomial equa-
tions. Many state-of-the-art efficient polynomial solvers to
these problems are based on Gr
¨
obner bases and the action-
matrix method that has been automatized and highly opti-
mized in recent years. In this paper we study an alternative
algebraic method for solving systems of polynomial equa-
tions, i.e., the sparse resultant-based method and propose
a novel approach to convert the resultant constraint to an
eigenvalue problem. This technique can significantly im-
prove the efficiency and stability of existing resultant-based
solvers. We applied our new resultant-based method to a
large variety of computer vision problems and show that
for most of the considered problems, the new method leads
to solvers that are the same size as the the best available
Gr
¨
obner basis solvers and of similar accuracy. For some
problems the new sparse-resultant based method leads to
even smaller and more stable solvers than the state-of-the-
art Gr
¨
obner basis solvers. Our new method can be fully
automatized and incorporated into existing tools for auto-
matic generation of efficient polynomial solvers and as such
it represents a competitive alternative to popular Gr
¨
obner
basis methods for minimal problems in computer vision.
1. Introduction
Computing camera geometry is one of the most important
tasks in computer vision [16] with many applications e.g.
in structure from motion [38], visual navigation [37], large
scale 3D reconstruction [18] and image localization [36].
The robust estimation of camera geometry is usually
based on solving so-called minimal problems [34, 23, 22],
i.e. problems that are solved from minimal samples of in-
put data, inside a RANSAC framework [13, 8, 35]. Since
the camera geometry estimation has to be performed many
times in RANSAC [13], fast solvers to minimal problems
are of high importance. Minimal problems often result in
complex systems of polynomial equations in several vari-
ables. A popular approach for solving minimal problems is
to design procedures that can efficiently solve only a spe-
cial class of systems of equations, e.g. systems resulting
from the 5-pt relative pose problem [34], and move as much
computation as possible from the “online” stage of solving
equations to an earlier pre-processing “offline” stage.
Most of the state-of-the-art specific minimal solvers are
based on Gr
¨
obner bases and the action-matrix method [9].
The Gr
¨
obner basis method was popularized in computer vi-
sion by Stewenius [39]. The first efficient Gr
¨
obner basis
solvers were mostly handcrafted [40, 41] and sometimes
very unstable [42]. However, in the last 15 years much ef-
fort has been put into making the process of constructing
the solvers more automatic [23, 28, 29] and the solvers sta-
ble [5, 6] and more efficient [28, 29, 27, 4, 31]. There are
now powerful tools available for the automatic generation
of efficient Gr
¨
obner basis solvers [23, 28].
While the Gr
¨
obner basis method for generating efficient
minimal solvers was deeply studied in computer vision and
all recently generated Gr
¨
obner basis solvers are highly op-
timized in terms of efficiency and stability, little attention
has been paid to an alternative algebraic method for solving
systems of polynomial equations, i.e. the resultant-based
method. The resultant-based method was manually applied
to several computer vision problems [24, 15, 15, 19, 22, 24].
However in contrast to the Gr
¨
obner basis method, there is
1
arXiv:1912.10268v1 [cs.CV] 21 Dec 2019

no general method for automatically generating efficient
resultant-based minimal solvers. The most promising re-
sults in this direction were proposed by Emiris [11] and
Heikkil
¨
a [17], where methods based on sparse resultants
were proposed and applied to camera geometry problems.
While these methods can be extended for general minimal
problems that appear in computer vision and can be autom-
atized, they usually lead (due to linearizations) to larger and
less efficient solvers than Gr
¨
obner basis solvers.
In this paper, we propose a novel approach to generat-
ing minimal solvers using sparse resultants, which is based
on adding an extra equation of a special form to the in-
put system. Our algorithm is inspired by the ideas ex-
plored in [17, 11], but thanks to the special form of added
equation and by solving the resultant as a small eigenvalue
problem, in contrast to a polynomial eigenvalue problem
in [17], the new approach achieves significant improve-
ments over [17, 11] in terms of efficiency of the generated
solvers. Specifically our contributions include,
A novel sparse resultant-based approach to generating
polynomial solvers based on adding an extra equation
of a special form and transforming the resultant matrix
constraint to a regular eigenvalue problem.
Two procedures to reduce the size of resultant matrix
that lead to faster solvers than the best available state-
of-the-art solvers for some minimal problems.
A general method for automatic generation of efficient
resultant-based polynomial solvers for many impor-
tant minimal problems that achieves competitive per-
formance in terms of speed and stability with respect
to the best available state-of-the-art solvers generated
by highly optimized Gr
¨
obner basis techniques [28, 31].
The automatic generator of resultant-based solvers will
be made publicly available.
2. Theoretical background and related work
In this paper we use notation and basic concepts from the
book by Cox et al. [9]. Our objective is to solve m polyno-
mial equations,
{f
1
(x
1
, . . . , x
n
) = 0, . . . , f
m
(x
1
, . . . , x
n
) = 0} (1)
in n unknowns, X = {x
1
, . . . , x
n
}. Let C[X] denote the
set of all polynomials in unknowns X with coefficients in C.
The ideal I = hf
1
, . . . , f
m
i C[X] is the set of all poly-
nomial combinations of our generators f
1
, . . . , f
m
. The set
V of all solutions of the system (1) is called the affine va-
riety. Each polynomial f I vanishes on the solutions of
(1). Here we assume that the ideal I generates a zero di-
mensional variety, i.e. the system (1) has a finite number of
solutions. Using the ideal I we can define the quotient ring
A = C[X]/I which is the set of equivalence classes over
C[X] defined by the relation a b (a b) I.
If I has a zero-dimensional variety then the quotient ring
A = C[X]/I is a finite-dimensional vector space over C.
For an ideal I there exist special sets of generators called
Gr
¨
obner bases which have the nice property that the remain-
der after division is unique. Using a Gr
¨
obner basis we can
define a linear basis for the quotient ring A = C[X]/I.
2.1. Gr
¨
obner Basis method
Gr
¨
obner bases can be used to solve our system of polyno-
mial equations (1). One of the popular approaches for solv-
ing systems of equations using Gr
¨
obner bases is the multi-
plication matrix method, known also as the action matrix
method [9, 43]. This method was recently used to effi-
ciently solve many of the minimal problems in computer
vision [22, 23, 28, 31]. The goal of this method is to trans-
form the problem of finding the solutions to (1) to a prob-
lem of eigendecomposition of a special multiplication ma-
trix [10]. Let us consider the mapping T
f
: A A of the
multiplication by a polynomial f C[X]. T
f
is a linear
mapping for which T
f
= T
g
iff f g I. In our case A is
a finite-dimensional vector space over C and therefore we
can represent T
f
by its matrix with respect to some linear
basis B of A. For a basis B = ([b
1
], . . . , [b
k
]) consisting of
k monomials, T
f
can be represented by k ×k multiplication
(action) matrix M
f
:= (m
ij
) such that T
f
([b
j
]) = [f b
j
] =
P
k
i=1
m
ij
[b
i
]. It can be shown [10] that λ C is an eigen-
value of the matrix M
f
iff λ is a value of the function f on
the variety V . In other words, if f is e.g. x
n
then the eigen-
values of M
f
are the x
n
-coordinates of the solutions of (1).
The solutions to the remaining variables can be obtained
from the eigenvectors of M
f
. This means that after finding
the multiplication matrix M
f
, we can recover the solutions
by solving the eigendecompostion of M
f
for which efficient
algorithms exist. Moreover, if the ideal I is a radical ideal,
i.e. I =
I, [10], then k is equal to the number of solutions
to the system (1). Therefore, Gr
¨
obner basis methods usu-
ally solve an eigenvalue problem of a size that is equivalent
to the number of solutions of the problem. For more details
and proofs we refer the reader to [9].
The coefficients of the multiplication matrix M
f
are poly-
nomial combinations of coefficients of the input polynomi-
als (1). For computer vision problems these polynomial
combinations are often found “offline“ in a pre-processing
step. In this step, a so-called elimination template is gener-
ated, which is actually an expanded set of equations con-
structed by multiplying original equations with different
monomials. This template matrix is constructed such that
after filling it with coefficients from the input equations and
performing Gauss-Jordan(G-J) elimination of this matrix,
the coefficients of the multiplication matrix M
f
can be ob-
tained from this eliminated template matrix.
The first automatic approach for generating elimination
templates and Gr
¨
obner basis solvers was presented in [23].

Recently an improvement to the automatic generator [23]
was proposed in [28] to exploit the inherent relations be-
tween the input polynomial equations and it results in more
efficient solvers than [23]. The automatic method from [28]
was later extended by a method for dealing with saturated
ideals [29] and a method for detecting symmetries in poly-
nomial systems [27].
In general, the answer to the question “What is the
smallest elimination template for a given problem?” is not
known. In [31] the authors showed that the method [28],
which is based on the grevlex ordering of monomials and
the so-called standard bases of the quotient ring A is not
optimal in terms of template sizes. The authors of [28]
proposed two methods for generating smaller elimination
templates. The first is based on enumerating and test-
ing all Gr
¨
obner bases w.r.t. different monomial order-
ings, i.e., the so-called Gr
¨
obner fan. By generating solvers
w.r.t. all these Gr
¨
obner bases and using standard bases
of the quotient ring A, smaller solvers were obtained for
many problems. The second method goes “beyond Gr
¨
obner
bases” and it uses a manually designed heuristic sampling
scheme for generating “non-standard” monomial bases B
of A = C[X]/I. This heuristic leads to more efficient
solvers than the Gr
¨
obner fan method in many cases. While
the Gr
¨
obner fan method will provably generate at least as
efficient solvers as the grevlex-based method from [28],
no proof can be in general given for the “heuristic-based”
method. The proposed heuristic sampling scheme uses
only empirical observations on which basis monomials will
likely result in small templates and it samples a fixed num-
ber (1000 in the paper) of candidate bases consisting of
these monomials. Even though, e.g. the standard grevlex
monomial basis will most likely be sampled during the sam-
pling, it is in general not clear how large templates it will
generate for a particular problem. The results will also de-
pend on the number of bases tested inside the heuristic.
2.2. Sparse Resultants
An alternate approach towards solving polynomial equa-
tions is that of using resultants. Simply put, a re-
sultant is an irreducible polynomial constraining co-
efficients of a set of n + 1 polynomials, F =
{f
1
(x
1
, . . . , x
n
), . . . , f
n+1
(x
1
, . . . , x
n
}) in n variables to
have a non-trivial solution. One can refer to Cox et al. [9]
for a more formal theory on resultants. We have n+1 equa-
tions in n variables because resultants were initially devel-
oped to determine whether a system of polynomial equa-
tions has a common root or not. If a coefficient of mono-
mial x
α
in the i
th
polynomial of F is denoted as u
i,α
the
resultant is a polynomial Res([u
i,α
]) with u
i,α
as variables.
Using this terminology, the basic idea for a resultant
based method is to expand F to a set of linearly independent
polynomials which can be linearised as M([u
i,α
])b, where
b is a vector of monomials of form x
α
and M([u
i,α
]) has
to be a square matrix that has full rank for generic val-
ues of u
i,α
, i.e. det M([u
i,α
]) 6= 0. The determinant of
the matrix M([u
i,α
]) is a non-trivial multiple of the resultant
Res([u
i,α
]) [9]. Thus det M([u
i,α
]) must vanish, if the resul-
tant vanishes, i.e. Res([u
i,α
]) = 0 = det M([u
i,α
]) = 0.
It is known that Res([u
i,α
]) vanishes iff the polynomial sys-
tem F has a solution [9]. This gives us the necessary condi-
tion for the existence of roots of F = 0. Hence the equation
det M([u
i,α
]) = 0 gives us those values of u
i,α
such that
F = 0 have a common root.
Resultants can be used to solve n polynomial equations
in n unknowns. The most common approach used for this
purpose is to hide a variable by considering it as a constant.
By hiding, say x
n
, we obtain n polynomials in n1 vari-
ables, so we can use the concept of resultants and compute
Res([u
i,α
], x
n
) which now becomes a function of u
i,α
as
well as x
n
. Algorithms based on hiding a variable attempt
to expand F to a linearly independent set of polynomials
that can be re-written in a matrix form as
M([u
i,m
], x
n
)b = 0, (2)
where M([u
i,α
], x
n
) is a square matrix whose elements are
polynomials in x
n
and coefficients u
i,α
and b is the vec-
tor of monomials in x
1
, . . . , x
n1
. For simplicity we will
denote the matrix M([u
i,α
], x
n
) as M(x
n
) in the rest of this
section. Here we actually estimate a multiple of the actual
resultant via the determinant of the matrix M(x
n
) in (2).
This resultant is known as a hidden variable resultant and
it is a polynomial in x
n
whose roots are the x
n
-coordinates
of the solutions of the system of polynomial equations. For
theoretical details and proofs see [9]. Such a hidden variable
approach has been used in the past to solve various minimal
problems [15, 19, 22, 24].
The most common way to solve the original system of
polynomial equations is to transform (2) to a polynomial
eigenvalue problem (PEP) [10] that transforms (2) as
(M
0
+ M
1
x
n
+ ... + M
l
x
l
n
)b = 0, (3)
where l is the degree of the matrix M(x
n
) in the hidden vari-
able x
n
and matrices M
0
, ..., M
l
are matrices that depend only
on the coefficients u
i,α
of the original system of polynomi-
als. The PEP (3) can be easily converted to a generalized
eigenvalue problem (GEP):
Ay = x
n
By, (4)
and solved using standard efficient eigenvalue algorithms.
Basically, the eigenvalues give us the solution to x
n
and the
rest of the variables can be solved from the corresponding
eigenvectors, y [9]. But this transformation to a GEP re-
laxes the original problem of finding the solutions to our
input system and computes eigenvectors that do not satisfy

the monomial dependencies induced by the monomial vec-
tor b. And many times it also introduces extra parasitic
(zero) eigenvalues leading to slower polynomial solvers.
Alternately, we can add a new polynomial
f
n+1
= u
0
+ u
1
x
1
+ ··· + u
n
x
n
(5)
to F and compute a so called u-resultant [9] by hiding
u
0
, . . . , u
n
. In general random values are assigned to
u
1
, . . . , u
n
. The u-resultant matrix is computed from these
n + 1 polynomials in n variables in a way similar to the one
explored above. For more details about u-resultant one can
refer to [9].
For sparse polynomial systems it is possible to ob-
tain more compact resultants using specialized algorithms.
Such resultants are commonly referred to as Sparse Re-
sultants. A sparse resultant would mostly lead to a more
compact matrix M(x
n
) and hence a smaller eigendecom-
position problem. Emiris et al. [12, 7] have proposed a
generalised approach for computing sparse resultants using
mixed-subdivision of polytopes. Based on [12, 7] Emiris
proposed a method for generating a resultant-based solver
for sparse systems of polynomial equations, that was di-
vided in “offline” and “online” computations. The resulting
solvers were based either on the hidden-variable trick (2) or
the u-resultant of the general form (5). As such the result-
ing solvers were usually quite large and not very efficient.
More recently Heikkil
¨
a [17] have proposed an improved
approach to test and extract smaller M(x
n
). This method
transforms (2) to a GEP (4) and solves for eigenvalues and
eigenvectors to compute solutions to unknowns. The meth-
ods [7, 11, 12, 17] suffer from the drawback that they re-
quire the input system to have as many polynomials as un-
knowns to be able to compute a resultant. Additionally, the
algorithm [17] suffers from other drawbacks and can not be
directly applied to most of the minimal problems. These
drawbacks can be overcome, as we describe in the supple-
mentary material. However, even with our proposed im-
provements the resultant-based method [17], which is based
on hiding one of the input variables in the coefficient field,
would result in a GEP with unwanted eigenvalues and in
turn unwanted solutions to original system (1). This leads
to slower solvers for most of the studied minimal problems.
Therefore, we investigate an alternate approach where
instead of hiding one of the input variables [11, 17] or us-
ing u-resultant of a general form (5) [11], we introduce an
extra variable λ and a new polynomial of a special form,
i.e., x
i
λ. The augmented polynomial system is solved
by hiding λ and reducing a constraint similar to (2) into
a regular eigenvalue problem that leads to smaller solvers
than [11, 17]. Next section lays the theoretical foundation
of our approach and outlines the algorithm along with the
steps for computing a sparse resultant matrix M(λ).
3. Sparse resultants using an extra equation
We start with a set of m polynomials from (1) in n vari-
ables x
1
, . . . , x
n
to be solved. Introducing an extra variable
λ we define x
0
= [x
1
, . . . , x
n
, λ] and an extra polynomial
f
m+1
(x
0
) = x
i
λ. Using this, we propose an algorithm
inspired by [17] and [11] to solve the following augmented
polynomial system for x
0
,
f
1
(x
0
) = 0, . . . , f
m
(x
0
) = 0, f
m+1
(x
0
) = 0. (6)
Our idea it to compute its sparse resultant matrix M(= M(λ))
by hiding λ in a way that allows us to solve (6) by reducing
its linearization (similar to (2)) to an eigenvalue problem.
3.1. Sparse resultant and eigenvalue problem
Our algorithm computes the monomial multiples of
the polynomials in (6) in the form of a set T =
{T
1
, . . . , T
m
, T
m+1
} where each T
i
denotes the set of
monomials to be multiplied by f
i
(x
0
). We may order mono-
mials in each T
i
to obtain a vector form, T
i
= vec(T
i
) and
stack these vectors as T = [T
1
, . . . , T
m
, T
m+1
] . The set
of all monomials present in the resulting extended set of
polynomials {x
α
i
f
i
(x
0
), x
α
i
T
i
, i = 1, . . . m + 1} is
called the monomial basis and is denoted as B = {x
α
|
α Z
n
0
}. The vector form of B w.r.t. some monomial
ordering is denoted as b. Then the extended set of polyno-
mials can be written in a matrix form,
M b = 0, (7)
The coefficient matrix M is a function of λ as well as the
coefficients of input polynomials (6). Let ε = |B|. Then
by construction [17] M is a tall matrix with p ε rows. We
can remove extra rows and form an invertible square matrix
which is the sparse resultant matrix mentioned in previous
section. While Heikkil
¨
a [17] solve a problem similar to (7)
as a GEP, we exploit the structure of newly added polyno-
mial f
m+1
(x
0
) and propose a block partition of M to reduce
the matrix equation of (7) to a regular eigenvalue problem.
Proposition 3.1. Let f
m+1
(x
0
) = x
i
λ, then there exists
a block partitioning of M in (7) as:
M =
M
11
M
12
M
21
M
22
, (8)
such that (7) can be converted to an eigenvalue problem of
the form X b
0
= λb
0
.
Proof: In order to block partition the columns in (8) we
need to partition B as B = B
λ
t B
c
where
B
λ
= B T
m+1
, B
c
= B B
λ
. (9)
Let us order the monomials in B, such that b = vec(B) =
vec(B
λ
) vec(B
c
)
T
=
b
1
b
2
T
. Such a partition of b in-
duces a column partition of M (7). We row partition M such

that the lower block is row-indexed by monomial multiples
of f
m+1
(x
0
) which are linear in λ (i.e. x
α
j
(x
i
λ), x
α
j
T
m+1
) while the upper block is indexed by monomial mul-
tiples of f
1
(x
0
), . . . , f
m
(x
0
). Such a row and column parti-
tion of M gives us a block partition as in (8). As
M
11
M
12
contains polynomials independent of the λ and
M
21
M
22
contains polynomials of the form x
α
j
(x
i
λ) we obtain
M
11
= A
11
, M
12
= A
12
M
21
= A
21
+ λB
21
, M
22
= A
22
+ λB
22
, (10)
where A
11
, A
12
, A
21
and A
22
are matrices dependent only on
the coefficients of input polynomials in (6). We assume here
that A
12
has full column rank. Substituting (10) in (8) gives
M =
M
11
M
12
M
21
M
22
=
A
11
A
12
A
21
A
22
+ λ
0 0
B
21
B
22
(11)
We can order monomials so that T
m+1
= b
1
. Now chosen
partition of M implies that M
21
is column indexed by b
1
and
row indexed by T
m+1
. As
M
21
M
22
has rows of form
x
α
j
(x
i
λ), x
α
j
T
m+1
= x
α
j
B
λ
. This gives us,
B
21
= I, where I is an identity matrix of size |B
λ
| and
B
22
is a zero matrix of size |B
λ
| × |B
c
|. This also means
that A
21
is a square matrix of same size as B
21
. Thus we
have a decomposition as
M = M
0
+ λM
1
=
A
11
A
12
A
21
A
22
+ λ
0 0
I 0
, (12)
where M is a p × ε matrix. If M is a tall matrix, so is A
12
from which we can eliminate extra rows to obtain a square
invertible matrix
ˆ
A
12
while preserving the above mentioned
structure, as discussed in Section 3.3. Let b =
b
1
b
2
T
.
Then from (7) and (12) we have
A
11
ˆ
A
12
A
21
A
22
b
1
b
2
+ λ
0 0
I 0
b
1
b
2
= 0
= A
11
b
1
+
ˆ
A
12
b
2
= 0,
A
21
b
1
+ A
22
b
2
λb
1
= 0 (13)
Eliminating b
2
from the above pair of equations we obtain
X
z }| {
(A
21
A
22
ˆ
A
1
12
A
11
) b
1
= λb
1
. (14)
If A
12
does not have full column rank, we change the par-
titioning of columns of M by changing the partitions, B
λ
=
{x
m
T
m+1
| x
i
x
m
B} and B
c
= B B
λ
by exploiting
the form of f
m+1
(x
0
). This gives us A
21
=I and A
22
=0. It
also results in a different A
12
which would have full column
rank. Hence from (12) we have
M = M
0
+ λM
1
=
A
11
A
12
I 0
+ λ
0 0
B
21
B
22
,(15)
which is substituted in (7) to get A
11
b
1
+ A
12
b
2
= 0 and
λ(B
21
b
1
+ B
22
b
2
) + b
1
= 0. Eliminating b
2
from these
equations we get an alternate eigenvalue formulation:
(B
21
B
22
ˆ
A
1
12
A
11
)b
1
= (1)b
1
. (16)
We note that (14) defines our proposed solver. Here we can
extract solutions to x
1
, . . . , x
n
by computing eigenvectors
of X. If in case
ˆ
A
12
is not invertible, we can use the al-
ternate formulation (16) and extract solutions in a similar
manner. It is worth noting that the speed of execution of
the solver depends on the size of b
1
(=|B
λ
|) as well the size
of
ˆ
A
12
while the accuracy of the solver largely depends on
the matrix to be inverted i.e.
ˆ
A
12
. Hence, in next section
we outline a generalized algorithm for computing a set of
monomial multiples T as well as the monomial basis B that
leads to matrix M satisfying Proposition 3.1.
3.2. Computing a monomial basis
Our approach is based on the algorithm explored in [17] for
computing a monomial basis B for a sparse resultant.
We briefly define the basic terms related to convex poly-
topes used for computing a monomial basis B. A New-
ton polytope of a polynomial NP(f) is defined as a con-
vex hull of the exponent vectors of the monomials occur-
ring in the polynomial (also known as the support of the
polynomial). Hence, we have NP(f
i
) = Conv(A
i
) where
A
i
= {α|α Z
n
0
} is the set of all integer vectors that are
exponents of monomials with non-zero coefficients in f
i
.
A Minkowski sum of any two convex polytopes P
1
, P
2
is
defined as P
1
+ P
2
= {p
1
+ p
2
| p
1
P
1
, p
2
P
2
}.
An extensive treatment of polytopes can be found from
[9]. The algorithm by Heikkil
¨
a [17] basically computes the
Minkowski sum of the Newton polytopes of a subset of in-
put polynomials, Q = Σ
i
NP(f
i
(x)). The set of integer
points in the interior of Q defined as B = Z
n1
(Q + δ),
where δ is a small random displacement vector, can pro-
vide a monomial basis B satisfying the constraint (2). Our
proposed approach computes B as a prospective monomial
basis in a similar way, albeit for a modified polynomial sys-
tem (6). Next we describe our approach and provide a de-
tailed algorithm for the same in the supplementary material.
Given a system of m( n) polynomials (1) in n vari-
ables X = {x
1
, . . . , x
n
} we introduce a new variable λ and
create n augmented systems F
0
= {f
1
, . . . , f
m
, x
i
λ} for
each variable x
i
X. Then we compute the support A
j
=
supp(f
j
) and the Newton polytope NP(f
j
) = conv(A
j
) for
each polynomial f
j
F
0
. The unit simplex NP
0
Z
n
is
also computed. For each polynomial system F
0
, we con-
sider each subset of polynomials F
sub
F
0
and compute its
Minkowski sum, Q = NP
0
+ Σ
fF
sub
NP(f). Then for var-
ious displacement vectors δ we try to compute a candidate
monomial basis B as the set of integer points inside Q + δ.

Citations
More filters
Proceedings ArticleDOI
01 Jun 2022
TL;DR: In this paper , a learning strategy for selecting a starting problem-solution pair that can be numerically continued to the problem and the solution of interest is proposed, which avoids computing large numbers of spurious solutions.
Abstract: We present an approach to solving hard geometric optimization problems in the RANSAC framework. The hard minimal problems arise from relaxing the original geometric optimization problem into a minimal problem with many spurious solutions. Our approach avoids computing large numbers of spurious solutions. We design a learning strategy for selecting a starting problem-solution pair that can be numerically continued to the problem and the solution of interest. We demonstrate our approach by developing a RANSAC solver for the problem of computing the relative pose of three calibrated cameras, via a minimal relaxation using four points in each view. On average, we can solve a single problem in under $70\ \mu s$ . We also benchmark and study our engineering choices on the very familiar problem of computing the relative pose of two calibrated cameras, via the minimal case of five points in two views.

5 citations

Posted Content
TL;DR: A fast and robust minimal solver for simultaneously estimating the focal length, radial distortion profile and motion parameters from homographies, and perform better or on par with state-of-the-art methods relying on pre-calibration procedures.
Abstract: In this paper we present a novel algorithm for onboard radial distortion correction for unmanned aerial vehicles (UAVs) equipped with an inertial measurement unit (IMU), that runs in real-time. This approach makes calibration procedures redundant, thus allowing for exchange of optics extemporaneously. By utilizing the IMU data, the cameras can be aligned with the gravity direction. This allows us to work with fewer degrees of freedom, and opens up for further intrinsic calibration. We propose a fast and robust minimal solver for simultaneously estimating the focal length, radial distortion profile and motion parameters from homographies. The proposed solver is tested on both synthetic and real data, and perform better or on par with state-of-the-art methods relying on pre-calibration procedures.

4 citations


Additional excerpts

  • ...Recently, alternative methods relying on resultants show promising results [2, 1]....

    [...]

Proceedings ArticleDOI
28 Mar 2022
TL;DR: A new method is proposed for constructing elimination templates for efficient polynomial system solving of minimal problems in structure from motion, image matching, and camera tracking using a heuristic greedy optimization strategy over the space of parameters to get a template with a small size.
Abstract: We propose a new method for constructing elimination templates for efficient polynomial system solving of minimal problems in structure from motion, image matching, and camera tracking. We first construct a particular affine parameterization of the elimination templates for systems with a finite number of distinct solutions. Then, we use a heuristic greedy optimization strategy over the space of parameters to get a template with a small size. We test our method on 34 minimal problems in computer vision. For all of them, we found the templates either of the same or smaller size compared to the state-of-the-art. For some difficult examples, our templates are, e.g., 2.1, 2.5, 3.8, 6.6 times smaller. For the problem of refractive absolute pose estimation with unknown focal length, we have found a template that is 20 times smaller. Our experiments on synthetic data also show that the new solvers are fast and numerically accurate. We also present a fast and numerically accurate solver for the problem of relative pose estimation with unknown common focal length and radial distortion.

2 citations

Proceedings ArticleDOI
01 Jan 2021
TL;DR: In this paper, a real-time algorithm for onboard radial distortion correction for UAVs equipped with an inertial measurement unit (IMU) that runs in real time is presented.
Abstract: In this paper we present a novel algorithm for onboard radial distortion correction for unmanned aerial vehicles (UAVs) equipped with an inertial measurement unit (IMU), that runs in real-time. This approach makes calibration procedures redundant, thus allowing for exchange of optics extemporaneously. By utilizing the IMU data, the cameras can be aligned with the gravity direction. This allows us to work with fewer degrees of freedom, and opens up for further intrinsic calibration. We propose a fast and robust minimal solver for simultaneously estimating the focal length, radial distortion profile and motion parameters from homographies. The proposed solver is tested on both synthetic and real data, and perform better or on par with state-of-the-art methods relying on pre-calibration procedures. Code available at: https://github.com/marcusvaltonen/HomLib.1

2 citations

Proceedings ArticleDOI
10 Jan 2021
TL;DR: In this paper, an interesting alternative resultant-based method for solving sparse systems of polynomial equations by hiding one variable is proposed, which results in a larger eigenvalue problem than the action matrix and extra variable resultant based methods.
Abstract: Many computer vision applications require robust and efficient estimation of camera geometry. The robust estimation is usually based on solving camera geometry problems from a minimal number of input data measurements, i.e., solving minimal problems, in a RANSAC-style framework. Minimal problems often result in complex systems of polynomial equations. The existing state-of-the-art methods for solving such systems are either based on Grobner bases and the action matrix method, which have been extensively studied and optimized in the recent years or recently proposed approach based on a resultant computation using an extra variable. In this paper, we study an interesting alternative resultant-based method for solving sparse systems of polynomial equations by hiding one variable. This approach results in a larger eigenvalue problem than the action matrix and extra variable resultant-based methods; however, it does not need to compute an inverse or elimination of large matrices that may be numerically unstable. The proposed approach includes several improvements to the standard sparse resultant algorithms, which significantly improves the efficiency and stability of the hidden variable resultant-based solvers as we demonstrate on several interesting computer vision problems. We show that for the studied problems, our sparse resultant based approach leads to more stable solvers than the state-of-the-art Grobner basis as well as existing resultant-based solvers, especially in close to critical configurations. Our new method can be fully automated and incorporated into existing tools for the automatic generation of efficient minimal solvers.

2 citations

References
More filters
Journal ArticleDOI
TL;DR: New results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form that provide the basis for an automatic system that can solve the Location Determination Problem under difficult viewing.
Abstract: A new paradigm, Random Sample Consensus (RANSAC), for fitting a model to experimental data is introduced. RANSAC is capable of interpreting/smoothing data containing a significant percentage of gross errors, and is thus ideally suited for applications in automated image analysis where interpretation is based on the data provided by error-prone feature detectors. A major portion of this paper describes the application of RANSAC to the Location Determination Problem (LDP): Given an image depicting a set of landmarks with known locations, determine that point in space from which the image was obtained. In response to a RANSAC requirement, new results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form. These results provide the basis for an automatic system that can solve the LDP under difficult viewing

23,396 citations


"A Sparse Resultant Based Method for..." refers background in this paper

  • ...The robust estimation is usually based on solving camera geometry problems from a minimal number of input data measurements, i.e. solving minimal problems in a RANSAC framework....

    [...]

  • ...We used the 3D model to estimate the pose of each image using the new P4Pfr resultant-based solver (28× 40) in a RANSAC framework....

    [...]

  • ...problems that are solved from minimal samples of input data, inside a RANSAC framework [14, 9, 36]....

    [...]

  • ...Since the camera geometry estimation has to be performed many times in RANSAC [14], fast solvers to minimal problems are of high importance....

    [...]

  • ...The slightly different results reported in [31] are due to RANSAC’s random nature and a slightly different P4Pfr formulation (40x50) used in [31]....

    [...]

Book
01 Jan 2000
TL;DR: In this article, the authors provide comprehensive background material and explain how to apply the methods and implement the algorithms directly in a unified framework, including geometric principles and how to represent objects algebraically so they can be computed and applied.
Abstract: From the Publisher: A basic problem in computer vision is to understand the structure of a real world scene given several images of it. Recent major developments in the theory and practice of scene reconstruction are described in detail in a unified framework. The book covers the geometric principles and how to represent objects algebraically so they can be computed and applied. The authors provide comprehensive background material and explain how to apply the methods and implement the algorithms directly.

15,558 citations

Journal ArticleDOI
TL;DR: This paper presents structure-from-motion and image-based rendering algorithms that operate on hundreds of images downloaded as a result of keyword-based image search queries like “Notre Dame” or “Trevi Fountain,” and presents these algorithms and results as a first step towards 3D modeled sites, cities, and landscapes from Internet imagery.
Abstract: There are billions of photographs on the Internet, comprising the largest and most diverse photo collection ever assembled. How can computer vision researchers exploit this imagery? This paper explores this question from the standpoint of 3D scene modeling and visualization. We present structure-from-motion and image-based rendering algorithms that operate on hundreds of images downloaded as a result of keyword-based image search queries like "Notre Dame" or "Trevi Fountain." This approach, which we call Photo Tourism, has enabled reconstructions of numerous well-known world sites. This paper presents these algorithms and results as a first step towards 3D modeling of the world's well-photographed sites, cities, and landscapes from Internet imagery, and discusses key open problems and challenges for the research community.

2,207 citations


"A Sparse Resultant Based Method for..." refers background in this paper

  • ...in structure from motion [39], visual navigation [38], large scale 3D reconstruction [19] and image localization [37]....

    [...]

Journal ArticleDOI
David Nister1
TL;DR: The algorithm is used in a robust hypothesize-and-test framework to estimate structure and motion in real-time with low delay and is the first algorithm well-suited for numerical implementation that also corresponds to the inherent complexity of the problem.
Abstract: An efficient algorithmic solution to the classical five-point relative pose problem is presented. The problem is to find the possible solutions for relative camera pose between two calibrated views given five corresponding points. The algorithm consists of computing the coefficients of a tenth degree polynomial in closed form and, subsequently, finding its roots. It is the first algorithm well-suited for numerical implementation that also corresponds to the inherent complexity of the problem. We investigate the numerical precision of the algorithm. We also study its performance under noise in minimal as well as overdetermined cases. The performance is compared to that of the well-known 8 and 7-point methods and a 6-point scheme. The algorithm is used in a robust hypothesize-and-test framework to estimate structure and motion in real-time with low delay. The real-time system uses solely visual input and has been demonstrated at major conferences.

2,077 citations


"A Sparse Resultant Based Method for..." refers background in this paper

  • ...The robust estimation of camera geometry is usually based on solving so-called minimal problems [35, 24, 23], i....

    [...]

  • ...systems resulting from the 5-pt relative pose problem [35], and move as much computation as possible from the “online” stage of solving equations to an earlier pre-processing “offline” stage....

    [...]

Journal ArticleDOI
TL;DR: Visual odometry is the process of estimating the egomotion of an agent (e.g., vehicle, human, and robot) using only the input of a single or If multiple cameras attached to it, and application domains include robotics, wearable computing, augmented reality, and automotive.
Abstract: Visual odometry (VO) is the process of estimating the egomotion of an agent (e.g., vehicle, human, and robot) using only the input of a single or If multiple cameras attached to it. Application domains include robotics, wearable computing, augmented reality, and automotive. The term VO was coined in 2004 by Nister in his landmark paper. The term was chosen for its similarity to wheel odometry, which incrementally estimates the motion of a vehicle by integrating the number of turns of its wheels over time. Likewise, VO operates by incrementally estimating the pose of the vehicle through examination of the changes that motion induces on the images of its onboard cameras. For VO to work effectively, there should be sufficient illumination in the environment and a static scene with enough texture to allow apparent motion to be extracted. Furthermore, consecutive frames should be captured by ensuring that they have sufficient scene overlap.

1,371 citations


"A Sparse Resultant Based Method for..." refers background in this paper

  • ...in structure from motion [39], visual navigation [38], large scale 3D reconstruction [19] and image localization [37]....

    [...]

Frequently Asked Questions (18)
Q1. What contributions have the authors mentioned in the paper "A sparse resultant based method for efficient minimal solvers" ?

In this paper the authors study an alternative algebraic method for solving systems of polynomial equations, i. e., the sparse resultant-based method and propose a novel approach to convert the resultant constraint to an eigenvalue problem. The authors applied their new resultant-based method to a large variety of computer vision problems and show that for most of the considered problems, the new method leads to solvers that are the same size as the the best available Gröbner basis solvers and of similar accuracy. 

One of the popular approaches for solving systems of equations using Gröbner bases is the multiplication matrix method, known also as the action matrix method [9, 43]. 

Using a Gröbner basis the authors can define a linear basis for the quotient ring A = C[X]/I .Gröbner bases can be used to solve their system of polynomial equations (1). 

Since the camera geometry estimation has to be performed many times in RANSAC [13], fast solvers to minimal problems are of high importance. 

The augmented polynomial system is solved by hiding λ and reducing a constraint similar to (2) into a regular eigenvalue problem that leads to smaller solvers than [11, 17]. 

A popular approach for solving minimal problems is to design procedures that can efficiently solve only a special class of systems of equations, e.g. systems resulting from the 5-pt relative pose problem [34], and move as much computation as possible from the “online” stage of solving equations to an earlier pre-processing “offline” stage. 

In order to block partition the columns in (8) the authors need to partition B as B = Bλ tBc whereBλ = B ∩ Tm+1, Bc = B −Bλ. (9)Let us order the monomials in B, such that b = vec(B) =[ vec(Bλ) vec(Bc) ]T = [ b1 b2 ]T . 

Stability measures include mean and median of Log10 of normalized equation residuals for computed solutions as well as the solvers failures as a % of 5K instances for which at least one solution has a normalized residual > 10−3. 

While Heikkilä [17] solve a problem similar to (7) as a GEP, the authors exploit the structure of newly added polynomial fm+1(x′) and propose a block partition of M to reduce the matrix equation of (7) to a regular eigenvalue problem. 

By generating solvers w.r.t. all these Gröbner bases and using standard bases of the quotient ring A, smaller solvers were obtained for many problems. 

the authors have NP(fi) = Conv(Ai) where Ai = {α|α ∈ Zn≥0} is the set of all integer vectors that are exponents of monomials with non-zero coefficients in fi. 

The most common way to solve the original system of polynomial equations is to transform (2) to a polynomial eigenvalue problem (PEP) [10] that transforms (2) as(M0 + M1 xn + ...+ Ml x l n)b = 0, (3)where l is the degree of the matrix M(xn) in the hidden variable xn and matrices M0, ..., Ml are matrices that depend only on the coefficients ui,α of the original system of polynomials. 

Using this terminology, the basic idea for a resultant based method is to expand F to a set of linearly independent polynomials which can be linearised as M([ui,α])b, whereb is a vector of monomials of form xα and M([ui,α]) has to be a square matrix that has full rank for generic values of ui,α, i.e. det M([ui,α]) 

If a coefficient of monomial xα in the ith polynomial of F is denoted as ui,α the resultant is a polynomialRes([ui,α]) with ui,α as variables. 

The authors evaluated the resultant-based solver for a practical problem of estimating the absolute pose of camera with unknown focal length and radial distortion from four 2D-to-3D point correspondences, i.e. the P4Pfr solver, on real data. 

in the last 15 years much effort has been put into making the process of constructing the solvers more automatic [23, 28, 29] and the solvers stable [5, 6] and more efficient [28, 29, 27, 4, 31]. 

Figure 1 (left) shows histogram of Log10 of normalized equation residuals for the “Rel.pose λ+E+λ” problem, where their solver is not only faster, but also more stable than the state-of-the-art solvers. 

For each polynomial system F ′, the authors consider each subset of polynomials Fsub ⊂ F ′ and compute its Minkowski sum, Q = NP0 + Σf∈Fsub NP(f).