HAL Id: inria-00590078
https://hal.inria.fr/inria-00590078
Submitted on 5 May 2011
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Uncalibrated 1D Projective Camera and 3D Ane
Reconstruction of Lines
Long Quan
To cite this version:
Long Quan. Uncalibrated 1D Projective Camera and 3D Ane Reconstruction of Lines. IEEE
Conference on Computer Vision and Pattern Recognition (CVPR ’97), Jun 1997, San Juan, Puerto
Rico. pp.60–65, �10.1109/CVPR.1997.609298�. �inria-00590078�
Uncalibrated 1D Projective Camera and 3D Affine Reconstruction of Lines
Long QUAN
CNRS-GRAVIR-INRIA
ZIRST – 655 avenue de l’Europe
38330 Montbonnot, France
Email: Long.Quan@inrialpes.fr
Abstract
We describe a linear algorithm to recover 3D affine
shape/motion from line correspondences over three views
with uncalibrated affine cameras. The key idea is the in-
troduction of a one-dimensional projective camera. This
converts the 3D affine reconstruction of “lines” into 2D
projective reconstruction of “points”. Using the full tenso-
rial representation of three uncalibrated 1D views, we prove
that the 3D affine reconstruction of lines from minimal data
is unique up to a re-ordering of the views. 3D affine line
reconstruction can be performed by properly rescaling im-
age coordinates instead of using projection matrices. The
algorithm is validated on both simulated and real image se-
quences.
1. Introduction
Using line segments instead of points as features has
attracted the attention of many researchers [11, 2, 29, 28,
27, 1] for various tasks such as pose estimation, stereo and
structure from motion. In this paper, we are interested in
structure from motion using line correspondences across
mutiple images. A minimum of three views is essential
for this, whereas two views suffice for point correspon-
dences. In the case of calibrated perspective cameras, the
main results on structure from line correspondences were
established in [11, 22, 2]: With at least six line correspon-
dences over three views, nonlinear algorithms are possible.
With at least thirteen lines over three views, a linear algo-
rithm is possible. The basic idea of the thirteen-line lin-
ear algorithm is similar to that of the eight-point algorithm
[12]: It is based on the introduction of a set of redundant
intermediate parameters. This provides a very heavy over-
parametrization of the problem that definitely leads to the
instability of the algorithm reported in [11]. The thirteen-
line algorithm was extended to uncalibrated camera case in
[7, 27]. The situation for uncalibrated camera case might be
expected to be better, as more free parameters are needed.
However, the 27 tensor components that are introduced as
intermediateparametersare still subject to 9 complicated al-
gebraic constraints. The algorithm can hardly be stable. A
subsequent nonlinear optimization step is almost unavoid-
able to refine the solution [2, 11, 22, 7].
In parallel, there has been a lot of work [23, 26, 20, 16,
17, 9, 10, 8, 14, 25] on structure from motion with sim-
plified camera models varing from orthographic projections
via weak and para-perspective to affine cameras, almost ex-
clusively for point features. These simplified camera mod-
els provide a good approximation to perpsective projection
when the depth of the object is small compared to the view-
ing distance. More importantly, they expose the ambiguities
that arise when perspective effects diminish. In such cases,
it is not only easier to use these simplified models but also
advisable to do so, as by explicitly eliminating the ambigu-
ities from the algorithm, one avoids computing parameters
that are inherently ill-conditioned. Another important ad-
vantage of working with uncalibrated affine cameras is that
the reconstruction is affine, rather than projective as with
uncalibrated projective cameras.
Motivated on the one hand by the lack of satisfactory
line-based algorithms for projective cameras and on the
other by the fact that the affine camera is a good model
for many practical cases, we investigated the properties of
line projection by affine cameras and proposed a linear al-
gorithm [18, 19] for affine structure from line correspon-
dences.
This paper is an extension of our previous work in which
the key advance introducing a one-dimensional projective
camera was made. The previous work concentrated on the
redundant data case to accomodate a factorization scheme
for lines. We were unable to solve for the reconstruction
ambiguity. In this paper, we use the same theoretical frame-
work but concentrate on the minimal data case. Instead
of using a projection matrix representation for reconstruc-
tion as in the previous work, we rely on a tensorial rep-
resentation of multi-views with one-dimensional cameras.
A complete analysis of the joint projection matrix reveals
the important role of the “epipoles” which, although redun-
dant with respect to the trilinear tensor, play a central role in
disambiguating the reconstruction. This new developement
allows us to finally prove that 3D affine reconstruction of
lines with the minimal data is unique up to a re-ordering
of views. Subsequently, a reconstruction algorithm based
on the rescaling of image coordinates is proposed and vali-
dated on both simulated and real images.
Throughout the paper, tensors and matrices are denoted
in upper case boldface, vectors in lower case boldface and
scalars in either plain letters or lower case Greek.
2. Review of the affine camera model for lines
As far as perspective (pin-hole) cameras are concerned,
the projection of a point x
= (
x; y ; z ; t
)
T
of
P
3
to a point
u
= (
u; v ; w
)
T
of
P
2
can be described by a
3
4
homoge-
neous projection matrix P:
u
=
P
3
4
x
:
(1)
For a restricted class of camera models, by setting the
third row of the perspective camera P to
(0
;
0
;
0
;
)
, we ob-
tain the affine camera initially introduced by Mundy and
Zisserman in [15]
A
3
4
=
0
@
p
11
p
12
p
13
p
14
p
21
p
22
p
23
p
24
0 0 0
p
34
1
A
M
2
3
0
1
3
t
3
1
:
(2)
This is the uncalibrated affine camera which emcom-
passes all the uncalibrated versions of the orthographic,
weak perspective and paraperspective camera models.
Now consider a line in
IR
3
through a point x
0
with direc-
tion d
x
:
x
a
=
x
0
+
d
x
:
The affine camera A
3
4
projects this to an image line:
A
3
4
x
1
= (
Mx
0
+
t
0
) +
Md
x
=
u
0
+
Md
x
;
with direction
d
u
=
M
2
3
d
x
;
(3)
passing through the image point
u
0
Mx
0
+
t
0
:
Equation (3) describes a linear mapping between direc-
tions of 3D lines and those of 2D lines. It can be derived
even more directly using projective geometry, by consid-
ering that the line with direction d
x
is the point at infinity
x
1
= (
d
T
x
;
0)
T
in
P
3
and the line with direction d
u
is the
point at infinity u
1
in
P
2
.
Comparing Equation (3) with Equation (1) which is a
projection from
P
3
to
P
2
, we see that Equation (3) is noth-
ing but a projective projection from
P
2
to
P
1
if we consider
the 3D and 2D directions of lines as 2D and 1D projective
points. This means that the affine reconstruction of lines
with a two-dimensional affine camera is equivalent to the
projective reconstruction of points with a one-dimensional
projective camera!
There have been many recent works [3, 5, 24, 13, 4, 6,
21, 22] on projective reconstruction and the geometry of
multi-views of two dimensional uncalibrated cameras. Par-
ticularly, the tensorial formalism developed by Triggs [24]
is very interesting and powerful. We are now extending this
study to the case of the one-dimensional camera.
3. Uncalibrated one-dimensional camera
First, rewrite Equation (3) in the following form:
u
=
M
2
3
x (4)
in which we use u
= (
u
1
; u
2
)
T
and x
= (
x
1
; x
2
; x
3
)
T
instead of d
u
and d
x
to stress that we are dealing with
“points” in the projective spaces
P
2
and
P
1
rather than line
directions in the vector spaces
IR
3
and
IR
2
. This exactly de-
scribes a one-dimensional projective camera which projects
a point x in
P
2
onto a point u in
P
1
.
We now examine the matching constraints between mul-
tiple views of the same point. There is a constraint only for
the case of 3 views.
Let the three views of the same point x be given as fol-
lows:
8
<
:
u
=
Mx
;
0
u
0
=
M
0
x
;
00
u
00
=
M
00
x
:
(5)
These can be rewritten in matrix form as
0
@
M u
0 0
M
0
0
u
0
0
M
00
0 0
u
00
1
A
0
B
B
@
x
?
?
0
?
00
1
C
C
A
= 0
;
(6)
which is the basic reconstruction equation for a one-
dimensional camera. The vector
(
x
;
?
;
?
0
;
?
00
)
T
can-
not be zero, and so
M u
0 0
M
0
0
u
0
0
M
00
0 0
u
00
= 0
:
(7)
The expansion of this determinant produces a trilinear
constraint of three views
2
X
i;j;k
=1
T
ij k
u
i
u
0
j
u
00
k
= 0
;
(8)
or in short
T
2
2
2
uu
0
u
00
= 0
:
where T
2
2
2
= (
T
ij k
)
is a
2
2
2
homogeneous
tensor whose components
T
ij k
are
3
3
minors of the fol-
lowing
6
3
joint projection matrix:
0
@
M
M
0
M
00
1
A
=
0
B
B
B
B
B
B
@
1
2
1
0
2
0
1
00
2
00
1
C
C
C
C
C
C
A
:
(9)
The components of the tensor can be made explicit as
T
ij k
= [
i
j
0
k
00
]
;
for
i; j
0
; k
00
= 1
;
2
:
where the bracket
[
ij
0
k
00
]
denotes the
3
3
minor of
i
-th,
j
0
-
th and
k
00
-th row vector of the above joint projection matrix
and bar “
” in
i
,
j
and
k
denotes the mapping
(1
;
2)
7!
(2
;
?
1)
:
It can be easily seen that any constraint obtained by
adding further views reduces to a trilinearity. This proves
the uniqueness of the trilinear constraint. Moreover, the
2
2
2
homogeneoustensor T
2
2
2
has
7 = 2
2
2
?
1
d.o.f., so it is a minimal parametrizationof three views since
three views have exactly
3
(2
3
?
1)
?
(3
3
?
1) = 7
d.o.f., up to a projective transformation in
P
2
.
Each correspondence over three views gives one linear
constraint on the tensor components
T
ij k
. With at least 7
points in
P
1
, the tensor components
T
ij k
can be estimated
linearly.
At this point, we have obtained a remarkable result that
for the one-dimensional projective camera, the trilinear ten-
sor encapsulates exactly the information needed for projec-
tive reconstruction in
P
2
. Namely, it is the unique matching
constraint, it minimally parametrizes the three views and it
can be estimated linearly. Contrast this to the 2D image
case in which the multilinear constraints are algebraically
redundant and the linear estimation is only an approxima-
tion based on over-parametrization.
3.1. 2D projective reconstruction by rescaling
According to Triggs [24], the projective reconstruction
in
P
3
can be viewed as being equivalent to the rescaling of
the image points in
P
2
. We have just proven that recover-
ing the directions of affine lines in 3D space is equivalent
to 2D projective reconstruction from one-dimensional pro-
jective images. Therefore, a reconstruction of the directions
of 3D affine lines can be obtained by rescaling the direction
vectors of image lines, viewed as points of
P
1
.
For each 1D image point through in views (cf. Equa-
tion (5)), the scale factors
,
0
and
00
–taken individually–
are arbitrary: However,taken as a whole
(
u
;
0
u
0
;
00
u
00
)
T
,
they encode the projective structure of the points x in
P
2
. One way to explicitly recover the scale factors
(
;
0
;
00
)
T
is to notice that the rescaled image coordinates
(
u
;
0
u
0
;
00
u
00
)
T
should lie in the joint image, or alterna-
tively to observe the following matrix identity:
0
@
M
u
M
0
0
u
0
M
00
00
u
00
1
A
=
0
@
M
M
0
M
00
1
A
?
I
3
3
x
:
The rank ofthe left matrix is thereforeat most 3. All
4
4
minors vanish. Expanding by cofactors in the last column
gives homogeneous linear equations in the components of
u,
0
u
0
and
00
u
00
with coefficients that are
3
3
minors of
the joint projection matrix:
T
j k
(
u
)
?
e
00
1
(
0
u
0
)
T
+
e
0
1
(
00
u
00
)
T
=
0
2
2
;
(10)
where T
j k
u is for
P
2
i
=1
T
ij k
u
i
, a
2
2
matrix.
There are two types of minors: Those involving three
views with one row from each view and those involving two
views with two rows from one view and one from the other.
The first type gives the 8 components of the tensor T
2
2
2
and the second type gives 12 components of the “epipoles”
e
1
;
e
2
;
e
0
1
;
e
0
2
;
e
00
1
;
e
00
2
. The epipoles are defined by analogy
with the 2D camera case, as the projection of one projection
center onto another view.
At present we only know
T
ij k
–the epipoles are still un-
known. To find the rescaling factors for projective recon-
strucion, we need to solve for the epipoles. One way to
proceed is as follows. Taking x to be the projection center
of the second view o
0
, and projecting into the three views,
Equation (10) reduces to
T
j k
e
2
=
?
00
e
0
1
e
00
T
2
:
As e
0
1
e
00
T
has rank 1, so does T
j k
e
2
. Its
2
2
determi-
nant must vanish, i.e.
j
T
j k
e
2
j
= 0
:
As each entry of the
2
2
matrix is homogeneous linear
in e
2
= (
u
1
; u
2
)
T
, the expansion of
j
T
j k
e
2
j
gives a homo-
geneous quadratic
u
2
1
+
u
1
u
2
+
u
2
2
= 0
;
(11)
where
; ;
are known in terms of
T
ij k
.
Doing the same thing with the projection center of the
third view o
00
gives
T
j k
e
3
=
0
e
00
1
e
0
T
3
:
and hence
j
T
j k
e
3
j
= 0
:
In other words, it leads to exactly the same quadratic equa-
tion (11) with e
3
replacing e
2
. The two solutions of the
quadratic (11) are e
2
and e
3
–only the ordering remains am-
biguous.
The other epipoles are easily obtained, e
0
1
and e
00
2
by
factorizing the matrix T
j k
e
2
and e
00
1
and e
0
1
by factorizing
T
j k
e
3
.
If the first solution set is
f
~
e
2
;
~
e
0
1
;
~
e
00
2
;
~
e
3
;
~
e
00
1
;
~
e
0
3
g
;
the reordering gives the second solution set
f
e
3
=
~
e
2
;
e
00
1
=
~
e
0
1
;
e
0
3
=
~
e
00
2
;
e
2
=
~
e
00
3
;
e
0
1
=
~
e
00
1
;
e
00
2
=
~
e
0
3
g
:
Once all the epipoles have been recovered, the scale fac-
tors of the image “points” for 3D direction reconstruction
can easily be recovered by solving the linear homogeneous
equation (10).
3.2. Retrieving normal forms for projection matri-
ces
The geometry of the three views is most conveniently,
and completely represented by the projection matrices asso-
ciated with each view. In the previous section, the trilinear
tensor was expressed in terms of the projection matrices.
Now we seek a map from the trilinear tensor representa-
tion back to the projection matrix representation of the three
views.
Without loss of generality, we can always take the fol-
lowing normal forms for the 3 projection matrices
M
=
?
I
2
2
0
;
M
0
=
?
A
2
2
c
;
M
00
=
?
D
2
2
f
:
(12)
It is straightforward to verify that the projection center
of the first view is Ker
(
M
1
) = (0
;
0
;
1)
T
, so that e
0
1
=
c and
e
00
1
=
f.
Now, the trilinear tensor
(
T
ij k
)
can be exhibited as
T
ij k
= (
?
1)
i
+1
(
d
ki
c
j
?
a
j i
f
k
)
:
(13)
As c and f are known,
a
ij
and
d
ij
can be solved linearly
from the eight homogeneous equations of (13).
Note that in our previous work [18], we recovered the
projection matrices nonlinearly without knowing epipoles,
whereas here we recover them linearly using the epipoles.
4. Uncalibrated translations and affine shape
To recover the full affine structure of the lines, we still
need to find the vector t
3
1
of the affine cameras defined
in (2). These represent the image translation and magnifica-
tion components of the camera. Recall that line correspon-
dences from two views do not impose any constraints on
camera motion: The minimum number of views required
is three. The recovery of the uncalibrated translations is
essentially linear once the uncalibrated rotations have been
recovered. A detailed linear algorithm is developed in our
previous work [18, 19].
The final reconstruction step of lines can be easily
formulated as a subspace selection and solved by SVD
[18, 19].
5. Affine-structure-from-lines theorem
In view of the results obtained above, we can establish
the following.
For the recovery of affine shape and affine motion from
line correspondences with an uncalibrated affine camera,
the minimum number of views needed is three and the mini-
mum number of lines required is seven for a linear solution.
The recovery is unique up to a re-ordering of the views.
This result can be compared with that of Koenderink and
Van Doorn [9] for affine structure with a minimum of two
views and five points.
6. Experimental results
The algorithm presented in this paper has been validated
with both simulated and real image sequences. Due to lack
of space, only an experiment based on real images will be
presented.
A Fujinon/Photometrics CCD camera is used to aquire a
sequence of images of a box of size
12
12
12
:
65
cm
. The
image resolution is
576
384
. A Canny-like edge detector
is first applied to each image. The contour points are then
linked and fitted to line segments by least squares. Line
correspondences across three views are selected by hand. A
total of 46 lines is selected, as shown in Figure 1.
The reconstruction algorithm generates infinite 3D lines.
To find 3D line segments, we reproject the 3D lines into one
