Complete Visual Metrology using Relative Affine
Structure
Adersh Miglani
Adersh.Miglani@gmail.com
Sumantra Dutta Roy
sumantra@ee.iitd.ac.in
Santanu Chaudhury
santanuc@ee.iitd.ac.in
J B Srivastava
jbsrivas@gmail.com
Abstract—We propose a framework for retrieving metric
information for repeated objects from single perspective image.
Relative affine structure, which is an invariant, is directly
proportional to the Euclidean distance of a three dimensional
point from a reference plane. The proposed method is based
on this fundamental concept. The first object undergoes 4 × 4
transformation and results in a repeated object. We represent
this transformation in terms of three relative affine structures
along X , Y and Z axes. Additionally, we propose the possible
extension of this framework for motion analysis - structure from
motion and motion segmentation.
I. INTRODUCTION
In computer vision, invariants are widely used for recog-
nition and classification of objects and three dimensional
reconstruction of a scene from one or more uncalibrated
images [1][2][3][4]. Broadly interpreted, all these vision tasks
use invariants for retrieving geometric properties of objects
from images. Here, we use a view-point invariant for retrieving
metric measurements of multiple objects with translational
and affine repetition from single perspective image. No prior
knowledge of camera’s internal and external parameters is
required in this setup. The camera model used in this work is
the central projection.
Repetition of two and three dimensional objects are fre-
quently used in multiple vision tasks. The relative affine struc-
ture is one of the widely used tools in the context of repeated
objects [4]. This is a projective invariant for repeated objects
and turns into an affine invariant under special cases such as
parallel projection. Prior work used relative affine structure for
reconstruction and recognition of three dimensional objects,
retrieving structure from motion and synthesizing new views
from multiple prespective images [2][3][5]. Here, we explored
its fundamental property that it is directly proportional to the
Euclidean distance of a point from the reference plane. This
led us to create another visual metrology framework beyond
traditional usages of relative affine structure.
In this paper, we blend and develop previous results on rela-
tive affine structure and single view metrology [4][6][7][8][9].
Earlier, relative affine structure was used for retrieving three
dimensional projective structure of one object from multiple
images taken from different view points [4][2]. Later, it
was used for projective reconstruction of repeated objects
with translational and affine repetition from single perspective
image [3]. And, homology, a plane projective transforma-
tion, was used for vanishing points based visual metrology
techniques and camera calibration [6][7][10][11][12]. Here,
we use relative affine structure and homology, together in
a different manner, for computing metric measurements of
repeated objects with minimal scene information and reference
metric measurements.
In single view metrology, we can compute the distance
between two parallel planes when the corresponding points
on the planes are along the direction normal to the planes
[7]. This is a requirement to establish homology between two
parallel planes [10]. But, our framework considers the general
configuration of corresponding points on the repeated objects
and is not restricted to such point correspondences.
Given reference measurements of first object, measurements
of repeated objects, irrespective of translation and affine
transformation, are computed. In case of affine repetition,
computed measurements are upto a uniform scale along the
reference direction that is normal to reference plane. Further-
more, proposed framework can be extended for vision tasks
involving multiple views such as motion analysis - structure
from motion, motion segmentation and tracking.
Section II describes relative affine structure and its proper-
ties suited for the proposed framework. The method to com-
pute measurements of repeated objects, individually, without
using relative affine structures for corresponding points is
developed in section III. Section IV has details on retrieving
metric information for affinely repeated objects up to uniform
scale along X, Y and Z directions, respectively. Section V
describes relationships between different object transformation
matrices and relative affine structures. The results and conclu-
sion with future work are discussed in sections VI and VII,
respectively.
II. OVERVIEW
A. Relative Affine Strcture
In three dimensional space, a plane π and a point M
1
/∈ π
are chosen. Given two views ψ and ψ
0
with projection centers
O and O
0
, H
π
is a homography that transfers image point m
to m
0
, where m and m
0
are image of point M ∈ π [10]. Image
points of M
1
in two views ψ and ψ
0
are m
1
and m
0
1
. These
are related by the following relationship which is derived in
[2]:
m
0
1
∼
=
H
π
m
1
+ ke
0
(1)
where, e
0
is epipole and k is relative affine structure. Above
relation has a scale factor which can be resolved by appropri-
ately scaling H
π
or e
0
such that [2],
m
0
0
∼
=
H
π
m
0
+ e
0
(2)
where m
0
0
and m
0
are images of a fixed point M
0
/∈ π. This
configuration is shown in figure (1). Geometrically, the relative
affine structure is defined as [2],
k =
X
1
λ
1
λ
0
X
0
(3)
where λ
1
and λ
0
are depth of point M
1
and M
0
and X
1
and
X
0
are perpendicular distance of these points from reference
plane π which is Y Z plane in this case. Thus, it is concluded
that:
The relative affine structure is proportional to the perpen-
dicular distance from a reference plane.
The scale factor for Eq. (1) is computed in [3]. It is ratio
of depths of M
1
with respect to projection centers O and O
0
.
λ
0
m
0
1
= λ(H
π
m
1
+ ke
0
N
) (4)
where e
0
N
is normalized epipole. After simplifying it further,
the expression for k is written as [3],
k =
(m
0
1
× e
0
N
)
T
((H
π
m
1
) × m
0
1
)
||(m
0
1
× e
0
N
)||
2
(5)
Let us consider the dual configuration of what is shown in
figure (1). An object S undergoes a transformation T , affine
or translation, and results in object S
0
. In three dimensional
space, a point M ∈ S is related by its corresponding point
M
0
∈ S
0
such as M
0
= T M. The image of M and M
0
in view
ψ with projection center O are m and m
0
. This configuration
can be considered as single image of two repeated objects
with transformation T or two different images of single object
when two cameras undergo same transformation T . This is
called isometry property. The relative affine structures for
corresponding points m and m
0
are k and k
0
. The expression
for k is given by Eq. (5). The expression for k
0
under
translational and affine repetition is given by Eq. (6) and (7),
respectively.
k
0
=
(m × e
0
N
)
T
((H
π
m
0
− 2m
0
) × m)
||(m × e
0
N
)||
2
(6)
k
0
=
(H
∞
m × e
0
N
)
T
((e
0
N
ν
T
πN
m
0
− m
0
) × m)
||(H
∞
m × e
0
N
)||
2
(7)
where ν
T
πN
= e
0
T
N
(H
π
−H
∞
) and H
∞
is infinite homography
between two views [3].
B. Apparatus for Proposed Framework
The chosen world coordinate system for repeated objects is
shown in figure (2). The three orthogonal planes π
Y Z
, π
ZX
and π
XY
are reference planes. For every point M
1
/∈
{π
Y Z
, π
ZX
, π
XY
} will have three relative affine structures,
k
x
, k
y
and k
z
, respectively.
k
x
= X
1
1
λ
λ
0
X
0
, k
y
= Y
1
1
λ
λ
0
X
0
, and k
z
= Z
1
1
λ
λ
0
X
0
(8)
Fig. 1. Geometry: Relative Affine Structure
Fig. 2. Geometry: Three Orthogonal Relative Affine Structures
where, X
0
and λ
0
are X coordinate and depth of a fixed
point M
0
/∈ {π
XY
, π
Y Z
, π
ZX
}. The ratio
λ
0
X
0
is a constant
and denoted by µ
x0
. Similarly, we can write,
k
x
= X
1
1
λ
µ
x0
, k
y
= Y
1
1
λ
µ
y0
, and k
z
= Z
1
1
λ
µ
z0
(9)
Therefore, each expression for k is proportional to the per-
pendicular distance from the chosen reference plane, e.g.
k
x
∝ X
1
, k
y
∝ Y
1
and k
z
∝ Z
1
. There are two constants of
proportionality. First constant is inverse of depth of the point
1
λ
which will vary for every point. Second constant (say µ
x0
)
is fixed for every point with respect to the reference plane (say
π
Y Z
).
The proposed framework use this fundamental concept
behind relative affine structure to determine 3D measurements
of translaionally and affinely repeated objects. We experiment
our framework on a perspective image of repeated cuboids, as
shown in figure (3). Any one cuboid is considered as principal
object and rest as auxiliary objects.
III. MEASUREMENTS OF INDIVIDUAL OBJECT
As described in section II-B, for a point M
i
, a relative affine
structure k
x
has a fixed constant µ
x0
and a variable constant
1
λ
i
. The fixed constant will be eliminated by taking ratio of two
relative affine structures for two different points with respect
to same reference plane, π
Y Z
.
Considering points M
2
= (X
2
0 0) and M
5
= (X
5
0 Z
5
)
as shown in figure (2), ratio of their relative affine structures
is reduced to the following expression,
k
x2
k
x5
=
X
2
X
5
λ
5
λ
2
λ
0
X
0
X
0
λ
0
=
X
2
X
5
λ
5
λ
2
(10)
The values of k
x2
and k
x5
are computed by Eq. (5). The
expression for X
5
can be written as
X
5
= X
2
k
x5
k
x2
λ
5
λ
2
(11)
The depth of points M
2
and M
5
, λ
2
and λ
5
, are computed
using vanishing points based method given in [6]. Given metric
value of X
2
(X
ref
) and metric value of X
5
is computed.
Thus, for any arbitrary point’s X coordinate can be computed
using relative affine structure and projective depth. Similarly,
we can compute the Y and Z coordinates of every point, given
reference metric measurements Y
ref
and Z
ref
along Y and Z
direction, respectively.
Y = Y
ref
k
y
k
yref
λ
λ
ref
and Z = Z
ref
k
z
k
zref
λ
λ
ref
(12)
IV. PRINCIPAL OBJECT AS REFERENCE
Consider a pair of corresponding points M
i
and M
0
i
on
affinely repeated objects S and S
0
. Alternately, we can arbi-
trarily choose a pair of points on two repeated objects. The
respective relative affine structures k
xi
and k
0
xi
for m
i
and m
0
i
can be computed by Eq. (5) and Eq. (7). The ratio of k
xi
and
k
0
xi
can be expressed as follows
k
xi
k
0
xi
=
X
i
X
0
i
λ
0
i
λ
i
λ
i0
λ
0
i0
X
0
i0
X
i0
(13)
k
xi
k
0
xi
=
X
i
X
0
i
λ
0
i
λ
i
ψ
x0
∼
=
X
i
X
0
i
λ
0
i
λ
i
(14)
Since ψ
x0
=
λ
i0
λ
0
i0
X
0
i0
X
i0
is fixed for all points, the ratio
k
xi
k
0
xi
can
be computed up to uniform scale along X-axis. The ratio
λ
0
i
λ
i
can be computed by solving equation (4).
λ
0
i
λ
i
=
||((H
π
m
i
) × e
0
N
)||
||m
0
i
× m
i
||
(15)
Equation (14) can be written as
X
0
i
∼
=
X
i
k
0
xi
k
xi
λ
0
i
λ
i
= X
i
α
xi
(16)
Similarly, we can write expressions for Y and Z directions as
below,
Y
0
i
∼
=
Y
i
k
0
yi
k
yi
λ
0
i
λ
i
= Y
i
α
yi
, Z
0
i
∼
=
Z
i
k
0
zi
k
zi
λ
0
i
λ
i
= Z
i
α
zi
(17)
Given reference measurements on the principal object along
X, Y and Z axes, measurements of affinely repeated object
can be computed up to a respective scale by Eq. (16) and (17).
V. TRANSFORMATION OF REPEATED OBJECT
Under specific configurations, relative affine structure,
which is projective structure, turns into affine structure. If the
reference plane is at infinity or in case of parallel projection,
relative affine structure approaches to affine structure [2]. Ratio
of two relative affine structures of a point with respect to
different reference planes does not depends on the depth. Thus,
that ratio is a projective structure. We have seen that relative
affine structure is proportional to Euclidean distance of a point
from the reference plane. Therefore, relative affine structure
subsumes projective, affine and Euclidean structures [2]. Here,
this statement is analyzed mathematically.
Suppose S and S
0
are repeated objects and are related by
S
0
= T (S), where T is a 4 × 4 general transformation matrix.
A point M
0
∈ S
0
is corresponding to M ∈ S. By using Eq.
(16) and (17), the relation between corresponding points can
be written as follows,
X
0
Y
0
Z
0
1
=
α
x
0 0 0
0 α
y
0 0
0 0 α
z
0
0 0 0 1
X
Y
Z
1
= K
X
Y
Z
1
(18)
where K is the transformation matrix in 3D Euclidean space.
A. Translational Repetition
If object S and S
0
are related by pure translation, the
transformation is represented as,
X
0
Y
0
Z
0
1
=
1 0 0 T
x
0 1 0 T
y
0 0 1 T
z
0 0 0 1
X
Y
Z
1
(19)
From Eq. (18) and (19),
X
0
= X + T
x
= Xα
x
⇒ T
x
= X(α
x
− 1) (20)
Similarly,
T
y
= Y (α
y
− 1), T
z
= Z(α
z
− 1) (21)
Therefore, translation vector can be represented in terms of
three relative affine structures with respect to uniform scale
along X, Y and Z axes, respectively.
B. Pure Rotational Repetition
If object S undergoes pure rotation and results in S
0
, the
transformation is represented as,
X
0
Y
0
Z
0
=
r
1
r
2
r
3
r
4
r
5
r
6
r
7
r
8
r
9
X
Y
Z
=
α
x
0 0
0 α
y
0
0 0 α
z
X
Y
Z
Therefore, the rotation between two points is equivalent to the
following diagonal matrix
R =
α
x
0 0
0 α
y
0
0 0 α
z
where three columns are scaled uniformly along X, Y and Z
directions, respectively.
C. Affine Repetition
The most general case is affine repetition that encapsulates
rotation, translation, scaling and shearing [10]. This trans-
formation in 3D Euclidean space is equivalent to K. Once
constants ψ
x0
, ψ
y0
and ψ
z0
along X, Y and Z directions are
computed from image (coordinates of M
0
), affine repetition
can be computed uniquely.
Fig. 3. Repeated Objects
VI. RESULTS
In our experiments, we consider a real image, as shown in
figure (3). It has objects with affine repetition. Table I and
II display the measurements (centimeter) of objects computed
using methods discussed in sections III and IV, respectively.
Based on the precision required for an application, the errors
can be further reduced by employing efficient techniques
for computing point correspondences and vanishing points
from image. Additionally, proper uncertainly analysis will also
improve the results [6].
TABLE I
MEASUREMENTS - INDIVIDUAL OBJECT
ID Source X5 Z5 Y6 Z6 X7 Y7 Z7
Scene 3.1 5.1 3.1 5.1 3.1 3.1 5.1
1
Image 3.3 5.0 3.3 4.8 3.4 3.5 4.6
Error -0.2 0.1 -0.2 0.3 -0.3 -0.4 0.5
Scene 3.1 5.1 3.1 5.1 3.1 3.1 5.1
2
Image 2.3 5.3 2.1 4.2 2.1 2.1 4.3
Error 0.8 -0.2 1.0 0.9 1.0 1.0 0.8
Scene 2.6 6.0 2.6 6.0 2.6 2.6 6.0
3
Image 2.4 6.1 1.5 5.7 2.5 1.3 5.8
Error 0.2 -0.1 1.1 0.3 0.2 1.3 0.2
TABLE II
MEASUREMENTS - PRINCIPAL (I
st
) OBJECT AS REFERENCE
ID Source X5 Z5 Y6 Z6 X7 Y7 Z7
Scene 3.1 5.1 3.1 5.1 3.1 3.1 5.1
2
Image 2.3 5.3 3.5 4.2 2.1 3.5 4.3
Error 0.8 -0.2 -0.4 0.9 1.0 -0.4 0.8
Scene 2.6 6.0 2.6 6.0 2.6 2.6 6.0
3
Image 2.4 5.9 3.2 5.2 2.6 2.9 5.1
Error 0.2 -0.1 -0.8 0.8 0.0 -0.3 0.9
VII. CONCLUSION
We extended prior work on relative affine structure for
computing three dimensional measurement from a single
perspective image of repeated objects. The transformation
between repeated objects can be represented in terms of
relative affine structures along three orthogonal directions.
Therefore, one invariant is used to analyze projective, affine
and Euclidean space for vision tasks. Camera transformation
for repeated object can also be expressed in terms of relative
affine structures.
Furthermore, three dimensional motion of an object or
a camera can be parameterized in terms of relative affine
structure. So, motion analysis related tasks such as motion
segmentation and tracking can use relative affine structure, an
invariant.
ACKNOWLEDGMENT
This work has been supported under the research grant,
towards the setting up of the ‘Program on Autonomous
Robotics’, from the Board of Research in Nuclear Sciences
(BRNS), India.
REFERENCES
[1] Y. Cao and J. McDonald, “Viewpoint invariant features from single
images using 3d geometry,” in Applications of Computer Vision (WACV),
2009 Workshop on, 2009, pp. 1–6.
[2] A. Shashua and N. Navab, “Relative affine structure: canonical model for
3d from 2d geometry and applications,” Pattern Analysis and Machine
Intelligence, IEEE Transactions on, vol. 18, no. 9, pp. 873–883, 1996.
[3] R. Choudhury, J. B. Srivastava, and S. Chaudhury, “Reconstruction-
based recognition of scenes with translationally repeated quadrics,”
Pattern Analysis and Machine Intelligence, IEEE Transactions on,
vol. 23, no. 6, pp. 617–632, 2001.
[4] A. Shashua and N. Navab, “Relative affine structure: theory and ap-
plication to 3d reconstruction from perspective views,” in Computer
Vision and Pattern Recognition, 1994. Proceedings CVPR ’94., 1994
IEEE Computer Society Conference on, 1994, pp. 483–489.
[5] S. Tebaldini, M. Marcon, A. Sarti, and S. Tubaro, “Uncalibrated view
synthesis from relative affine structure based on planes parallelism,” in
Proceedings of the International Conference on Image Processing, ICIP
2008, October 12-15, 2008, San Diego, California, USA. IEEE, 2008,
pp. 317–320.
[6] A. Criminisi, “Accurate visual metrology from single and multiple
uncalibrated images,” Ph.D. dissertation, University of Oxford, Dept.
Engineering Science, 1999, d.Phil. thesis.
[7] A. Criminisi, I. Reid, and A. Zisserman, “Single view metrology,” Int.
J. Comput. Vision, vol. 40, no. 2, pp. 123–148, Nov. 2000.
[8] K. Peng, L. Hou, R. Ren, X. Ying, and H. Zha, “Single view
metrology along orthogonal directions,” in Proceedings of the 2010
20th International Conference on Pattern Recognition, ser. ICPR ’10.
Washington, DC, USA: IEEE Computer Society, 2010, pp. 1658–1661.
[Online]. Available: http://dx.doi.org/10.1109/ICPR.2010.410
[9] G. Wang, Z. Hu, F. Wu, and H.-T. Tsui, “Single view
metrology from scene constraints,” Image Vision Comput.,
vol. 23, no. 9, pp. 831–840, Sep. 2005. [Online]. Available:
http://dx.doi.org/10.1016/j.imavis.2005.04.002
[10] R. I. Hartley and A. Zisserman, Multiple View Geometry in Computer
Vision, 2nd ed. Cambridge University Press, ISBN: 0521540518, 2004.
[11] R. Cipolla, T. Drummond, and D. Robertson, “Camera Calibration from
Vanishing Points in Images of Architectural Scenes,” in Proc. British
Machine Vision Conference. Nottingham, UK: British Machine Vision
Association, 1999, pp. 382–391.
[12] E. Guillou, D. Meneveaux, E. Maisel, and K. Bouatouch, “Using
vanishing points for camera calibration and coarse 3d reconstruction
from a single image,” The Visual Computer, vol. 16, no. 7, pp. 396–410,
2000. [Online]. Available: http://dx.doi.org/10.1007/PL00013394