![Fig. 1. Comparison between experimental (dashed-blue) and analytical uncertainty bounds of the plane homography coefficients for various noise levels and image correspondences (Chc in green, and Cho in red). In each case, the experimental homography represents 1000 simulations with different noise samples of variance σm and N point correspondences. From top to bottom, (1) σ = 1 [pix] with N = 20; (2) σ = 0.05[pix] with N = 20; (3) σ = 1/3[pix] with N = 4; (4) σ = 1[pix] with N = 4.](/figures/fig-1-comparison-between-experimental-dashed-blue-and-mv9rh4ji.webp)
PLANAR HOMOGRAPHY: ACCURACY ANALYSIS AND APPLICATIONS
Shahriar Negahdaripour
∗
, Ricard Prados, Rafael Garcia
Computer Vision and Robotics Group
Dept. of Electronics, Informatics and Automation
University of Girona, Spain
ABSTRACT
Projective homography sits at the heart of many problems in
image registration. In addition to many methods for estimat-
ing the homography parameters [5], analytical expressions
to assess the accuracy of the transformation parameters have
been proposed [4]. We show that these expressions provide
less accurate bounds than those based on the earlier results
of Weng et al. [7]. The discrepancy becomes more critical in
applications involving the integration of frame-to-frame ho-
mographies and their uncertainties, as in the reconstruction
of terrain mosaics and the camera trajectory from flyover im-
agery. We demonstrate these issues through selected exam-
ples.
1. INTRODUCTION
The registration of various frames in a video mosaic has nu-
merous applications, including generation of terrain mosaics
from flyover image transects in underwater and airborne sys-
tems [3]. The abilities to accurately compute the view-to-
view transformation parameters and to assess their accura-
cies are equivalently important issues. For example, repro-
jection error bounds can be used to establish search windows
in processing of the data recursively to improve the image
matching and registration.
Let I and I
0
be the images of a scene from two camera
viewpoints. Under many conditions and for a number of
applications, it may be assumed that the scene is (approxi-
mately) planar, and thus the image transformation I → I
0
can be described by a planar projective homography. Given
a pair of image correspondences {p = (x, y, 1)
T
and p
0
=
(x
0
, y
0
, 1)
T
in I and I
0
, respectively, the projective homog-
raphy H establishes the constraint between each correspon-
dence according to ρ
0
p
0
= Hp, where ρ
0
is a constant scaling.
A number of methods, including closed-form linear methods,
have been given to estimate H (up to a scale) from N ≥ cor-
respondences {p
i
, p
0
i
}, i = 1 : N. Assuming that the image
position measurements are corrupted by Gaussian noise, Cri-
minisi et al. [4] give closed-form expressions to estimate the
variance of the 8 independent parameters
1
. The derivation is
∗
S. Negahdaripour’s permanent address: Dept. Electrical and Computer
Engineering, University of Miami, Coral Gables, FL 33143. This paper
describes work while on sabbatical leave at the University of Girona, Spain.
1
Nine elements determined up to a constant scale factor.
claimed to be based on the earlier results of Weng et al. [7],
who reported a comprehensive error analysis of motion and
structure parameters from image correspondences. Criminisi
et al. also stated that their results, being better conditioned
than solutions given in [2] if only N = 4 correspondences
are used, also provide better estimates when the N > 4
matchings have relatively small measurement noise levels.
We will show that these expressions provide less accurate
bounds than those derived from a singular value perturbation
analysis, as first proposed by Weng et al. [7]. We also explore
the behavior of the uncertainty bounds in applications involv-
ing the integration of frame-to-frame homographies, as in the
reconstruction of mosaics and the camera trajectory from fly-
over imagery.
2. TERMINOLOGY AND BACKGROUND
Let H denote the planar homography that maps points p in I
onto p
0
in I
0
based on up-to-scale transformation p
0
= Hp.
For simplicity, we express the up-to-scale homography as
2
H =
h
1
h
2
h
3
h
4
h
5
h
6
h
7
h
8
1
If h denotes the vector form of H, the above homograhy
can be expressed by two linear constraint equations on h.
If we have N correspondences {p
i
, p
0
i
} (i = 1 : N), 2N
constraint equations can be written Ah = 0, where
A=
x
1
y
1
1 0 0 0 −x
0
1
x
1
−x
0
1
y
1
−x
0
1
0 0 0 x
1
y
1
1 −y
0
1
x
1
−y
0
1
y
1
−y
0
1
.
.
.
x
N
y
N
1 0 0 0 −x
0
N
x
N
−x
0
N
y
N
−x
0
N
0 0 0 x
1
y
1
1 −y
0
N
x
N
−y
0
N
y
N
−y
0
N
Let A = UΣΣV
T
denote the singular value decomposition of
A. Equivalently, we consider the eigenvalue decomposition
Q = A
T
A = VD
2
V
T
, where ΣΣ = D
2
. The linear solution
of the 9×1 unit homography vector
ˆ
h is the eigenvector as-
sociated with the smallest singular value:
ˆ
h = υυ
9
(diagonal
elements of ΣΣ are arranged in descending order, and υυ
i
is
the i-th column of V). Subsequently, h can be determined
by scaling: h =
ˆ
h/
ˆ
h
9
. We are interested in the variations of
2
This representation assumes that h
9
6= 0.
0-7803-9134-9/05/$20.00 ©2005 IEEE
Authorized licensed use limited to: UNIVERSITAT DE GIRONA. Downloaded on April 26,2010 at 10:40:19 UTC from IEEE Xplore. Restrictions apply.
the solution with noisy observations. Define the observation
vector m of N left-and-right matching pixel coordinates:
m = [x
1
, y
1
, x
0
1
, y
0
1
, x
2
, y
2
, x
0
2
, y
0
2
, . . . , x
N
, y
N
, x
0
N
, y
0
N
]
4N×1
The noisy measurement vector
ˆ
m = m + δm comprises
zero-mean Gaussian noise vector δm with covariance C
m
.
A simplified assumption is a normal distribution(0, σ); that
is, C
m
= σ
2
I
4N×4N
, which is reasonable when the errors
are primarily due to quantization effects and the localization
inaccuracies of the feature detector, e.g. Harris interest point
operator [1]. Outliers violate the assumed normal noise dis-
tribution model. However, it is typically the case that some
robust estimation method, e.g. RANSAC or LMedS, can be
used as a first step to identify the outliers, before the pro-
posed linear solution is applied to the inliers (which satisfy
the assumed noise model). Here, we assume knowledge of σ,
and are interested to estimate the variances of h
i
(i = 1 : 8).
3. COVARIANCE OF PROJECTIVE HOMOGRAPHY
Due to space limitation, we refer the reader to section 5 in [4],
given for the estimation of the covariance of the homography
parameters. This is denoted C
hc
here. We also derive C
ho
,
the covariance derived here as the new estimate.
From δm, we can determine the variation in the measure-
ment matrix A
2N×9
, or Q
9×9
. Assume that Q is perturbed
by ∆Q, where q
81×1
and δq
81×1
are the corresponding vec-
tor forms. For small variations –max{δq
i
} << 1, where
q
i
denotes i-th element of q –it can be shown [6, 7] that up
to first-order, the eigenvalues and eigenvectors ofQ vary ac-
cording to δλ
i
= υυ
T
i
∆Q υυ
i
and δυυ
i
= VΨ
i
V
T
ΠΠ
i
δq where
Ψ
i
= diag{(λ
i
− λ
1
)
−1
, . . . , 0, . . . , (λ
i
− λ
9
)
−1
}
ΠΠ
i
=
υ
i1
I
9×9
υ
i2
I
9×9
. . . υ
i9
I
9×9
The covariances of the eigenvectors C
υ
i
=VΨ
i
V
T
ΠΠ
i
C
q
ΠΠ
T
i
V
Ψ
T
i
V
T
allows us to write
C
ˆ
h
= C
υ
9
= VΨ
9
V
T
ΠΠ
9
C
q
ΠΠ
T
9
VΨ
T
i
V
T
Recalling that the homography h is determined from
ˆ
h by
scaling (such that h
9
= 1) the covariances of h and
ˆ
h are
related by the transformation C
ho
= J C
ˆ
h
J
T
where
J =
∂h
∂
ˆ
h
=
1/h
9
0 . . . 0 −h
1
/h
2
9
0 1/h
9
. . . 0 −h
1
/h
2
9
.
.
.
0 0 . . . 1/h
9
−h
8
/h
2
9
8×9
In the above equations, C
q
is determined from
C
q
=
∂q
∂a
"
∂a
∂m
C
m
∂a
∂m
T
#
∂q
∂a
T
where a is the vector form of A:
a = [ x
1
, y
1
, 1, 0, 0, 0, −x
0
1
x
1
, −x
0
1
y
1
, −x
0
1
,
0, 0, 0, x
1
, y
1
, 1, −y
0
1
x
1
, −y
0
1
y
1
, −y
0
1
, . . .
0, 0, 0, x
N
, y
N
, 1, −y
0
N
x
N
, −y
0
N
y
N
, −y
0
N
]
18N×1
4. SIMULATIONS
We use computer simulations to compare the closed-form ex-
pressions for estimating the variances of the projective ho-
mography parameters, given in [4] and derived here (denoted
C
hc
and C
ho
, respectively). In all but two tests, we start
with N = 20 points p = [x(i, j), y(i, j), f] on a regular grid
i = 64 : 64 : 320 and j = 64 : 64 : 256 (f = 320 is
assumed). In these two our simulations, only a minimum 4
points near the image corners are used. We construct match-
ing pairs {p, p
0
} based on a pre-specified homographyH; we
use the well-know interpretation H = R + tn
T
in terms of
the motion {R, t} of a camera relative to a planar scene with
surface normal n = [−P, −Q, 1]/Z
o
, where P and Q control
the surface slant and tilt angles, and Z
o
its distance from the
camera. Measurements noise from a normal distribution with
standard deviation σ
m
is then added to corresponding pairs to
subsequently estimate the homography, and calculate its er-
ror. The process is repeated 1000 times with different noise
samples to compute statistical measures. In case 1, the ex-
perimental variance of each parameter in h
i
(i = 1 : 8) is
compared with the analytical estimates σ
hk
=
p
diag{C
hk
}
(k = c, o). In the second case, the estimated noisy homo-
graphies map certain corners of a real image into 1000 noisy
matches in the 2nd view. The final experiment deals with
the generation of a 10-frame sequence, and the integration
of frame-to-frame homographies and the corresponding vari-
ances.
4.1. Case 1
In the fist simulation, we have usedP = 0.2, Q = −0.2, and
Z
o
= 10, with translational motion t = [0.1, 0.2, 0.05] and
rotational matrix R = R
x
R
y
R
z
, where R
a
denotes a rotation
about axis a with angle θ
a
(θ
x
= 0.05 [rad], θ
y
= 0.05 and
θ
y
= 0).
Fig. 1 (top) shows the results for σ
m
= 1 with N = 20
correspondences (horizontal axis corresponds to homogra-
phy parameters h
i
(i = 1 : 8), and the vertical axis is the
estimation error). Blue crosses depict the errors of the ho-
mography parameters for each of 1000 simulations, with red
circles giving the (zero) mean error. The dashed blue en-
velop is the ±3σ error bound computed experimentally, and
the other two envelops in green and red are derived from an-
alytical bounds ±3σ
hc
and ±3σ
ho
, respectively. The latter
nearly coincides with the experimental results. In [4], the
authors claim that their solution provides better estimates in
two cases: 1) Relatively small measurement noise levels, or
2) when minimum N = 4 image correspondences are uti-
lized in the estimation of the homography. These cases are
tested in the next three plots corresponding to σ
m
= 0.05
with N = 20 σ
m
= 1/3 with N = 4, and σ
m
= 1 with
N = 4. In all cases, the new results consistently provide a
more accurate estimate of the ±3σ
h
error bounds.
Authorized licensed use limited to: UNIVERSITAT DE GIRONA. Downloaded on April 26,2010 at 10:40:19 UTC from IEEE Xplore. Restrictions apply.
4.2. Case 2
Fig. 2 shows the original and transformed images based on
an assumed homography. In addition, selected corners have
been mapped with 1000 homographies, estimated from noisy
correspondences (σ
m
= 1 is assumed). The green and red
uncertainty ellipses of the transformed points have been de-
termined from the homography variances σ
hc
and σ
ho
, re-
spectively. The results for 4 selected points, A − D, demon-
strate once again that σ
ho
provides a more accurate and tighter
uncertainty bound.
4.3. Case 3
A 10-frame sequence has been constructed based on a pre-
scribed homography. Fig. 3 shows the sequence with an
inter-frame motion of about 13-14 pixels. The blue stars de-
pict the true positions of 3 sample pixels in various frames, at
the center of uncertainty ellipses (computed from the two dif-
ferent techniques) that bound noisy positions of these points
based on homographies estimated from 1000 simulations with
noisy correspondences. For each of these three points, the
noisy positions and bounding ellipses in frames 1, 4, 7 and
10 have been given in subsequent plots. As in previous 2
cases, our new results provide a tighter fit of the projected
point distributions.
5. SUMMARY
Ability to not only estimate the transformation between frames
but also to assess the confidence in these estimates is impor-
tant in many applications involving motion estimation from
video imagery. Computation of projective homography from
frame-to-frame correspondences has been extensively stud-
ied in recent years [5], and analytical uncertainty bounds
of the homography parameters and reprojection errors have
been proposed [4]. Based on earlier results of Weng et al. [7],
we have provided new expressions to estimate these bounds
more accurately. This has been verified in a number of ex-
periments based on the estimation of homography parame-
ters, as well as the reprojection of image points based on
the estimated homographies. These results are particularly
important when dealing with long image sequences where
frame-to-frame estimates need to be integrated to establish
the camera position, to build an image mosaic, or generally
to register various frames of a video sequence. Under inves-
tigation is the direct use of these uncertainty bounds in the
construction of photo-mosaics.
Acknowledgement: This work has been funded in part by the Span-
ish Ministry of Education and Science under grant CTM2004-04205,
and in part by the Generalitat de Catalunya under grant 2003PIV-
B00032.
6. REFERENCES
[1] D.P. Capel, “Image mosaicing and super-resolution,” Ph.D.
Thesis, Dept. of Engineering Science, Univ. of Oxford, 2001.
1 2 3 4 5 6 7 8
−0.05
−0.04
−0.03
−0.02
−0.01
0
0.01
0.02
0.03
0.04
0.05
Coefficients h
i
of the homography
Mean and 3 sd of errors
1 2 3 4 5 6 7 8
−2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
x 10
−3
Coefficients h
i
of the homography
Mean and 3 sd of errors
1 2 3 4 5 6 7 8
−0.02
−0.015
−0.01
−0.005
0
0.005
0.01
0.015
0.02
Coefficients h
i
of the homography
Mean and 3 sd of errors
1 2 3 4 5 6 7 8
−0.05
−0.04
−0.03
−0.02
−0.01
0
0.01
0.02
0.03
0.04
0.05
Coefficients h
i
of the homography
Mean and 3 sd of errors
Fig. 1. Comparison between experimental (dashed-blue) and
analytical uncertainty bounds of the plane homography coef-
ficients for various noise levels and image correspondences
(C
hc
in green, and C
ho
in red). In each case, the experi-
mental homography represents 1000 simulations with differ-
ent noise samples of variance σ
m
and N point correspon-
dences. From top to bottom, (1) σ = 1 [pix] with N = 20;
(2) σ = 0.05[pix] with N = 20; (3) σ = 1/3[pix] with
N = 4; (4) σ = 1[pix] with N = 4.
Authorized licensed use limited to: UNIVERSITAT DE GIRONA. Downloaded on April 26,2010 at 10:40:19 UTC from IEEE Xplore. Restrictions apply.
50 100 150 200 250 300 350
50
100
150
200
250
300
A
B
C
D
102 104 106 108 110 112
26
27
28
29
30
31
32
33
34
35
90 95 100 105
192
194
196
198
200
202
204
206
208
210
345 350 355 360 365 370 375 380 385 390
135
140
145
150
155
160
165
170
290 295 300 305 310 315 320 325 330 335
270
275
280
285
290
295
300
305
Fig. 2. (top) Two views transformed according to homogra-
phy H
3
superimposed on two color channels. Selected cor-
ners are mapped based on homographies that are calculated
from noisy image features (with variance σ
m
= 1), and re-
peated 1000 times with different noise samples. For each
corner, the analytical 3σ uncertainty ellipse is given based
on C
hc
(green) and C
ho
(red). (bottom) Results have been
magnified for 4 selected cornersA, B, C and D.
[2] J.C. Clarke, “Modelling uncertainty: A primer”, Tech. Report
2161/98, Univ. of Oxford, Dept. of Engineering Science, 1998.
[3] R. Garcia, X. Cufi, and V. Ila, “Recovering Camera Motion in
a Sequence of Underwater Images through Mosaicking”, LNCS
no. 2652, pp. 255-262, Eds. Springer-Verlag, 2003.
[4] A. Criminisi, I. Reid, and A. Zisserman, “A Plane Measuring
Device,” Image and Vis. Comp. vol. 17(8) pp. 625–634, 1999.
[5] R.I Hartley, and A. Zisserman, Multiple View Geometry in
Computer Vision, Cambridge Univ. Press, 2000.
[6] Z. Sun, V. Ramesh, and A.M. Tekalp, “Error characterization
of the factorization method,” CVIU, vol. 82(2), 2001.
[7] J. Weng, T. S. Huang, and N. Ahuja, “Motion and Structure
from Two Perspective Views: Algorithms, Error Analysis, and
Error Estimation,” IEEE Trans. on Patt. Anal. and Mach. Int.,
vol. 11(5), pp.451–476, 1989.
−150 −100 −50 0 50 100 150 200 250
−150
−100
−50
0
50
100
150
200
28.6 28.8 29 29.2 29.4 29.6 29.8 30 30.2
60.2
60.4
60.6
60.8
61
61.2
61.4
55.5 56 56.5 57 57.5 58
63.5
64
64.5
65
65.5
80.5 81 81.5 82 82.5 83 83.5 84 84.5
68.5
69
69.5
70
70.5
71
71.5
104 105 106 107 108 109
75
75.5
76
76.5
77
77.5
78
78.5
79
79.5
80
Point (20,60)
−87 −86.5 −86 −85.5 −85
−51.4
−51.2
−51
−50.8
−50.6
−50.4
−50.2
−50
−49.8
−49.6
−47.5 −47 −46.5 −46 −45.5 −45 −44.5 −44 −43.5 −43
−51
−50.5
−50
−49.5
−49
−48.5
−48
−9 −8.5 −8 −7.5 −7 −6.5 −6 −5.5 −5 −4.5
−47.5
−47
−46.5
−46
−45.5
−45
−44.5
−44
27 28 29 30 31 32 33
−42
−41.5
−41
−40.5
−40
−39.5
−39
−38.5
−38
−37.5
Point (-100,-50)
106 106.5 107 107.5 108 108.5
101.2
101.4
101.6
101.8
102
102.2
102.4
102.6
102.8
103
125.5 126 126.5 127 127.5 128 128.5 129 129.5 130 130.5
107.5
108
108.5
109
109.5
110
110.5
111
111.5
144 145 146 147 148 149 150
115.5
116
116.5
117
117.5
118
118.5
119
119.5
120
120.5
162 163 164 165 166 167 168 169
125
126
127
128
129
130
Point (100,100)
Fig. 3. Estimated positions of three sample points in a
10-frame sequence based on homographies estimated from
noisy correspondences, repeating the experiment 1000 times.
Analytical uncertainty ellipses have been given for C
hc
(green) and C
ho
. For each sample point, the estimated po-
sitions and ellipses in frames 1, 4, 7, 10 are given in a 2 × 2
plot.
Authorized licensed use limited to: UNIVERSITAT DE GIRONA. Downloaded on April 26,2010 at 10:40:19 UTC from IEEE Xplore. Restrictions apply.