Proc. CVPR{96, San Francisco, California, June 16{21, 1996.
Longer version available as Tech. rep. ISRN KTH/NA/P{96/06{SE.
Edge Detection and Ridge Detection with Automatic Scale Selection
Tony Lindeberg
Computational Vision and ActivePerception Laboratory (CVAP)
KTH (Royal Institute of Technology), Stockholm, Sweden
Abstract:
When extracting features from image data, the
type of information that can be extracted may be strongly
dependent on the scales at which the feature detectors
are applied. This article presents a systematic metho dol-
ogy for addressing this problem. A mechanism is presented
for
automatic selection of scale levels
when detecting one-
dimensional features, such as edges and ridges.
Anovel concept of a
scale-spaceedge
is introduced, de-
ned as a connected set of points in scale-space at which:
(i) the gradient magnitude assumes a local maximum in
the gradient direction, and (ii) a normalized measure of the
strength of the edge response is locally maximal over scales.
An imp ortant property of this denition is that it allows the
scale levels to vary along the edge.
Two sp ecic measures of edge strength are analysed in
detail. It is shown that by expressing these in terms of
-
normalized derivatives
, an immediate consequence of this
denition is that
ne scales
are selected for
sharp edges
(so as to reduce the shap e distortions due to scale-space
smoothing), whereas
coarse scales
are selected for
diuse
edges
, such that an edge model constitutes a valid abstrac-
tion of the intensity prole across the edge.
With slight modications, this idea can be used for for-
mulating a ridge detector with automatic scale selection,
having the characteristic property that the selected scales
on a
scale-spaceridge
instead reect the
width
of the ridge.
1 Intro duction
One of the most intensively studied subproblems in
computer vision concerns how to detect edges from
image data. The importance of edge information for
early machine vision is usually motivated from the ob-
servation that under rather general assumptions ab out
the image formation process, a discontinuityinimage
brightness can be assumed to correspond to a discon-
tinuity in either depth, surface orientation, reectance
or illumination.
A non-trivial aspect of edge based analysis of im-
age data, however, concerns what should be meantby
a discontinuity in image brightness. Real-world image
data are inherently discrete, and for a function dened
The support from the Swedish Research Council for Engi-
neering Sciences, TFR, is gratefully acknowledged.
Address:
KTH, NADA, S-100 44 Stockholm, Sweden
Email:
tony@bion.kth.se
WWW:
http://www.bion.kth.se/~tony
on a discrete domain, there is no natural notion of a
\discontinuity". This means that there is no inherent
way to judge what are the edges in a given discrete
image. Therefore, the concept of an image edge is only
what we dene it to be
.
Since the pioneering work by Roberts (1965), a
large numb er of approaches have b een developed for
detecting edges. Early schemes, such as the Sob el op-
erator and the Prewitt op erator, focused on the de-
tection of p oints at which the gradient magnitude was
high, and computed derivative approximations either
directly from the pixels or using local least-squares t-
ting (Haralick 1984). Torre and Poggio (1980) as well
as Marr and Hildreth (1980) motivated the need for a
smoothing operator to precede dierentiation, Canny
(1986) considered the problem of determining an op-
timal smoothing lter of nite supp ort constituting
the \best" trade-o b etween detection and localiza-
tion prop erties, given a constraint on the probability
of obtaining multiple responses to a single edge. De-
riche (1987) extended this approach to lters with in-
nite supp ort. Similar concepts were developed by Korn
(1988). Bergholm (1987) proposed to track edges from
coarse to ne scales.
Today, one example of a state-of-the-art edge detec-
tor consists of pre-smoothing the image by a Gaussian
kernel followed by non-maximum suppression. The lat-
ter corresp onds to detecting points at which the gra-
dient magnitude assumes a maximum in the gradient
direction, and can be given either an algorithmic or dif-
ferential geometric denition. In this way, edges can be
detected at any scale in scale-space (Lindeb erg 1994).
The sub ject of this article is to extend the ab ove-
mentioned ideas
to include the scale dimension already
in the edge denition
,tosimultaneously allow for au-
tomatic determination of scale levels appropriate for
extracting a given edge. To illustrate the need for such
amechanism, g. 1 shows the result of computing edges
from an image at a number of dierent scales. As can
be seen, dierenttypes of edge structures give rise to
edge curves at dierent scales. For example, the shadow
of the arm only app ears as a connected edge curveat
coarse scales. If such coarse scales are used at the nger
tip, however, the shap e distortions due to scale-space
1
Figure 1:
Edges computed at dierent scales in scale-space
(
t
=1
:
0, 4
:
0 and 16
:
0 from top to bottom) using a dier-
ential geometric formulation of non-maximum suppression.
(Image size: 256*256 pixels.)
smoothing will be substantial. Hence, to extract this
edge with a reasonable trade-o between detection and
localization prop erties, the only reasonable choice is to
al low the scale levels to vary along the edge
.
For this reason, and in view of the fact that the
choice of scale levels crucially aects the performance
of any edge detector, and dierent scale levels will, in
general, be required in dierent parts of the image, we
argue that it is essential to
complement edge detectors
by explicit mechanisms which automatically adapts the
scale levels to the local image structure
.
2 Principle for scale selection
The scale-space representation
L
of a given signal
f
is obtained byconvolving
f
by Gaussian kernels
g
of
various widths
t
=
2
(Witkin 1983, Koenderink 1984,
Florack
et al.
1992, Lindeberg 1994). From this repre-
sentation, a scale-space derivative is dened by
L
x
y
(
;
t
)=
@
x
y
(
g
(
;
t
)
f
)=
g
x
y
(
;
t
)
f:
The output from these operators can b e used as a ba-
sis for expressing a large number of visual operations,
such as feature detection, matching, and computation
of shape cues. A particularly convenient framework for
expressing these is in terms of multi-scale dierential
invariants or singularities of these.
A basic problem for anysuch feature detector, how-
ever, concerns at what scales the image features should
be extracted, and what image features should be re-
garded as signicant. Early work addressing this prob-
lem was presented in (Lindeberg 1993a) for blob-like
image structures. Then, in (Lindeb erg 1993b, 1994)
an extension was presented to other aspects of image
structure. A general heuristic principle was proposed
stating that
local maxima over scales
of (p ossibly non-
linear) combinations of
normalized derivatives
,
@
=
p
t@
x
;
(1)
serveas
indicators reecting the spatial extent of cor-
responding image structures
. Specically,itwas sug-
gested that this idea could b e used as a guide for scale
selection algorithms, which automatically adapt the lo-
cal scale of pro cessing to the lo cal image structure.
The sub ject of this article is to develop in more
detail how this scale selection principle applies to the
detection of one-dimensional image features, suchas
edges and ridges. For reasons that will become apparent
later, we shall also extend this notion to scale selection
based on
-parameterized normalized derivatives
@
x;
;
norm
=
t
=
2
@
x
:
(2)
3 Scale selection for edge detection
Atany image point, introduce a local coordinate sys-
tem (
u; v
) with the
v
-axis parallel to the gradient direc-
tion at that p oint, and the
u
-direction is perpendicular.
Then, at any scale in scale-space, an edge p ointcan
be dened as a p ointat which the second directional
derivative
L
vv
in the
v
-direction is zero, and the third
directional derivative
L
vvv
is negative:
L
vv
=0
;
L
vvv
<
0
;
(3)
If this denition is applied at all scales in scale-space,
it sweeps out an
edge surface
. In view of the scale selec-
tion principle reviewed in the previous section, a natu-
ral extension of non-maximum suppression is by den-
ing a
scale-spaceedge
as a curve on this surface, such
that some suitably selected measure of edge strength
E
;
norm
L
assumes locally maxima with resp ect to scale
on this curve. In dierential geometric terms, a scale-
space edge is thus dened as a connected set of p oints
f
(
x; y
;
t
)
2
R
2
R
+
g
(a curve ;) that satises
8
>
>
<
>
>
:
@
t
(
E
;
norm
L
(
x; y
;
t
)) = 0
;
@
tt
(
E
;
norm
L
(
x; y
;
t
))
<
0
;
L
vv
(
x; y
;
t
)=0
;
L
vvv
(
x; y
;
t
)
<
0
:
(4)
Of course, there are several ways of expressing the con-
dition that
E
;
norm
L
should assume lo cal maxima over
scales on the edge curve. In (4), this condition is formu-
lated as in terms of the partial derivatives of
E
;
norm
L
with resp ect to the scale parameter. A natural alterna-
tive is to consider a directional derivative in the tangent
plane to the edge surface, and to choose this direction
as the steepest ascent direction of the scale parameter.
What remains to turn this idea into an opera-
tionally useful denition is to dene the measure of
edge strength. Based on the
-parameterized normal-
ized derivative concept in (2), we shall here consider
the following two dierential expressions:
G
;
norm
L
=
t
(
L
2
x
+
L
2
y
)
;
T
;
norm
L
=
;
t
3
(
L
3
x
L
x
3
+3
L
2
x
L
y
L
x
2
y
+3
L
x
L
2
y
L
xy
2
+
L
3
y
L
y
3
)
:
The rst entity, the square gradient magnitude, is the
presumably simplest measure of edge strength to think
of. The second entity originates from the sign condition
in the edge denition. These entities are both useful in
practice, but have slightly dierent properties.
Qualitative properties.
For a
diuse step edge
, dened
as the primitive function of a one-dimensional Gaus-
sian,
f
t
0
(
x; y
)=
R
x
x
0
=
;1
g
(
x
0
;
t
0
)
dx
0
,eachofthese
measures of edge strength assume a unique maximum
over scales at
t
G
;
norm
=
t
T
;
norm
=
1
;
t
0
(for
x
= 0).
Requiring this maximum to o ccur at
t
0
gives
=
1
2
.
For a
Gaussian blob
with
L
(
x; y
;
t
)=
g
(
x; y
;
t
0
+
t
)
with edges at
x
2
+
y
2
=
t
0
+
t
, the selected scales are
t
G
;
norm
=
3
;
t
0
and
t
T
;
norm
=
6
13
;
6
t
0
.
Finally, for a local mo del of an
edge bifurcation
,
expressed as
L
(
x
;
t
)=
1
4
x
4
+
3
2
x
2
(
t
;
t
b
)+
3
4
(
t
;
t
b
)
2
,
with edges at
x
1
(
t
)=(
t
b
;
t
)
1
=
2
when
t
t
b
,wehave
(
G
;
norm
L
)(
x
1
(
t
);
t
)=4
t
(
t
b
;
t
)
3
, and the selected
scales are
t
G
;
norm
=
3+
t
b
and
t
T
;
norm
=
3
5+3
t
b
.
In summary, this short investigation shows that the
scale selection method has the qualitative propertyof
reecting the degree of diuseness of the edge
. More-
over, since the edge strength decreases rapidly at a bi-
furcation, this prevents the selected scale from being
too close to the bifurcation scale.
Figure 2:
Results of edge detection with automatic scale
selection based on lo cal maxima over scales of
G
;
norm
L
(with
=
1
2
). Image size: 256
256 pixels.
Figure 3:
The 50 and 10 most signicant edges from the
arm image as ranked on the integrated
-normalized gradi-
ent magnitude along the scale-space edge.
4 Exp eriments: Edge detection
Let us now apply the integrated edge detection scheme
to dierent real-world images. In brief, edges are ex-
tracted as follows (Lindeberg 1996): The dierential
descriptors in the edge denition (4) (rewritten in
terms of partial derivatives in Cartesian coordinates)
are computed at a number of scales in scale-space.
Then, a polygon approximation is constructed of the in-
Figure 4:
Three-dimensional view of the 10 most signi-
cant scale-space edges from the arm image drawn as curves
in three-dimensional scale-space with the selected scale rep-
resented as the heightover the image plane.
Figure 5:
Three-dimensional view of the three strongest
scale-space edges extracted from a detail of a table in which
the edges are sub ject to out-of-fo cus blur.
tersections of the two zero-crossing surfaces of
L
vv
and
@
t
(
E
;
norm
) that satisfy the sign conditions
L
vvv
<
0
and
@
t
(
E
;
norm
)
<
0. Finally, a signicance measure is
computed for eachedge byintegrating the normalized
edge strength measure along the curve
G
(;) =
Z
(
x
;
t
)
2
;
q
(
G
;
norm
L
)(
x
;
t
)
ds;
T
(;) =
Z
(
x
;
t
)
2
;
4
q
(
T
;
norm
L
)(
x
;
t
)
ds:
Fig. 2 shows the result of applying this scheme to
two real-world images. As can b e seen, the sharp edges
due to ob ject boundaries are extracted as well as the
diuse edges due to illumination eects (the occlusion
shadows on the arm and the cylinder, the cast shadow
on the table, as well as the reection on the table).
(Recall from g. 1 that for this image it is impossible
to capture the entire shadow edge at one scale without
introducing severe shape distortions at the nger tip.)
Fig. 3 illustrates the ranking on signicance ob-
tained from the integrated edge strength along the
curve. Whereas there are inherent limitations in us-
ing suchanentityasthe
only
measure of saliency,note
that this measure captures essential information.
Fig. 4 gives a three-dimensional illustration of how
the selected scale levels vary along the edges. The
scale-space edges have b een drawn as three-dimensional
curves in scale-space, overlayed on a low-contrast copy
of the original grey-level image in suchaway that
the heightover the image plane represents the selected
scale. Observe that coarse scales are selected for the
diuse edge structures due to illumination eects and
that ner scales are selected for the sharp edge struc-
tures due to ob ject b oundaries. Fig. 5 shows another
example, from a detail of table, for which the eects of
focus blur are strong. Note how the selected scales cap-
ture the amount of out-of-focus blur along the edges.
Fig. 6 shows the result of applying edge detection
with scale selection based on lo cal maxima over scales
of
T
;
norm
L
to an image containing a large amountof
ne-scale information. From a rst view, these results
may app ear very similar to the result of traditional edge
detection at a xed (very ne) scale. A more detailed
study,however, reveals that a number of shadowedges
are extracted, whichwould be impossible to detect at
the same scale as the dominant ne-scale information.
In this context, it should also be noted that the ne-
scale edge detection in this case is not the result of any
manual setting of tuning parameters. It is a direct
con-
sequence
of the scale-space edge concept, and is the re-
sult of applying the same mechanism as extracts coarse
scale levels for diuse image structures.
Figure 6:
The 1000 strongest scale-space edges extracted
using scale selection based on local maxima over scales of
T
;
norm
L
(with
=
1
2
). (Image size: 256
256 pixels.)
5 Scale selection for ridge detection
By just slight mo dication, ridge detection algorithms
can be formulated in a similar way.Ifwefollow a dier-
ential geometric approach, and dene a bright (dark)
ridge p ointasapointforwhich the brightness assumes
a maximum (minimum) in the main principal curva-
ture direction (Haralick 1983; Eberly
et al.
1994; Ko en-
derink and van Doorn 1994), then in a lo cal (
p; q
)-
system characterized by the mixed second-order deriva-
tive being zero, this denition can be written
8
<
:
L
p
= 0
;
L
pp
<
0
;
j
L
pp
j j
L
qq
j
;
or
8
<
:
L
q
= 0
;
L
qq
<
0
;
j
L
qq
j j
L
pp
j
:
In analogy with section 3, let us rst sweep out a ridge
surface in scale-space by applying this ridge denition
at all scales. Then, given a measure
R
;
norm
L
of
normalized ridge strength, dene a
scale-space ridge
as
acurve on this surface along which the ridge strength
measure assumes local maxima with respect to scale.
Here, wehave considered the following measures:
M
;
norm
L
=
t
max(
j
L
pp
j
;
j
L
qq
j
)
;
N
;
norm
L
=
t
2
(
L
2
pp
;
L
2
qq
)
2
;
A
;
norm
L
=
t
2
(
L
pp
;
L
qq
)
2
:
For a Gaussian ridge dened by
f
(
x; y
)=
g
(
x
;
t
0
), it
can be shown that for all these ridge strength measures
the selected scale will then be
t
R
;
norm
=
2
3
;
2
t
0
. Re-
quiring this scale to be
t
M
;
norm
=
t
0
,lp gives
=
3
4
.
Figure 7:
The 10 strongest bright ridges extacted us-
ing scale selection based on lo cal maxima over scales of
A
;
norm
(with
=
3
4
). (Image size: 140
140.)
Figure 8:
Three-dimensional view of the ve strongest
scale-space ridges extracted from the image in g. 7.
