Chapter 7
Boundary Finding with Parametrically Deformable
Models
Lawrence H. Staib and James S. Duncan
7.1 Introduction
This work describes an approach to finding objects in images based on de-
formable shape models. Boundary finding in two and three dimensional images is
enhanced both by considering the bounding contour or surface as a whole and by
using model-based shape information.
Boundary finding using only local information has often been frustrated by
poor-contrast boundary regions due to occluding and occluded objects, adverse
viewing conditions and noise. Imperfect image data can be augmented with the
extrinsic information that a geometric shape model provides. In order to exploit
model-based information to the fullest extent, it should be incorporated explicitly,
specifically, and early in the analysis. In addition, the bounding curve or surface
can be profitably considered as a whole, rather than as curve or surface segments,
because it tends to result in a more consistent solution overall.
These models are best suited for objects whose diversity and irregularity of
shape make them poorly represented in terms of fixed features or parts. Smoothly
deformable objects do not necessarily have an obvious decomposition that can
be exploited. A uniform shape representation that describes the entire shape is
therefore needed and it should describe a relatively broad class of shapes.
For a representation to be useful for modeling it should be concise. Methods
based on explicitly listing points or patches on the surface are verbose because of
the implicit redundancy. Parametric representations capture the overall shape in a
small number of parameters. This means that the optimization of a match measure
between data and a model can occur in a lower dimensional space.
Boundary finding is formulated as a optimization problem using parametric
Fourier models which are developed for both curves and surfaces. The model is
matched to the image by optimizing in the parameter space the match between the
model and a boundary measure applied to the image. Probability distributions on
the parameters of the representation can be incorporated to bias the model to a
particular overall shape while allowing for deformations. This leads to a maximum
a posteriori objective function.
7.2 Related Work in Boundary Finding
Local edge detectors applied to real images produce spurious edges and gaps.
These problems can only be overcome by the incorporation of information from
higher scale organization of the image and models of the objects sought. Contextual
information has been used for boundary determination via grouping [1], relaxation
labeling [2] and scale-space methods [3]. These methods, by themselves, will not
necessarily find complete boundaries. Pixel search methods associate edge elements
by finding an optimal path through a two-dimensional image, based on criteria
1
designed to find boundaries. The typical objective function combines boundary
strength and low overall curvature [4]. Pixel search does not generalize obviously
to three dimensions because there is no natural ordering of voxels in a surface.
An alternative method for boundary analysis is the Hough transform [5]. The
Hough approach is similar to the current method in that it finds shapes by looking
for maxima in a parameter space. However, the storage and computational com-
plexity of the Hough method are a great disadvantage, especially if deformations
are envisaged.
Other investigators have considered whole-boundary methods that adjust a
tentative curve or surface mesh in order to match to the image. By considering
the boundary as a whole, a structure is imposed on the problem that bridges gaps
and results in overall consistency.
For curve finding, Gritton and Parrish [6] used a flexible bead chain, where the
beads are putative boundary points. The beads are attracted towards pixels that
have a higher gradient magnitude. Cooper [7] formulated boundary estimation
using maximum likelihood. A boundary adjustment scheme similar to the bead
chain algorithm [6] is presented to perform the optimization. Kass et al. [8] used
energy-minimizing snakes that are attracted to image features such as lines and
edges while internal spline forces impose a smoothness constraint. The weights of
the smoothness and image force terms in the energy functional can be adjusted for
different behavior. The solution is found using variational methods.
For surface finding, Terzopoulos et al. [9] used energy-minimizing meshes that
are attracted to image features such as lines and edges while internal spline forces
impose a smoothness constraint. The goal was to find surfaces implied by silhou-
ettes in two-dimensional images. This idea has also been used for finding symmetry
surfaces from scale space stacks of two-dimensional images [10], surfaces in range
images [11, 12] and surfaces in three-dimensional images [13].
Other whole-boundary methods optimize in a parameter space. Parametric
representations are useful for modeling because they capture the overall shape
concisely. This means that the optimization of a match measure between data and
a model can occur in a lower dimensional space. Widrow [14] used parametrized
templates called rubber masks to model objects. The parameters are sizes and
relationships between subparts. Yuille et al. [15] used a similar method for finding
features in images of faces. Both of these methods describe the overall shape of
the structure using very few parameters. However, the object must have sufficient
structure to be represented in terms of parts and a new model must be developed for
each new object. Work has also been done developing deformable templates based
on Markov models of two-dimensional boundaries incorporating knowledge of shape
from statistical features [16]. In the next section we will discuss parametrizations
for surfaces in more detail.
Pentland and his group have developed a physically-based method for analyzing
shape [17, 18]. Shapes are represented by the low-order frequency displacement
eigenvectors corresponding to the free vibration modes of the object. Thus, it is
similar to a Fourier representation. The shape is recovered using the finite element
method.
2
7.3 Curve and Surface Representations
Implicit equations are a traditional and natural representation which define a
relationship between coordinates such that all points that satisfy this relationship
belong to the structure. Such representations are ideal for determining whether
specific points belong to the object but there is no general way for generating
such points. Because such operations will be crucial for this work, only explicit
parametric representations will be considered further.
An arbitrary curve can be represented explicitly by two functions of one param-
eter: x(s) and y(s). A surface can be represented explicitly by three function of
two parameters: x(u, v), y(u, v) and z(u, v). A surface is indexed or parametrized
by the two parameters (u, v). While a curve’s points are naturally ordered (by
arclength), there is no natural ordering of points on an arbitrary surface. Cer-
tain classes of curves and surfaces can be represented as a single function. For
example, curves expressible as a single function of one parameter, r(θ), are radial
deformations of a circle. Similarly, surfaces expressible as a function of two an-
gles, r(θ, φ), are radial deformations of a sphere and are parametrized by (θ, φ).
Surfaces expressible as a single function of two coordinates, z(x, y), are perpendic-
ular deformations of a plane and thus the points in the plane, (x, y), provide the
parametrization.
The main approaches to parametric modeling in computer vision have been
polynomials [19], superquadrics [17, 20], spherical harmonics [5, 21] and generalized
cylinders [22]. All of these parametrizations are restricted to a limited class of
objects.
7.3.1 Polynomials
Second degree algebraic surfaces have been used extensively because of their
simplicity and conciseness. Conics are second degree curves including ellipses,
parabolas and hyperbolas. Quadrics are second degree surfaces which include
spheres, ellipsoids, cones, cylinders, planes, paraboloids and hyperboloids. Their
conciseness, however, greatly limits their expressiveness. Higher order polynomial
surfaces are expressed using implicit representations.
7.3.2 Superquadrics
Superquadrics are an extension of quadrics using an exponent that allows
the shape to vary from an ellipsoid to a rectangular parallelepiped. The two-
dimensional analog is the superellipse. Superquadrics can be expressed parametri-
cally by:
x(u, v) = x
0
+ a
1
sign(cos v cos u)| cos u|
²
1
| cos v|
²
2
y(u, v) = y
0
+ a
2
sign(sin v cos u)| cos u|
²
1
| sin v|
²
2
z(u, v) = z
0
+ a
3
sign(sin u)| sin u|
²
1
(1)
The surface parameters u and v represent latitude and longitude. The exponent
²
1
controls the squareness in the u plane and ²
2
controls the squareness in the v
plane. The parameters a
1
, a
2
and a
3
control the size in the x, y and z directions.
3
The basic shape can be altered by such operations as twisting, bending and
tapering [23], as can any explicit representation. The main disadvantage of su-
perquadrics is that even with these altering operations, superquadrics are limited
by their doubly symmetric cross-section and thus still only represent a very lim-
ited family of shapes (without resorting to composition). Superquadrics have been
augmented by deformations according to spline models [9] and strain modes [17] in
order to increase their expressiveness. Hyperquadrics [24] are a generalization of
superquadrics that allow smooth deformations from shapes with convex polyhedral
bounds, although no explicit parametrized form is possible.
7.3.3 Generalized Cylinders
Generalized cylinders (or cones) are a way of representing elongated objects.
They are defined by a one-dimensional curve representing the spine of the object
and a two-dimensional cross-section that is swept along the spine to define the
surface. This cross-section may vary along the spine. The actual properties of this
representation depend on the choices of spine (sweeping rule) and cross-section.
Practical choices usually limit the class of object that is representable. The most
common restriction is to straight, homogeneous generalized cylinders (SHGCs)
where the spine is straight and the cross-section shape is constant (allowing scal-
ing). These can be defined by [25]:
x(u, v) = r(u)x(v) + pz(u)
y(u, v) = r(u)y(v) + qz(u)
z(u, v) = z(u) (2)
where u varies along the spine, v varies along the cross-section, r(u) defines the
scaling, x(t) and y(t) define the cross-section shape and z(u), p and q define the
spine. If the spine is allowed to bend, the cross-section is usually taken to be
perpendicular to the axis. The cylinder radius must therefore be greater than the
radius of curvature or else the boundary will cross itself. If the spine and cross-
section are represented parametrically, as opposed to directly as an explicit list of
coordinates or segments, generalized cylinders can be completely parametric.
An object can be represented by a generalized cylinder only if there exists an
axis that a cross-section can sweep along in order to define the surface. The choices
for the form of the spine and the cross-section further limit the expressibility of
the representation.
7.3.4 Spherical Harmonics
Spherical harmonics have been used as a type of surface representation for
radial or stellar surfaces (r(θ, φ)). The surface is represented as a weighted sum of
spherical harmonics which are orthogonal over the sphere. A surface is represented
in polar coordinates by:
r(θ, φ) =
M
X
m=0
N
X
n=0
(A
mn
cos nθ + B
mn
sin nθ) sin
n
φ P (m, n, cos φ) (3)
4
Figure 1: The contour (dark line) at the left is constructed from three component
ellipses shown at three different times.
where P(m,n,x) is the nth derivative of the mth Legendre polynomial as a function
of x. The parameters of the representation are the weights A
mn
and B
mn
.
This is a type of Fourier representation, as defined below, but restricted to
stellar surfaces. Stellar surfaces are obtained by deforming a sphere by moving
points only in the radial direction. This means that all surface points must be
seen from one point in the interior. Thus, spherical harmonics model a somewhat
limited class of objects.
7.4 Fourier Models
Smoothly deformable objects do not necessarily have an obvious decomposition
that can be exploited. A uniform shape representation that describes the entire
shape is therefore needed and it should describe a relatively broad class of shapes.
Fourier representations are those that express the function in terms of an or-
thonormal basis. The motivation for a basis representation is that it allows us to
express any object as a weighted sum of a set of known functions. An orthonormal
set is desirable because it makes the parameters (weights) distinct.
For example, to express the one-dimensional function f(t) on the interval (a, b)
in terms of the basis φ
k
(t), we write:
f(t) =
∞
X
k=1
p
k
φ
k
(t) where p
k
=
Z
b
a
f(t)φ
k
(t) dt (4)
5
