International Journal of Computer Vision 62(1/2), 145–159, 2005
c
2005 Springer Science + Business Media, Inc. Manufactured in The Netherlands.
The Promise and Perils of Near-Regular Texture
∗
YANXI LIU, YANGHAI TSIN AND WEN-CHIEH LIN
The Robotics Institute, Carnegie Mellon University, 5000 Forbes Ave., Pittsburgh, PA 15213, USA
yanxi@cs.cmu.edu
ytsin@cs.cmu.edu
wclin@cs.cmu.edu
Received December 10, 2002; Revised June 23, 2003; Accepted July 1, 2003
First online version published in November, 2004
Abstract. Motivated by the low structural fidelity for near-regular textures in current texture synthesis algorithms,
we propose and implement an alternative texture synthesis method for near-regular texture. We view such textures
as statistical departures from regular patterns and argue that a thorough understanding of their structures in terms of
their translation symmetries can enhance existing methods of texture synthesis. We demonstrate the perils of texture
synthesis for near-regular texture and the promise of faithfully preserving the regularity as well as the randomness
in a near-regular texture sample.
Keywords: near-regular texture, texture synthesis, lattice, texture analysis, symmetry groups
1. Motivation
Near-regular textures are common in our daily life.
They can be observed in man-made products, by hand
or by machine, ranging from buildings to fabrics,
as well as in nature and biological process of life
science (Feynman, 1998; Chambers, 1995; Senechal,
1995; Zee, 1999; Hargittai and Hargittai, 2000). Hu-
mans have an innate ability to perceive and take ad-
vantage of symmetry (Leyton, 1992). Rao and Lohse
(1993) showed that regularity plays an important role
in human texture perception. However, it is not obvious
how to automate this powerful insight.
Mathematically speaking, regular texture refers to
periodic patterns that present non-trivial translation
symmetry, with the possible addition of rotation, re-
flection and glide-reflection symmetries (Miller Jr.,
1972; Coxeter, 1980; Gr¨unbaum and Shephard, 1987).
When studying periodic patterns, a useful fact from
mathematics is the answer to Hilbert’s 18th prob-
∗
This work is funded in part by an NSF research grant #
IIS-0099597.
lem: there is only a finite number of symmetry
groups for all possible periodic patterns in dimension n
(Bieberbach, 1910). When n = 1 there are seven frieze
groups, and when n = 2 there are 17 wallpaper
groups. Here group is referring to the symmetry group
of a periodic pattern. A symmetry group is composed of
transformations that keep the pattern setwise invariant.
In computer vision and computer graphics, the appli-
cation of this classic mathematics for regular or near-
regular pattern analysis has yet to be fully explored.
Only recently, have computer algorithms of symmetry
group classification been developed for periodic pat-
terns in real images under Euclidean (Liu and Collins,
2000; Liu et al., 2004) and affine transformations (Liu
and Collins, 2001), based on a careful analysis of the
basic tile shapes of regular patterns. In computer graph-
ics, one interesting recent work (Kaplan and Salesin,
2000) is to find Escher-like tilings by deforming a sin-
gle closed planar figure to tile a plane.
Near-regular texture is referring to textures that
are not strictly symmetrical. The irregularity can be
caused by various statistical departures from regular
textures. These departures can happen along different
146 Liu, Tsin and Lin
Figure 1. Symmetry or regularity of images spans a continuous,
multi-dimensional space.
dimensions of symmetry (Liu, 2001), for example,
color (single, multi) (Tsin et al., 2001), intensity (ir-
regular statistical alterations, random noise), global or
local geometric deformations (affine, projective, ran-
dom) (Liu and Collins, 2000, 2001), and resolution. See
Fig. 1 from Liu (2001) for some examples of symmetry
dimensions. The focus of this paper is on faithful tex-
ture synthesis of near-regular textures where departure
from regularity is primarily caused by statistical color
and intensity variations, while the underlying struc-
tural regularity remains. There are many examples of
this type of near-regular textures, e.g. brick walls, tiled
floors, carpets, and woven sheets, where the texture pat-
terns (each brick, tile, straw or bamboo strip) vary only
locally. The idea of viewing a random texture as a dis-
torted version of a regular texture was expressed in an
early paper by Zucker (1976). More recently, we have
demonstrated a computational model for near-regular
textures that vary along geometry, lighting and color
dimensions (Liu et al., 2004).
Existing work on texture synthesis has achieved
impressive results for a variety of different types
of textures (e.g., De Bonet, 1997; Efros and
Leung, 1999; Ashikhmin, 2001; Efros and Freeman,
2001; Hsu and Wilson, 1998; Wei and Levoy, 2000;
Hertzmann et al., 2001; Xu et al., 2001; Liang
et al., 2001; Zhu et al., 2000; Kwatra et al., 2003;
Cohen et al., 2003). These texture synthesis algorithms
share a common theme of local neighborhood-based
statistical approaches. Distinctions can be drawn be-
tween approaches that constructively establish statis-
tical models for the input texture (Cross and Jain,
1983; Zhu et al., 1997) versus others that seek to
find matching joint statistics directly in the input
samples (De Bonet, 1997; Portilla and Simoncelli,
2000; Zhu et al., 2000). More recently, non-parametric
estimation of texture PDFs has become popular
(Efros and Leung, 1999; Wei and Levoy, 2000; Efros
and Freeman, 2001; Liang et al., 2001). These tex-
ture synthesis algorithms are relatively simple to
implement, fast to run (Wei and Levoy, 2000; Liang
et al., 2001) and able to reproduce a large variety of
textures, from regular to random,asclaimed by the au-
thors. However, after reviewing the results of existing
work applied to near-regular textures, we observe that
the structural regularity is usually not well preserved
in the synthesized texture. This is especially true when
the input sample has interlocking near-regular patterns,
or is oriented obliquely. For example, we have not yet
seen an existing texture synthesis algorithm that pre-
serves the regularity in a brick wall sample (Fig. 2(a)).
In addition, the structural property of near-regular tex-
tures has not been used as an objective measure for
texture synthesis algorithms (Lin et al., 2004).
This situation motivates us to propose and implement
an alternative texture synthesis method for near-regular
texture that is particularly faithful to its structural prop-
erty while preserving the randomness observed in the
input data. Figures 2 and 3 demonstrate two sample
results from our texture synthesis algorithm in contrast
to the texture synthesis results reported in Efros and
Freeman (2001).
Section 2 defines basic properties of regular texture
such as generating tile, symmetry groups and lattice
types. In Section 3 we explain our texture analysis and
synthesis algorithm and demonstrate some experimen-
tal results. Section 4 discusses several relevant issues
in near-regular texture synthesis, from window size to
the concept of textons. Section 5 concludes with a sum-
mary and future research directions.
2. Regular Texture Analysis
A symmetry of a 2D periodic pattern P is a distance
preserving mapping g: R
2
× I ⇒ R
2
× I such that
g(P) = P, where I can either be gray values in the
range of [0, 255] or RGB intensity values. It can be
proven that all symmetries of P form its symmetry
group. All the translation symmetries of a periodic pat-
tern form its translation subgroup, a group generated
by two linearly independent, shortest translation sym-
metries
t
1
,
t
2
of P (Schattschneider, 1978). Mathemat-
ically speaking, symmetry groups are defined only for
periodic patterns of infinite extent. In practice, we an-
alyze a periodic pattern bounded within a finite image
area, and thus use the concept of symmetry group G
The Promise and Perils of Near-Regular Texture 147
Figure 2. (a) input texture sample. (b) texture synthesis result from Efros and Freeman (2001). This is one of the best results on brick wall
texture synthesis that we can find. However, the regularity in the input texture sample is not faithfully preserved in the synthesized texture: two
short bricks are stacked together and there are more than two brick sizes in the synthesized image. (c) the texture synthesis result of our algorithm
proposed in this paper.
Figure 3. (a) input texture sample. (b) texture synthesis result from Efros and Freeman (2001). Straw pattern: one vertical line is terminated
midway. (c) the texture synthesis result of our algorithm proposed in this paper.
148 Liu, Tsin and Lin
of P to mean G is the symmetry group of an infinite
periodic pattern for which P is a finite region with more
than one period.
Each 2D regular texture is a 2D periodic pattern
that contains a non-empty parallelogram T . The or-
bit of T under the action of its translation symme-
try subgroup produces simultaneously a covering (no
gaps) and a packing (no overlaps) of the original pat-
tern (Gr¨unbaum and Shephard, 1987; Schattschneider,
1978). We call the smallest such parallelogram the tile
of the texture. For a given regular texture its tile is
uniquely defined in shape, size, and orientation but not
in location, thus its pixelwise intensity and color con-
tent may vary, depending on where the lattice of the
texture pattern is anchored.
A mature mathematical theory for wallpaper-like
regular texture has been known for over 100 years
(Fedorov, 1885; Gr¨unbaum and Shephard, 1987),
namely the theory of wallpaper groups.
1
For
monochrome planar periodic patterns, there are sev-
enteen wallpaper groups describing patterns extended
by two linearly independent translational generators.
Despite the infinite variety of regular texture instanti-
ations, this finite set of symmetry groups and their 17
corresponding lattice/tile structures completely char-
acterize the possible structural symmetry of any 2D
periodic pattern. There are only five possible lattice
shapes (Coxeter and Moser, 1980), therefore five tile
shapes, and they form a shape hierarchy (Fig. 4):
1. parallelogram,
2. rectangular,
3. rhombic,
4. square, and
5. hexagonal.
Figure 4. There are only five possible types of tiles in 2D regular
textures.
Each lattice unit or tile shape is a parallelogram. A
rectangular tile has angles of 90
o
.Arhombic tile has
equal-length edges. Square and hexagonal tiles are spe-
cial cases of rectangle and rhombic, respectively.
Work in structural texture analysis (Enrich and
Foith, 1978; Lu and Fu, 1978) is also based on the idea
of a unit pattern together with a set of well-defined
placement rules. However, its generality and computa-
tional tractability are limited: unit patterns are either re-
gions centered about a local maximum that is bounded
on all sides by local minima (Enrich and Foith, 1978)
or square texture regions with an unspecified window
size (Lu and Fu, 1978). Conners and Harlow (1980) use
mathematical tiling theory for the analysis of texture,
but they do not take advantage of the complete char-
acterization of lattice types and the inner structures of
2D regular texture afforded by wallpaper groups, and
their characterization of pattern elements is dominated
by the inertia feature alone.
One essential element in our method is to acknowl-
edge the regularity in a near-regular texture by first
locating the generating “tile” precisely. This computa-
tional effort is guided by the basic principles and un-
derstanding of tiles and their symmetries, as concisely
summarized in their wallpaper groups. In order to find
tiles in a given 2D near-regular pattern we developed
an algorithm in Liu and Collins (2000) and refined
in Liu et al. (2004), based on regions of dominance,
for locating the underlying lattice of a given pattern.
Figure 5 shows the variations of shapes, sizes and ori-
entations of lattices automatically generated from three
real-world near-regular patterns. In addition, the gener-
ating translation vectors and a typical tile are indicated
as an example on one of the three textures. The
t
1
,
t
2
translational symmetries of a regular pattern alone fix
the size, shape and orientation of the lattice, but leave
open the question of where the lattice is located on the
pattern. Any offset of the lattice on a pattern carves
the pattern into a set of similar tiles, any one of which
can generate the whole 2D pattern. For perception pur-
poses (Liu and Collins, 2001; Liu et al., 2004), a motif
(a representative tile) can be chosen that reflects the
symmetry property of the whole pattern. For synthesis
purposes, on the other hand, the tiles could be chosen
to optimize the “blending” effects (Section 3.1).
3. Our Method for Texture Synthesis
Perfect regularities are rarely found in the real world,
while varying degrees of deviation from regularity
The Promise and Perils of Near-Regular Texture 149
Figure 5. Examples of imperfect, real-world near-regular patterns overlayed with automatically detected underlying lattices using an algorithm
developed in Liu et al. (2004). Notice the different shape, size and orientations of the tiles. The arrows drawn on the middle image give an
example of the two shortest generating translations
t
1
,
t
2
for this texture pattern. The region bounded by the two vectors (enclosed by the two
vectors and two dotted lines) indicates a tile for this pattern. For each near-regular texture, there exists a well-defined tile that is bounded by the
two linearly independent translations of its wallpaper pattern.
are common to observe. Our research interest is to
capture both regularity and randomness by combin-
ing the mathematical theory of regular patterns with
statistical modeling of data in texture analysis and
synthesis.
We treat a set of tiles carved by the detected lattice as
multiple samples of the same tile. We define these tiles
as minimum tiles {t
i
} since by definition of regular pat-
terns there are no 2D regions smaller than these tiles that
can tile the whole texture pattern under its translation
subgroup. Correspondingly, we define a set of maxi-
mum tiles {T
i
} by circumscribing each minimum tile
t
i
with the smallest rectangularly shaped convex hull.
Note that depending on the shape and orientation of the
t
i
’s, maximum tiles T
i
can be in any possible orienta-
tion and aspect ratio. The minimum (maximum) tile set
also contains tiles centered on half-way shifted lattice
points (i.e. at locations ((n + 1/2)
t
1
, (m + 1/2)
t
2
) from
the anchored lattice position, where m, n are integers).
Fortexture synthesis, at each time a tile is randomly
chosen from these tile sets. This process provides the
promise of capturing statistical color and intensity vari-
ations from different tiles, which can give the generated
texture more natural appearance, while reproducing its
regularity.
3.1. Algorithm for Texture Synthesis
of Near-Regular Patterns
Input:asample near regular texture S
Output:asynthesized texture S
statistically similar
to S.
Stage 1 (analysis):
• First determine the translational symmetry vectors
t
1
,
t
2
from the given sample near-regular texture pat-
tern. In our experiments, these vectors can either
be (1) computed automatically (Liu and Collins,
2001; Liu et al., 2004); (2) indicated by the user
by clicking on three nearest corresponding points of
the texture, or (3) computed first and verified by the
user.
• Determine where the lattice should be anchored such
that all the minimum tiles t
i
are uniquely defined. This
is one parameter that the user can control to make
the boundary of the tiles align with low frequency
regions for the benefit of better blending results. In
our experiments, most lattice locations have been
hand-located.
• For each t
i
construct the corresponding maximum tile
sets T and T
h
. T contains all the T
i
s centered on the
