Local Derivative Pattern Versus Local Binary Pattern: Face
Recognition With High-Order Local Pattern Descriptor
Author
Zhang, Baochang, Gao, Yongsheng, Zhao, Sanqiang, Liu, Jianzhuang
Published
2010
Journal Title
IEEE Transactions on Image Processing
DOI
https://doi.org/10.1109/TIP.2009.2035882
Copyright Statement
© 2010 IEEE. Personal use of this material is permitted. However, permission to reprint/
republish this material for advertising or promotional purposes or for creating new collective
works for resale or redistribution to servers or lists, or to reuse any copyrighted component of
this work in other works must be obtained from IEEE.
Downloaded from
http://hdl.handle.net/10072/32176
Griffith Research Online
https://research-repository.griffith.edu.au
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 2, FEBRUARY 2010 533
Local Derivative Pattern Versus Local Binary
Pattern: Face Recognition With High-Order
Local Pattern Descriptor
Baochang Zhang, Yongsheng Gao, Senior Member, IEEE, Sanqiang Zhao, and Jianzhuang Liu, Senior Member, IEEE
Abstract—This paper proposes a novel high-order local pattern
descriptor, local derivative pattern (LDP), for face recognition.
LDP is a general framework to encode directional pattern fea-
tures based on local derivative variations. The
-order LDP is
proposed to encode the
( 1)
-order local derivative direction
variations, which can capture more detailed information than
the first-order local pattern used in local binary pattern (LBP).
Different from LBP encoding the relationship between the central
point and its neighbors, the LDP templates extract high-order
local information by encoding various distinctive spatial relation-
ships contained in a given local region. Both gray-level images
and Gabor feature images are used to evaluate the comparative
performances of LDP and LBP. Extensive experimental results
on FERET, CAS-PEAL, CMU-PIE, Extended Yale B, and FRGC
databases show that the high-order LDP consistently performs
much better than LBP for both face identification and face verifi-
cation under various conditions.
Index Terms—Face recognition, Gabor feature, high-order local
pattern, local binary pattern (LBP), local derivative pattern (LDP).
I. INTRODUCTION
A
good object representation or object descriptor is one of
the key issues for a well-designed face recognition system
[4], [32]. Representation issues include: what representation is
desirable for the recognition of a pattern and how to effectively
extract the representation from the original input image. An
efficient descriptor should be of high ability to discriminate
between classes, has low intraclass variance, and can be easily
computed. Many holistic methods, such as Eigenface [24] and
Fisherface [3] built on principal component analysis (PCA)
and linear discriminant analysis (LDA) respectively, have been
proved successful.
Manuscript received March 08, 2009; revised August 16, 2009. First pub-
lished November 03, 2009; current version published January 15, 2010. This
work was supported in part by the Australian Research Council (ARC) under
Discovery Grants DP0451091 and DP0877929, and in part by the Natural Sci-
ence Foundation of China under Grant 60903065. The associate editor coordi-
nating the review of this manuscript and approving it for publication was Dr.
Laurent Younes.
B. Zhang is with the School of Automation Science and Electrical Engi-
neering, Beihang University, Beijing 100191, China (e-mail: bczhang@buaa.
edu.cn).
Y. Gao and S. Zhao are with the Griffith School of Engineering, Griffith Uni-
versity, Nathan Campus, Brisbane, QLD 4111, Australia (e-mail: yongsheng.
gao@griffith.edu.au; s.zhao@griffith.edu.au).
J. Liu is with the Department of Information Engineering, The Chinese Uni-
versity of Hong Kong, Hong Kong (e-mail: jzliu@ie.cuhk.edu.hk).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TIP.2009.2035882
Recently, local descriptors have gained much attention in the
face recognition community for their robustness to illumination
and pose variations. One of the local descriptors is local feature
analysis (LFA) proposed by Penev
et al. [18]. In LFA, a dense
set of local-topological fields are developed to extract local fea-
tures. Through discovering a description of one class objects
with the derived local features, LFA is a purely second-order
statistic method. Gabor wavelet is a sinusoidal plane wave with
particular frequency and orientation, modulated by a Gaussian
envelope [6]. It can characterize the spatial structure of an input
object, and thus is suitable for extracting local features. Elastic
Bunch Graph Matching (EBGM) [27] represents a face by a
topological graph where each node contains a group of Gabor
coefficients, known as a jet. It achieves a noticeable perfor-
mance in the FERET test [20]. The feasibility of the component
or patch based face recognition is also investigated in [12], in
which the component-based face recognition approaches clearly
outperform holistic approaches.
The recently proposed local binary pattern (LBP) features are
originally designed for texture description [16], [17], [21]. The
operator has been successfully applied to facial expression anal-
ysis [31], background modeling [11] and face recognition [1].
In face recognition, it achieves a much better performance than
Eigenface, Bayesian and EBGM methods, providing a new way
of investigating into the face representation. The idea behind
using the LBP features is that a face can be seen as a composi-
tion of micropatterns [1]. LBP in nature represents the first-order
circular derivative pattern of images, a micropattern generated
by the concatenation of the binary gradient directions. However,
the first-order pattern fails to extract more detailed information
contained in the input object. To the best of our knowledge, no
high-order local pattern operator has been investigated for face
representation. In fact, the high-order operator can capture more
detailed discriminative information. Some high-order nonlocal
pattern methods have been successfully used to solve the face
recognition problem. The PCA representation can hardly cap-
ture some variations in the training dataset, such as pose in
face recognition. Independent Component Analysis (ICA) takes
higher-order statistics into account, and is suitable for learning
complex structure in the dataset [2], [13]. In [10], 25 local auto-
correlation coefficients are exploited to calculate the high-order
primitive features, which are further combined with LDA and
appear robust against changes in facial expression. We can also
find other high-order techniques used in face recognition such
as the mutual information for feature selection [22], in which
high-order statistic method is used to select more discriminative
1057-7149/$26.00 © 2010 IEEE
Authorized licensed use limited to: GRIFFITH UNIVERSITY. Downloaded on February 2, 2010 at 23:20 from IEEE Xplore. Restrictions apply.
534 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 2, FEBRUARY 2010
features. In Tensorface [26], the algebra of higher-order tensors
offers a potent mathematical framework for analyzing ensem-
bles of faces resulting from the interaction of any number of
underlying factors. The feasibility of a high-order neural net-
work is also investigated in [25].
In this paper, we propose a novel object descriptor, the high-
order Local Derivative Pattern (LDP), for robust face recogni-
tion. In our framework, LBP can be conceptually considered
as a nondirectional first-order local pattern, which is the binary
result of the first-order derivative in images. The second-order
LDP can capture the change of derivative directions among local
neighbors, and encode the
turning point in a given direction. The
-order LDP is a local pattern presented in a general form
which captures detailed relationship in a local neighborhood.
Compared to LBP, the high-order LDP achieved superior perfor-
mance in our comparative experiments. Moreover, we propose
to extend LDP to feature images, Gabor real and imaginary fea-
tures, for face recognition, which can effectively enhance the
performance of the proposed LDP method, and LBP as well.
Different from the learning-based approaches, LDP features
are directly extracted from gray-level images or feature images
without any training procedure. Like LBP, LDP is a micropat-
tern representation which can also be modeled by histogram to
preserve the information about the distribution of the LDP mi-
cropatterns.
The remaining part of this paper is organized as follows. Sec-
tion II introduces and discusses the high-order LDP in detail.
Section III extends LDP to the feature domain. In Section IV, ex-
tensive experiments on FERET [20], CAS-PEAL [7], CMU-PIE
[23], Extended Yale B [9], [14], and FRGC [19] databases are
conducted to evaluate the performance of the proposed method
on face recognition. Finally, conclusions are drawn in Section V
with some discussions.
II. H
IGH-ORDER LOCAL PATTERN
In this section, we provide a brief review of local binary pat-
tern (LBP), and then introduce the second-order local deriva-
tive pattern (LDP) to calculate the first-order derivative direction
variation. After that, the definition and feasibility of the general
-order LDP are presented and discussed. Finally, the spatial
histogram is described for modeling the distribution of LDP of
a face.
A. Local Binary Pattern
Derived from a general definition of texture in a local neigh-
borhood, LBP is defined as a grayscale invariant texture mea-
sure and is a useful tool to model texture images. LBP later has
shown excellent performance in many comparative studies, in
terms of both speed and discrimination performance [1], [11],
[17], [31]. The original LBP operator labels the pixels of an
image by thresholding the 3
3 neighborhood of each pixel
with the value of the central pixel and concatenating the results
binomially to form a number. The thresholding function
for the basic LBP can be formally represented as
(1)
Fig. 1. Example of 8-neighborhood around
Z
.
Fig. 2. Example of obtaining the LBP micropattern for the region in the black
square.
where
, is an 8-neighborhood point around as
shown in Fig. 1. An LBP can also be considered as the concate-
nation of the binary gradient directions, and is called a micropat-
tern. Fig. 2 shows an example of obtaining an LBP micropattern
when the threshold is set to zero. The histograms of these mi-
cropatterns contain information of the distribution of the edges,
spots, and other local features in an image. LBP has been suc-
cessfully used for face recognition [1]. Different from statistic
learning methods tuning a large number of parameters, the LBP
method is very efficient due to its easy-to-compute feature ex-
traction operation and simple matching strategy.
B. Local Derivative Pattern
LBP actually encodes the binary result of the first-order
derivative among local neighbors by using a simple threshold
function as shown in (1), which is incapable of describing more
detailed information. In this paper, we investigate the feasi-
bility and effectiveness of using high-order local patterns for
face representation. An LDP operator is proposed, in which the
-order derivative direction variations based on a binary
coding function. In this scheme, LBP is conceptually regarded
as the nondirectional first-order local pattern operator, because
LBP encodes all-direction first-order derivative binary result
while LDP encodes the higher-order derivative information
which contains more detailed discriminative features that the
first-order local pattern (LBP) can not obtain from an image.
Given an image
, the first-order derivatives along 0 ,
45
,90 and 135 directions are denoted as where
,45 ,90 and 135 . Let be a point in , and ,
be the neighboring point around (see Fig. 1). The
four first-order derivatives at
can be written as
(2)
(3)
(4)
(5)
Authorized licensed use limited to: GRIFFITH UNIVERSITY. Downloaded on February 2, 2010 at 23:20 from IEEE Xplore. Restrictions apply.
ZHANG et al.: LOCAL DERIVATIVE PATTERN VERSUS LOCAL BINARY PATTERN 535
Fig. 3. Illustration of LDP templates. (a-1) The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
. (a-2) The template for
calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
. (a-3) The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
.
(a-4) The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
. (b-1) The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
. (b-2) The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
. (b-3) The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
. (b-4) The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
. (c-1)
The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
. (c-2) The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
. (c-3) The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
. (c-4) The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
. (d-1) The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
. (d-2)
The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
. (d-3) The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
. (d-4) The template for calculating
f
(
I
(
Z
)
;I
(
Z
))
and
f
(
I
(
Z
)
;I
(
Z
))
.
ref
1
and
ref
2
are the reference
points to be aligned to the point of
Z
. (a)
=0
,
ref
1=
3
,
ref
2=1
; (b)
=45
,
ref
1=
3
,
ref
2=1
; (c)
=90
,
ref
1=
3
,
ref
2=1
;
(d)
=135
,
ref
1=
3
,
ref
2=1
.
The second-order directional LDP, ,in direction
at
is defined as
(6)
where
is a binary coding function determining the types
of local pattern transitions. It encodes the co-occurrence of two
derivative directions at different neighboring pixels as
(7)
Finally, the second-order Local Derivative Pattern,
,
is defined as the concatenation of the four 8-bit directional LDPs
(8)
It can be seen from the above equations that the proposed LDP
operator labels the pixels of an image by comparing two deriva-
tive directions at two neighboring pixels and concatenating the
results as a 32-bit binary sequence. The derivative direction
comparisons defined in (7) are performed on 16 templates
(Fig. 3) reflecting various distinctive spatial relationships in a
local region. Different from LBP encoding the binary derivative
gradient directions, the second-order LDP encodes the change
of the neighborhood derivative directions, which represents the
second-order pattern information in the local region.
Fig. 4 illustrates the types of local pattern transitions in an
LDP template that are encoded into “1” and “0”, respectively.
Each of the 16 LDP templates in Fig. 3 can be classified as either
a 3-point template or a 4-point template. For a 3-point template,
(7) assigns a “0” to a monotonically increasing or decreasing
pattern [see Fig. 4(a-2)], while a “turning point” pattern is la-
beled as a “1” [see Fig. 4(a-1)]. Similarly, for a 4-point template,
a “gradient turning” pattern [see Fig. 4(b-1)] is labeled as a “1”
and monotonically increasing or decreasing pattern is labeled as
a “0” [see Fig. 4(b-2)]. This operator extracts higher-order local
pattern information, i.e., the changes of first-order derivative di-
rection information, into a binary string.
An example of the second-order LDP computation is illus-
trated in Fig. 5. To calculate the second-order directional Local
Derivative Pattern,
,in direction at ,
the four templates in Fig. 3(a) are applied on the image by
aligning
and to , respectively. When applying
Template (a-1) by aligning
to , the two derivative di-
rections defined by the two arrows in the template are monoton-
ically increasing as shown by the left case in Fig. 4(b-2). Thus,
“0” is assigned to this bit. Similarly, applying Templates (a-2),
(a-3), and (a-4) with
aligned to , the two derivative
directions defined by the two arrows in the templates are indi-
cated by the left case in Fig. 4(b-1), the left case in Fig. 4(b-2),
and the right case in Fig. 4(a-1), creating “101” for the next
three bits. Repeat the above procedure with the same four tem-
plates by aligning
to , we can get “0100” for the last
4 bits of the 8-bit
. Now, we have “01010100” for
the 0
direction. In the same way, templates in Fig. 3(b)–(d) are
applied on the image in Fig. 5 to obtain
in
,90 and 135 directions, respectively (see ,
and in Fig. 5). Finally, a 32-bit
is gen-
erated by concatenating the four 8-bit directional LDPs as de-
fined in (8).
Authorized licensed use limited to: GRIFFITH UNIVERSITY. Downloaded on February 2, 2010 at 23:20 from IEEE Xplore. Restrictions apply.
536 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 2, FEBRUARY 2010
Fig. 4. Meanings of “0” and “1” for the second-order LDP. (a) The three points
LDP template: both cases in (a-1) result in a “1”; both cases in (a-2) result in a
“0”. (b) The four points LDP template: both cases in (b-1) result in a “1”; both
cases in (b-2) result in a “0”.
C. -Order Local Derivative Pattern
To calculate the third-order Local Derivative Pattern, we first
compute the second-order derivatives along 0
,45,90 and
135
directions, denoted as where ,45,90,
135
. The third-order Local Derivative Pattern, ,in
direction at is defined as
(9)
In a general formulation, the
-order LDP is a binary string
describing gradient trend changes in a local region of directional
-order derivative images as
(10)
where
is the -order derivative in direction
at
. is defined in (11), which
encodes the
-order gradient transitions into binary pat-
terns, providing an extra order pattern information on the local
region
(11)
The high-order local patterns provide a stronger discrimina-
tive capability in describing detailed texture information than
the first-order local pattern as used in LBP. However, they tend
to be sensitive to noise when the order becomes high. In the
design of the proposed approach, the last-order operation (11)
only preserves the coarse gradient direction transition informa-
tion instead of conventional difference information. This can
alleviate the noise sensitivity problem in the high-order LDP
representation, making it more robust and stable in binary en-
coding identity patterns in human faces. In representing and rec-
ognizing many objects, such as human faces, the relative orien-
tation information of each local region with respect to the object
as a whole is part of the object identity patterns. This is partic-
ularly important in designing a highly discriminative object de-
scriptor for distinguishing similar objects. In face recognition,
for example, it has been demonstrated in [8] that edge orienta-
tions provide strong identity descriptive capability for classifica-
tion. Even the average orientation value of a face Line Edge Map
(LEM) [8] contains identity information, which can be used for
face prefiltering. Therefore, the proposed LDP builds upon di-
rectional derivatives, without losing generality of the method, in
four directions with a 45
representation resolution. Other num-
bers of directions can also be used. The
-order LDP is a local
pattern string defined as
(12)
It labels each pixel of the image with a 32-bit binary string en-
coding local texture pattern around the pixel in 16 measuring
templates as illustrated in Fig. 3.
In calculating the
of a given image , the
above binary pattern encoding process can be illustrated using
the LDP template in Fig. 3, and the binary coding depicted in
Fig. 4. Applying the template in Fig. 3(a-1) on
by
aligning the reference point
to the point to be
computed, the first encoded bit of
is assigned “0” if
the changes of
enclosed in the template of Fig. 3(a-1)
along the two arrows are as Fig. 4(b-2), and “1” otherwise.
Similarly, the second, third and fourth bits of
are labeled using the LDP templates (a-2) to (a-4) in Fig. 3,
respectively, by aligning the reference point
to in
. The bits 5 to 8 of are determined using
the LDP templates (a-1) to (a-4) in Fig. 3, respectively, one
more time by aligning the reference point to
to .
This first 8 bits of
is from the n -order LDP
in 0
direction, . The rest of the 3 8 bits of
can be determined in the same way by applying
the LDP templates (b-1)–(b-4), (c-1)–(c-4), and (d-1)–(d-4)
to
, and , respectively. Fig. 6
provides visualized examples of LBP and LDP representations
Authorized licensed use limited to: GRIFFITH UNIVERSITY. Downloaded on February 2, 2010 at 23:20 from IEEE Xplore. Restrictions apply.
