Handbook of Statistics, Vol. 31
ISSN: 0169-7161
13
Copyright © 2013 Elsevier B.V. All rights reserved
http://dx.doi.org/10.1016/B978-0-44-453859-8.00013-8
Soft Biometrics for Surveillance: An Overview
D.A. Reid
1
, S. Samangooei
1
, C. Chen
2
, M.S. Nixon
1
,
and A. Ross
2
1
School of Electronics and Comput er Science,
University of Southampton, UK
2
Lane Department of Computer Science an d Electrical Engineering,
West Virginia University, USA
Abstract
sp005Biometrics is the science of automatically recognizing people b ased on physical
or behavioral characteristics such as face, fingerprint, iris, hand, voice, gait,
and signature. More recently, the use of soft biometric traits has been
proposed to improve the performance of traditional biometric systems and
allow identification based on human descriptions. Soft biometric traits include
characteristics such as height, weight, body geometry, scars, marks, and t attoos
(SMT), gender, etc. These traits offer several advantages over traditional
biometric techniques. Soft biometric traits can be typically described using
human understandable labels and measurements, allowing for retrieval and
recognition solely based on verbal descriptions. Unlike many primary biometric
traits, soft biometrics can be obtained at a distance without subject cooperation
and from low quality video footage, making t hem ideal for use in surveillance
applications. This chap ter will introduce the current state of the art in the
emerging field of soft biometrics.
Keywords: soft biometrics, surveillance, survey, fusion, identification
1. s0005Introduction
p0005Biometrics provides an automated method to identify people based on their
physical or behavioral characteristics. Classical examples of biometric traits include
fingerprints, irises, and faces which have been evaluated and demonstrated to be
useful in several different applications ranging from lap top access to border control
systems. Although t hese traits have been successfully incorporated in o perational
To protect the rights of the author(s) and publisher we inform you that this PDF is an uncorrected proof for internal business use only
by the author(s), editor(s), reviewer(s), Elsevier and typesetter SPS. It is not allowed to publish this proof online or in print. Thi s proof
copy is the copyright property of the publisher and is confidential until formal publication.
B978-0-44-453859-8.00013-8 10013
978-0-44-453859-8 Reid et al.
1
2 D.A. Reid et al.
Fig. 1.sp010 Surveillance frame displaying a few challenges in establishing identity of individuals in surveillance
videos.
1
systems, there are several challenges that are yet to be addressed. For example, the
utility of these traits significantly decreases when the input data is degraded or when
the distance between the sensor and the subject increases. Thus, the use of alternate
human attributes may be necessary to establish an individual’s identity.
p0010 Soft biometric traits are physical or behavioral features which can be described by
humans. Height, weight, hair color, and ethnicity are common examples of soft t raits:
they are not unique to the individual but can be aggregated to provide discriminative
biometric signatures. Altho ugh these types of biometric traits have only been recently
considered in biometrics, they have tremendous potential for hu man identification
by enhancing th e recognition performance of primary biometric traits.
p0015 Identification from a distance has become important due to the ever-increasing
surveillance infrastructure that is being deployed in society. Primary biometric traits
capable of identifying humans from a distance, viz., face and gait, are negatively
impacted by the limited frame rate and low image resolution of most CCTV cameras.
Figure
1 shows an example of a typical CCTV video frame. This frame shows a
suspect in the murder of a Hamas commander in Dubai in 2010. The murder involved
an 11-strong hit squad u s ing fake European passports to enter the country. All the
members of the hit squad used disguises including wigs and fake beards during the
operation. From Fig.
1 it can be observed that although the image is at l ow resolution
and the subjects’ face and ocular features are occluded, a number of soft biometric
features such as hair color, skin color, and body geometry can be deduced. Soft
biometric traits can be extracted fro m very low quality d ata such as th ose generated
by surveillance cameras. They also require limited cooperation from the subject and
can be non-intrusively obtained, making them ideal in surveillance applications.
p0020 One of the main advantages of soft biometric traits is th eir relationship with
conventional human descriptions (
Samangooei et al., 2008); humans naturally
use soft biometric traits to identify and describe each other. On the other hand,
1
Arab ian Business
http://www.arabianbusiness.com/ for-hamas-murder-suspects-40450.html.
To protect the rights of the author(s) and publisher we inform you that this PDF is an uncorrected proof for internal business use only
by the author(s), editor(s), reviewer(s), Elsevier and typesetter SPS. It is not allowed to publish this proof online or in print. This proof
copy is the copyright property of the publisher and is confidential until formal publication.
B978-0-44-453859-8.00013-8 10013
978-0-44-453859-8 Reid et al.
Soft Biometrics for Surveillance: An Overview 3
translating conventional primary biometric features into human descriptive forms
may not always be possible. This is the semantic gap
that exists bet ween h ow
machines and people recognize humans. Soft biometrics bridge this gap, allowing
conversion between human descriptions and biometrics. Very oft en, in eyewitness
reports, a physical description of a suspect may be available (e.g., “The perpetrator
was a short male with brown hair”). An appropriate automation scheme can convert
this description into a soft biometric feature set. Thus, by using soft biometrics,
surveillance footage archives can be automatically searched based on a human
description.
p0025Biometric traits should exhibit limited variations across multiple observations of
a subject and large variations across multiple subjects. The extent of these variations
defines the discriminative ability of the t rait and, hence, its identification potential.
Soft biometrics, by definition, exhibit low variance across subjects and as such rely
on statistical analysis to identify suitable combinations of traits and their application
potential. This chapter examines the current state of the art in the emerging field
of soft biometrics. Section 2 introduces the performance metrics used in biometrics.
Section 3 discusses how soft traits can be used to improve the performance of classical
biometric systems based on primary biometric traits. Using soft biometric traits to
identify humans is reviewed in Section
4. Identifying gender from facial images is
explored in Section
5. Finally, Section 6 explores some of the possible applications
of soft biometrics.
2. s0010Performance metrics
p0030The variation between multiple observations of an individual (intra-class variance)
and the variation between subjects (inter-class variance) define the performance of
a biometric system. If t he intra-class variance of a biometric trait is low, then the
trait is said to demonst rate p ermanence and repeatability. If the inter-class variance
is high, then that biometric trait can be successfully used to distinguish between
people.
p0035Once an unknown subject’s biometric signat ure (i.e., the feature set extracted from
the biometric data) has been determined, the system must then identify the subject
based on a database containing a labeled set of biometric signatures. The labels
correspond to the identity of the su bjects in the database. This is typically performed
by calculating the similarity between the input biometric signature (known as the
probe) and the signatures within the database (known as the gallery). Generally, each
identity in the d atabase will be ranked based on this similarity measure, producing an
ordered list of identities. The rank-1 retrieval performance of the biometric system
details the probability of the correct identity being first in this ordered list.
p0040While the above definition is for an identification system, it is possible for a
biometric system to verify an individual’s identity. In such a scenario, the input
biometric signature is lab eled with an identity. Thus, the input signature is only
compared against those biometric signatures in the database having the same identity
label. If the similarity measure exceeds a threshold, then a “match” or an “accept”
is said to have occurred; else, it is deemed to be a “non-match” or a “reject.”
To protect the rights of the author(s) and publisher we inform you that this PDF is an uncorrected proof for internal business use only
by the author(s), editor(s), reviewer(s), Elsevier and typesetter SPS. It is not allowed to publish this proof online or in print. Thi s proof
copy is the copyright property of the publisher and is confidential until formal publication.
B978-0-44-453859-8.00013-8 10013
978-0-44-453859-8 Reid et al.
4 D.A. Reid et al.
Fig. 2.sp015 Example intra/inter-class distributions.
p0045 When attempting to verify an identity, two errors can occur. A false accept (FA)
occurs when the input signature is incorrectly “matched” against a different identity
in the database. A false reject (FR) occurs when the input signature is incorrectly
“rejected” when matched against th e true identity in the datab ase. These errors occur
when ( a) the biometric signatures of two subjects are very similar or (b) the biometric
signature of a s ingle subject varies with time. Typically, a threshold is chosen to
define the similarity required for a “match.” This t hreshold determines the number
of FA and FR errors. Figure
2 shows the intra-class and inter-class distances (i.e.,
a dissimilarity measure) of a sample biometric recognition algorithm. To achieve
the best performance, the chosen threshold must minimize the FA and FR errors
by separating the intra-class and inter-class distributions as optimally as possible.
Receiver Operat ing Characteristic (ROC ) curves show the trade-off between FA and
FR errors at d ifferent threshold values; an example is shown in Fig.
3. The accuracy
of two different ROC curves can be easily assessed by locating the point where the
FA rate equals the FR rate: this is known as the equal error rate (EER). The EER
provides a method of assessing the recognition performance of different biometric
systems.
3.s0015 Incorporating soft biometrics in a fusion framework
p0050 Primary biometric traits such as face, fingerprints, and iris can suffer from noisy
sensor data, non-universality, and lack of distinctiveness. Further, in certain
applications, these traits may fail to achieve high recognition rates. Multimodal
biometric systems (
Ross et al., 2006) can solve these problems by combining multiple
biometric traits, resulting in a biometric signatu re that is robust and more distinctive.
To protect the rights of the author(s) and publisher we inform you that this PDF is an uncorrected proof for internal business use only
by the author(s), editor(s), reviewer(s), Elsevier and typesetter SPS. It is not allowed to publish this proof online or in print. This proof
copy is the copyright property of the publisher and is confidential until formal publication.
B978-0-44-453859-8.00013-8 10013
978-0-44-453859-8 Reid et al.
Soft Biometrics for Surveillance: An Overview 5
Fig. 3. sp020Examples of ROC curves of fou r different techniques (Samangooei, 2010).
Multimodal systems offer improved performance, but th e time taken to verify users
can drastically increase thereby causing inconvenience to the subjects and reducing
the throughput of the system. Soft biometric traits have been investigated to solve
this problem (Jain et al., 2004b).
p0055
Jain et al. (2004a,b,c) experimented w ith the integration of soft biometrics in
a biometric system. The primary biometric system compares the input biometric
signature obtained from a u s er against each subject in the database. This determines
the probability, P(ω
i
|x),i = 1,2, . . . ,n, where n is the number of subjects within
the database and P(ω
i
|x) is the probability that the identity of the input primary
feature vector x is subject to ω
i
. The secondary soft biometric system uses one
or more soft traits to confirm the out put of the primary biometric system. The
authors used height, gender, and ethnicity for this purpose. Gender and ethnicity
were automatically obtained from facial images using the technique discussed in
(
Lu and Jain, 2004). The height data was not available within the test data and,
hence, a random height was assigned to each user. The soft biometric feature vector
y upd ates P(ω
i
|x) resulting in P(ω
i
|x,y) that is calculated using Bayes’ theorem:
P(ω
i
|x,y) =
P(y|ω
i
)P(ω
i
|x)
P
n
i=1
P(y|ω
i
)P(ω
i
|x)
. (1)
p0060Experiments were performed on a 263-subject database using both multimodal
and unimodal primary b iometric systems. The author s fir s t considered the fusion
of a fingerprint-based unimodal biometric s ystem with a single soft biometric
trait (one of height, gender, and ethnicity). It was observed that fusion resulted
To protect the rights of the author(s) and publisher we inform you that this PDF is an uncorrected proof for internal business use only
by the author(s), editor(s), reviewer(s), Elsevier and typesetter SPS. It is not allowed to publish this proof online or in print. Thi s proof
copy is the copyright property of the publisher and is confidential until formal publication.
B978-0-44-453859-8.00013-8 10013
978-0-44-453859-8 Reid et al.
