Offline Signature Verification Using Online Handwriting Registration
Yu Qiao, Jianzhuang Liu
Department of Information Engineering
The Chinese University of Hong Kong
qiao@gavo.t.u-tokyo.ac.jp, jzliu@ie.cuhk.edu.hk
Xiaoou Tang
Microsoft Research Asia
Beijing, China
xitang@microsoft.com
Abstract
This paper proposes a novel framework for offline sig-
nature verification. Different from previous methods, our
approach makes use of online handwriting instead of hand-
written images for registration. The online registrations en-
able robust recovery of the writing trajectory from an in-
put offline signature and thus allow effective shape match-
ing between registration and verification signatures. In ad-
dition, we propose several new techniques to improve the
performance of the new signature verification system: 1.
we formulate and solve the recovery of writing trajectory
within the framework of Conditional Random Fields; 2. we
propose a new shape descriptor, online context, for align-
ing signatures; 3. we develop a verification criterion which
combines the duration and amplitude variances of hand-
writing. Experiments on a benchmark database show that
the proposed method significantly outperforms the well-
known offline signature verification methods and achieve
comparable performance with online signature verification
methods.
1. Introduction
Signature is a socially accepted authentication method
and is widely used as proof of identity in our daily life.
Automatic signature verification by computers h as received
extensive research interests in the field of pattern recogni-
tion. Depending on the format of input information, auto-
matic signature verification can be classified into two cat-
egories: online signature verification [6, 13, 8, 12, 5] and
offline signatu re verification [7, 23, 15]. In the former case,
a hand pad together with an instructed pen [6, 13, 8, 5] or a
video camera [12] is used to obtain the online information
of pen tip (position, speed, and pressure). Therefore the in-
put is a sequence of features. In the latter case, the input
is a two-dimensional sig nature image captured by a scanner
or other imaging device. Online signature verificatio n has
been shown to achieve much higher verification rate than of-
fline verification [6, 13, 8, 12, 5, 7, 23, 15]. The state of the
art of online verification achieves equal error rates (EERs)
ranging from 2% to 5% [6, 13, 8, 12, 5], while the EERs of
offline verification are still as high as 10%-30% [12, 7, 23].
This diff erence is largely due to the availability of dynamic
information in online system [5, 15]. Roughly speaking, the
matching and annotation problems for 2D images are more
difficult and time consuming than those for 1D sequences.
Although online verification outperforms the offline one, its
use of special devices for recording the pen-tip trajectory
increases its system cost and brings constraints on its appli-
cations. In some situations, such as check transaction and
document verification, offline signature is obligatory. This
paper focuses on offline signature verification, and our ob-
jective is to discriminate between genuine signatures and
skilled forgeries which are written by careful imitation.
Various features have been proposed for signature verifi-
cation tasks. These features can be roughly divided into two
types [13, 5]: 1) global features which are extracted from
the whole signature, including block codes [7, 23], Wavelet
and Fourier series [13], etc.; 2) local features which are cal-
culated to describe the geometrical and topological charac-
teristics of local segments, such as position, tangent direc-
tion, and curvature [13, 8, 5, 12]. The global features can be
extracted easily and are robust to noise. But they only de-
liver limited information for signatur e verification [13, 8].
On the other hand, local features provide rich descrip tions
of writing shapes and are powerful for discriminating writ-
ers, but the extraction of reliable local features is still a hard
problem.
The local features based approaches are more popular in
online verification than in the offline one. This is because it
is much easier to calculate local shape features and to find
their corresponding relations in 1D sequences than in 2D
images. This fact inspires us to consider recovering writ-
ing tr ajectories from offline sig nature images. Then local
features can be calculated and aligned more efficiently and
effectively by using recovered trajectories. Similar idea had
been adopted by Lee and Pan [15], where they proposed lo-
cal tracing algorithms to find the dynamic information of
signatures. However, as pointed out in [14, 22], the local
1-4244-1180-7/07/$25.00 ©2007 IEEE
Oine
Si
g
nature
Online
Si
g
nature
Pre
p
rocessin
g
Online
Signature
Examination
Result
Recover Writing
Trajectory
Registration
Pre
p
rocessin
g
Database
Verication
Figure 1. System diagram.
tracing methods are sensitive to noise and writing variance
and it is d ifficult to design tracing algorithms which can be
applied to variant writing styles. In fact, direct recovery of
writing trajectory is still an open p roblem in handwriting
research [18, 14, 22]. To circumvent this difficulty, we pro-
pose a novel approach that uses online signatures in the reg-
istratio n phase. We develop this approach based on the ob-
servation that registration needs to be done only once and it
has to be done in person in-situ (such as in a bank) where an
online d evice is easily available. In the verification phase,
the procedure is exactly the same as an offline system thus
is convenient to use. In our algorithm, we take advantage
of the online registration data to recover writing trajectory
of an offline input signature image and make the verifica-
tion decision based on the recovered trajectory. The system
diagram is shown in Fig.1.
In the context of signature verification, tested signatures
whether genuine or skillfully forged, usually have similar
shapes with the registration ones. (If a tested signature has
a shape very different from the registration, it can be easily
identified as a forgery.) This similarity allows us to recover
the writing trajectory by using online signatures as exam-
ples. In this paper, we model signatu res using Conditional
Random Fields (CRFs) and reduce the recovery of writing
trajectory to a CRFs inference problem.
Different from online signature verification, our problem
is based on the recovered trajectories which do not have dy-
namic writing information such as speed, pressure, and ori-
entation. To compensate the loss of the dynamic informa-
tion, we introduce a new descriptor, online context, which
summarizes shape information by vector histograms and is
robust to local deformation. We develop a new verification
criterion that combines two types of variances in handwrit-
ing duration and amplitude. Experimental results show that
our method achieves EERs which are lower than those of
the previous offline verificatio n methods and are compara-
ble to the online methods.
(a) Sequence (online signature)
(b) Graph (Offline sigmature)
Termina
l
vertex
Connecon
vertex
Juncon
vertex
Figure 2. Examples of sequence and graph. Circles represent
points/vertices and black lines represent edges. Gray lines in (a)
correspond to pen movements out of paper. Gray area in (b) rep-
resent original image.
2. Recovery of writing trajectories
The recovery of a writing trajectory from an o ffline sig-
nature image is to find a trajectory within the offline image
which is most similar to the trajectory of the online reg-
istrations. There are two basic problems, how to evaluate
the similarity and how to find the best path? This paper
answers these two questions within the framework of con-
ditional random fields. The similarity is defined through the
state and transition functions of CRFs, and the problem to
find the best trajectory is reduced to the inference problem
of CRFs.
2.1. Representation of online/offline s ignatures
The online signature consists of a set of consecutive
strokes, denoted by S = s
1
,s
2
, ..., s
m
where m is the num-
ber of strokes. We sample points with equal space along
each stroke. Then we connect all the strokes to get a single
sequence of points denoted by L = p
1
,p
2
, ..., p
n
(Fig.2a).
For each two consecutive points p
i
and p
i+1
in L,weadda
directed edge e
i
: p
i
→ p
i+1
between them.
The offline signature image is modeled by graph G =
y
1
y
2
y
3
y
n-1
y
n
x
1
x
2
x
3
x
n-1
x
n
…
Figure 3. Chain structured conditional random field.
(E,V ) (Fig. 2b) where V = {v
i
} denotes a set of vertices
and E = {e
k
} denotes a set of edges. G is constructed f rom
the skeleton of input image. Each vertex v
i
corresponds to
a point sampled from the skeleton. There are three kinds
of vertex: terminal vertex which is a pixel with degree one,
junction vertex which is a pixel or a cluster of connected
pixels with degree three or more, and connection vertex,
which is a two-degree pixel sampled along a segment of
skeleton between two terminal or junction vertices. I f there
exists a segment of skeleton between v
i
and v
j
, two directed
edges e
i1
: v
i
→ v
j
and e
i2
: v
j
→ v
i
are added into E.For
edge e, we define its continuous set N(e) as a set of edges
which can be traced continuously next to e. N (e) includes
the edges whose start vertex is the end vertex of e and also
the edges which may start a new stroke after e.
For each directed edge e, we calculate its three features
(θ, a, b),whereθ is its directed angle, a and b represent its
normalized positions. These features will be used to evalu-
ate the similarity cost between online and offline signatures
in the next.
Given online registration signature L and offline signa-
ture G, we want to find the corresponding relations between
the elements in L and the elements in G. As the writing or-
der of the edges in L is known, we can trace the edges in G
according to the orders of their corresponding edges in L,
thus recover th e writing trajectory. Next, we will formulate
the problem within the framework of conditional random
fields.
2.2. Conditional random fields
Conditional Random Fields (CRFs) are non-generative
graphical models first proposed by Lafferty et al. in [9].
CRFs define a conditional prob ability distribution of label
set Y = {y
i
}
n
i=1
given observed feature set X = {x
i
}
n
i=1
.
We use a first order chain structured CRF as shown in Fig.3.
Compared with HMM, CRFs relax the independent as-
sumption of states by capturing dependencies among obser-
vations. CRFs can avoid the label bias problem [9] exhib-
ited by HMMs and MEMMs. CRFs need a small number o f
training samples and allow flexible features, as it need not to
specify a complete distribution for explaining observations.
These facts make it suitable for our task.
Conditioned on X, random variable y
i
obeys the Markov
property i.e., p(y
i
|X, y
j
(j = i)) = p(y
i
|X, y
i−1
,y
i+1
).
According to the Hammersley-Clifford theorem [4], the
conditional probab ility can be written into the following
form [9]:
p
Λ
(Y |X)=
1
Z(X)
exp(
i,k
λ
k
f
k
(y
i−1
,y
i
,X,i)
+
i,k
µ
k
g
k
(y
i
,X,i)), (1)
where Λ={λ
k
,µ
k
} represents the set of parameters and
Z(X) is a normalized factor. f
k
(y
i
,y
i−1
,X,i) is a transi-
tion function
1
at positions i − 1 and i; g
k
(y
i
,X,i) is a state
function at position i.
A straightforward approach is to assume that CRF has
the same topology as G and to use the vertices of G as ob-
servations and the indices of the points in L as labels. How-
ever, this approach has several disadvantages: 1) G may
include double traced lines, so one observation may corre-
spond to more than one state; 2) G may have complex topol-
ogy which will increase the computational complexity; 3) it
is difficult to account for between-edge relations if we use
vertices/points as states, since a function of Eq.(1) includes
at most two states.
Due to these considerations, we use a chain structured
CRF (Fig. 3), where the online signature is regarded as its
observation, each observation x
i
represents an edge in L
and label y corresponds to the index of directed edge e
y
in
G.
In this way, functions g
k
represent the directional and
positional differences which are used to punish the shape
difference between X and Y . Functions f
k
represent the
smoothness which are used to evaluate the continuity of
edges, thus to make the recovered writing path smooth.
2.3. Parameters estimation for CRFs
The online registration signatures are used to train the
CRFs. We calculate the matching cost for each two sig-
natures b y using online context based dynamic time warp-
ing (DTW). Then the set of training signatures is divided
into one or more clusters, w here the pairwise matching
cost in each cluster must be less than a threshold. Let
S
1
,S
2
, ..., S
M
denote a cluster. In the next, we will train
a CRF for each cluster. Define the mean signature S
m
as
the o ne which has the minimum DTW matching cost with
the remaining signatures. We divide S
m
into n edges with
equal length and then divide other signatures according to
their aliments to S
m
. In this way, all the signatures are di-
vided into n edges. For each edge, we calculate its three
features.
1
f
k
and g
k
are called ”features” in [9], however, the term ”feature” has
a meaning of the properties of edges in this paper. To av oid confusion, we
use term ”function”.
Let X denote the observed feature sequence of S
m
,and
Y
(j)
denote the indexes of the edges in S
j
(j = m). (X,
Y
(j)
) represent a training pair for CRFs. Totally, the num-
ber of training samples is N = M −1. Note that observation
X is fixed during the training phase.
Given training samples D = {(X,Y
(j)
)}
N
j=1
, we n eed
to estimate the parameters Λ={λ
k
,µ
k
}. Since state func-
tion g
k
can be seen as a special form of f
k
, henceforth we
will omit the discussion on g
k
and µ
k
for simplicity. The
learning of CRFs aims to find the p arameters which maxi-
mize the conditional log-likelihood:
L(Λ) =
Y
log p
Λ
(Y |X). (2)
To reduce overfitting, we use a Gaussian prior p(Λ) with
a diagonal covariance matrix for regulation. Similar prior is
adopted in Bayesian-CRF [19]. Thus Eq. (2) becomes
L(Λ) =
Y
log p
Λ
(Y |X) −
k
λ
2
k
2σ
2
k
+const, (3)
where σ
2
k
is the variance of λ
k
, which can be approximated
by the reciprocal of the variance of f
k
in experiments.
It can be shown that the above likelihood function is con-
vex on Λ, which guarantees the convergence to a global
optimization [1 2]. The Generalized I terative Scaling (GIS)
[9, 2] algorithm has been adopted to calculate the optimal
parameter s. GIS algorithm iteratively increases L(Λ) by
updating the parameters as: λ
k
← λ
k
+ δ
k
. Due to the use
of Gaussian prior, our training algorithm is different from
that in [9]. The details are as the following:
1. Initialize the parameters Λ
(0)
.
For each function f
k
, calculate its mean:
¯
f
k
=
1
N
k
i
f
k
(y
i−1
,y
i
,X,i). (4)
For each training pair (X, Y ), calculate:
T (X, Y )=
1
N
Y
i
f
k
(y
i−1
,y
i
,X,i). (5)
2. Repeat the following steps until convergence. In the
t-thstep(t=1,2,...),
2.1 Calculate δ
(t)
k
by solving the equations:
¯
f
k
=
λ
(t)
k
+ δ
(t)
k
σ
2
k
+
Y
p
Λ
(t)
(Y |X)
i
f
k
(y
i−1
,y
i
,X,i)exp{δ
(t)
k
T (X, Y )} .
(6)
2.2 Update Λ
(t+1)
by λ
(t+1)
k
= λ
(t)
k
+ δ
(t+1)
k
.
It can be shown that the roots {δ
(t)
k
} of Eq. (6) always lead
to the increase of the likelihood. The derivations of this
equation can be found in [20].
2.4. Inference by dynamic programming
The recovery o f writing order fro m an offline signature
image is formulated as the inference problem of CRFs. For
an input signatu re image, we build its graph model G and
calculate the features of its edges. Feature sequence X used
in the training is also used as the observation during infer-
ence. Y denotes a set of indices of the edges in G.By
solving the inference problem of the CRF, we can deter-
mine the corresponding relations between the edges in X
and the edges in G, that is, to determine the writing order of
the edges in G. Formally, the inference problem is denoted
by
Y ∗ =argmax
Y
p
Λ
(Y |X), (7)
subjected to
e
y
i
∈{N(e
y
i−1
) ∪ e
y
i−1
}. (8)
The above constraint is used to ensure that the edges can be
traced continuously along the recovered order. As Z(X) is
a constant, Eq. (7) is equal to
max
y
1
,y
2
,...,y
n
{
i,k
λ
k
f
k
(y
i−1
,y
i
,X,i)+
i,k
µ
k
g
k
(y
i
,X,i)}.
(9)
The above form reminds us that dynamic programming (or
Viterbi decoding) can be used to optimize it. Define the
following function F (y, i):
If i =0, F (y, 0) =
k
µ
k
g
k
(y,X,0);
else if i>0,
F (y, i)=max
y
{F (y
,i− 1) +
k
λ
k
f
k
(y
,y,X,i)}
+
k
µ
k
g
k
(y,X,i).
(10)
We can iteratively update F (y,i) using Eq. (10). Finally,
F (y, N) will be the solution for Eq. (10). Let Y =
y
1
,y
2
,...,y
n
denote the optimal configuration of Eq. (10).
The recovered writin g trajectory is e
y
1
→ e
y
2
, ..., → e
y
n
.
To speed up the process, we identify the candidate start and
end points for each stroke at first and then do bidirectional
search from these candidates [21]. The ECR (Edge Conti-
nuity Relation) analysis method proposed in [22] is used to
simplify graph model G.
A problem of the above method is that some edges of
G may not be traced. To overcome this, we use a refining
algorithm to insert the un-traced edges into the recovered
Figure 4. Examples of recovery results. Writing trajectories are
shown as the arrows.
trajectories. The algorithm works in a greedy fashion. In
each step, it selects the edge which can be inserted in the
smoothest way. If too many edges are still not traced af-
ter refinement, the signature must be very different from the
online registration and it should be recognized as a forgery.
Since there are more than one CRF models for a subject,
we use each of them to recover the writing trajector ies and
the final result is selected as the one with the largest likeli-
hood. Some examples of the recovery are shown in Fig. 4.
It can be seen that the proposed method can effectively deal
with double traced edges and junctions of multiple strokes,
which are regarded as the hard problems in recovery [22]. In
our experiments, most failed recovered signatures are forg-
eries.
3. Signature verification
In this section, we compare the recovered trajectory with
the online registrations to reach a verification decision. It
seems that the condition a l probab ility defined in Eq.(1) can
be used as a criterion for verification. However, Eq. (1) only
measures the conditional distribution given one signature
example and it doesn’t describe the signature distribution
of a writer. Moreover, it is just a coarse description and
cannot account for the detailed shape information of writing
trajectory, which is important for signature verification.
Handwriting is a complex biomechanical process, which
includes the movements of fingers, wrist, and forearm. It
has been shown that humans generate handwriting through
controlling the magnitude and direction of speed [17]. And
handwriting exhibits variances in both duration (time) and
amplitude (space).
Although the writing trajectory is recovered, our prob-
lem is still different from the online signature verification,
since the recovered trajectory doesn’t have the dynamic in-
formation such as speed and pressure. Moreover, the re-
covered trajectory is within the skeleton calculated by the
p
p
2
θ
1
θ
2
l
2
θ
l
p
1
p
2
(a) Signature
(
b
)
Grids
π
0
π/2
3π/2
2π
-
1
-
1/2
-
1/8 0 1/8 1/2 1
Figure 5. Online Context.
thinning process. It is well known that thinning algorithms
are sensitive to noise and may result in unwanted a rtifact
lines [18, 22]. Thus the recovered trajectory does not al-
ways superpose the original writing trajectory. To reduce
the unfavorable effects of these problems, we introduce on-
line context based dynamic time warping for aligning the
trajectories. And we develop a verification criterion which
combines the time a nd amplitude variances.
3.1. Online context
For each point p along the signature, we define the online
context to describe its relative shap e information along the
trajectory. We draw vectors from p to all the other sampling
points along the trajectory (Fig. 5a). Each vector p → p
1
is
represented by two parameters (θ, l),whereθ is the direc-
tional angle, and l is the normalized length of the trajectory
between p and p
1
(the term ”normalized” means the length
is divided by the total length of the signature). These vectors
provide rich descriptions of the detailed shape. However,
because of the large number of vectors, it is not easy to deal
with them directly. Thus, we use a histogram to summarize
the distribution of these vectors on the two parameters of θ
and l. Specially we introduce the grids whose vertical axis
corresponds to θ and whose horizontal axis corresponds to
the logarithm
2
of l (Fig. 5b). Then we count the number of
vectors falling into each gr id. Formally, the online context
h
k
is defined as
h
p
(k)=#{p
i
= p : p → p
i
∈ Grid(k)}. (11)
Online context is related to the shape context proposed
in [1]. Both compute the distribution of the vectors from
one point to others for describing shapes. However, we
use curve length other than Euclidean distance to construct
2
The use of logarithm is to make online context more sensitive to the
nearby points than to the faraway ones.
