Smith ScholarWorks
1/265(3 &,(0&($&6.596%.,&$5,104 1/265(3 &,(0&(
O&ine Signature Veri'cation via Structural
Methods: Graph Edit Distance and Inkball Models
Paul Maergner
University of Fribourg, Switzerland
Nicholas Howe
Smith College0+18(4/,5+('6
Kaspar Riesen
University of Applied Sciences and Arts Western Switzerland
Rolf Ingold
University of Fribourg, Switzerland
Andreas Fischer
University of Fribourg, Switzerland
1..185+,4$0'$'',5,10$.813-4$5 +>244&+1.$3813-44/,5+('6&4&#)$&26%4
$351)5+( 1/265(3 &,(0&(41//104
<,410)(3(0&(31&((',0*+$4%((0$&&(25(')13,0&.64,10,01/265(3 &,(0&($&6.596%.,&$5,104%9$0$65+13,:('$'/,0,453$5131) /,5+
&+1.$3"13-413/13(,0)13/$5,102.($4(&105$&5 4&+1.$3813-44/,5+('6
(&1//(0'(',5$5,10
$(3*0(3$6.18(,&+1.$4,(4(0$42$30*1.'1.)$0',4&+(30'3($4;,0( ,*0$563(!(3,=&$5,107,$ 536&563$.
(5+1'43$2+',5,45$0&($0'0-%$..1'(.41/265(3 &,(0&($&6.596%.,&$5,104 /,5+1..(*(135+$/2510
+>244&+1.$3813-44/,5+('6&4&#)$&26%4
Offline Signature Verification via Structural
Methods: Graph Edit Distance and Inkball Models
Paul Maergner
∗
, Nicholas R. Howe
§
, Kaspar Riesen
‡
, Rolf Ingold
∗
and Andreas Fischer
∗†
∗
Department of Informatics, University of Fribourg, Fribourg, Switzerland
†
Institute of Complex Systems, University of Applied Sciences and Arts Western Switzerland, Fribourg, Switzerland
‡
Institute for Information Systems, University of Applied Sciences and Arts Northwestern Switzerland, Olten, Switzerland
§
Department of Computer Science, Smith College, Northampton, Massachusetts, USA
paul.maergner@unifr.ch, nhowe@cs.smith.edu, kaspar.riesen@fhnw.ch, rolf.ingold@unifr.ch, andreas.fischer@unifr.ch
Abstract—For handwritten signature verification, signature
images are typically represented with fixed-sized feature vectors
capturing local and global properties of the handwriting. Graph-
based representations offer a promising alternative, as they are
flexible in size and model the global structure of the handwriting.
However, they are only rarely used for signature verification,
which may be due to the high computational complexity involved
when matching two graphs. In this paper, we take a closer look at
two recently presented structural methods for handwriting anal-
ysis, for which efficient matching methods are available: keypoint
graphs with approximate graph edit distance and inkball models.
Inkball models, in particular, have never been used for signature
verification before. We investigate both approaches individually
and propose a combined verification system, which demonstrates
an excellent performance on the MCYT and GPDS benchmark
data sets when compared with the state of the art.
Index Terms—offline signature verification, structural pattern
recognition, graph edit distance, inkball models
I. INTRODUCTION
Handwritten signatures are broadly used for personal au-
thentication and there has always been an interest in verifying
their authenticity. Unfortunately, the verification of signatures
often has to rely on only a few genuine specimens. This makes
signature verification a challenging task even for humans.
Nevertheless, it has been shown that state-of-the-art automatic
signature verification systems are able to achieve a level of
accuracy that is similar to other biometric systems [1].
The development of automatic signature verification sys-
tems remains an active field of research. Hereby, the pat-
tern recognition community distinguishes two different cases
of signature verification: online signature verification uses
dynamic characteristics, like timing information, speed, and
pressure, while offline signature verification is limited to static
information, i.e. the image of the signature. The offline case
applies to more use cases, but it is also the more difficult
task [2]. In this work, we consider the offline case.
Commonly, state-of-the-art systems for offline signature ver-
ification employ statistical pattern recognition, i.e. they rep-
resent the handwriting with fixed-size feature vectors. These
vectors consist of either local information, such as histogram
of oriented gradients (HOG), local binary patterns (LBP), or
Gaussian grid features taken from signature contours [3], or
global information, e.g. geometrical features like number of
branches in the skeleton, Fourier descriptors, number of holes,
moments, projections, distributions, position of barycenter,
tortuosities, directions, curvatures and chain codes [1], [4].
For signature verification, these feature vectors are then used
in conjunction with statistical classifiers, such as dynamic time
warping (DTW), support vector machines (SVM), or hidden
Markov models (HMM) [5].
A more powerful representation formalism is offered by
graphs used for structural pattern recognition. Graphs consist
of nodes and edges, which model relations between the nodes.
For signatures, these nodes commonly represent keypoints
on the signature or elementary strokes. Relations that exist
between these parts in the global structure of the signature
are modeled with edges. However, the power of graphs comes
at the expense of high computational complexity [6], which
may be the reason why they have been only rarely used
for signature verification so far. Examples include the early
proposal to represent signatures based on stroke primitives
by Sabourin et al. [7], the modular graph matching approach
proposed by Bansal et al. [8], and the use of basic concepts
of graph theory by Fotak et al. [9].
Recently, Maergner et al. [10] have introduced a general
framework for graph-based signature verification based on
the graph edit distance between labeled graphs. Promising
verification results are reported for so-called keypoint graphs
that have also been used for handwriting recognition [11]
and keyword spotting [12] before. To overcome the high
computational complexity of matching two graphs, they use
a bipartite approximation of the graph edit distance [13].
Another promising approach to structural handwriting anal-
ysis are inkball models. They have been introduced by Howe
in [14] as a technique for performing segmentation-free word
spotting when limited training data are available. Later, they
have also been used as a complex feature with HMM for hand-
writing recognition [15]. Inkball models share some similar
properties with keypoint graphs. However, inkball models are
rooted trees that are directly and efficiently matched with a
skeleton image.
In this paper, we investigate inkball models for the first
time for signature verification. Furthermore, we propose a
combined system that integrates keypoint graphs and inkball
models as two complementary handwriting models. On two
benchmark data sets, we evaluate the two structural methods
individually, combined, and in comparison with the current
state of the art.
In the remainder of this paper, we formally introduce
keypoint graphs and graph edit distance in Section II, and
inkball models in Section III. Then, we elaborate on how
we use them individually as well as combined for offline
signature verification in Section IV. Afterwards, we evaluate
the different approaches on two publicly available data sets
and compare our results with the state of the art Section V.
Finally, we present our conclusion and outlook in Section VI.
II. GRAPH EDIT DISTANCE
The first structural method considered in this paper is the
approach introduced by Maergner et al. in [10]. In their
approach signature images are binarized and thinned into
skeleton images and from that keypoint graphs are created.
These graphs are then compared using an approximation of
the graph edit distance. Afterwards the resulting graph edit
cost is normalized. These steps are briefly reviewed in the next
subsections. For more details, we refer the reader to [10].
A. Image Processing
The signature images are binarized and skeletonized. First, a
difference of Gaussians filter is applied on the grayscale image.
Afterwards, a global threshold is used to create a binary image.
Lastly, the thinning algorithm introduced by Zhang and Suen
in [16] is applied to get a skeleton image.
B. Graph Representation
Formally, a labeled graph is defined as a four-tuple
g = (V, E, µ, ν), where
• V is the finite set of nodes,
• E ⊆ V × V is the set of edges,
• µ : V → L
V
is the node labeling function,
• ν : E → L
E
is the edge labeling function.
Keypoint graphs are extracted from a skeleton image of
handwriting (for an example, see Fig. 1). Nodes represent
points on the skeleton and they are labeled with the coordinates
of these points. The edges are unlabeled and undirected and
they connect nodes that are next to each other on the skeleton.
The points that are represented by nodes are the end- and
junction-points of the skeleton. Then, the left outer most pixel
of circular structures are added if they do not contain any end-
or junction-points. Afterwards, additional points are added by
tracing along the skeleton and adding points after traveling a
distance of D
keypoint
without hitting an already selected point.
After the graph has been created, the node labels are
centered so that the average of all node labels is equal to
(0, 0). This normalization ensures that the graph representation
is translation-invariant.
Fig. 1. Keypoint Graph
C. Approximated Graph Edit Distance
One of the most flexible ways to compare two graphs
is graph edit distance (GED) [17], [18]. It measures the
dissimilarity of two graphs by calculating the cost of the
cheapest transformation of graph g
1
= (V
1
, E
1
, µ
1
, ν
1
) into
graph g
2
= (V
2
, E
2
, µ
2
, ν
2
). Hereby, a transformation consists
of a sequence of edit operations, which are commonly defined
as substitutions, deletions, and insertions of nodes and edges
respectively. Given an appropriate cost function, graph edit
distance can handle any kind of labeled graph. Unfortunately,
the computation time of the exact GED is exponential in the
number of nodes of the two graphs. Therefore, exact GED is
in practice only applicable to rather small graphs.
To overcome the computational problem, a bipartite
approximation of GED proposed by Riesen and Bunke [13]
is used. Their approximation framework reduces the compu-
tation of GED to an instance of a linear sum assignment
problem (LSAP) with cubic complexity. The lower bound of
GED introduced in [19] is considered.
D. Cost Function
The node substitution cost is the Euclidean distance between
the node labels. Formally,
c(u → v) =
p
(x
u
− x
v
)
2
+ (y
u
− y
v
)
2
,
where (x
u
, y
u
) and (x
v
, y
v
) are the labels (i.e. coordinates) of
nodes u and v respectively.
The insertion and deletion costs of nodes and edges rely
on the average length m(g
1
) of all edges in the graph g
1
.
Formally, the node deletion and insertion cost is defined as
c(u → ε) = c(ε → v) = m(g
1
),
and the edge deletion and insertion cost as
c(e
1
→ ε) = c(ε → e
2
) = 2 · m(g
1
).
The edge substitution cost is set to zero: c(e
1
→ e
2
) = 0.
E. Normalization of Graph Edit Distance
The graph edit distance is normalized with the maximal
graph edit distance possible when comparing the two graphs.
The maximal graph edit distance is the cost of deleting all
nodes and edges from the first graph and inserting all the
nodes and edges from the second graph. This normalization
calculates how close the dissimilarity is to the maximal
dissimilarity instead of just the distance. Formally, the graph
edit distance based comparison of two signature images r and
t is defined as follows:
d
GED
(r, t) =
GED(g
r
, g
t
)
GED
max
(g
r
, g
t
)
, (1)
where g
r
and g
t
are the keypoint graphs of the signatures
images r and t respectively, GED(g
r
, g
t
) is the lower bound
of the graph edit distance calculated using the bipartite graph
matching framework, and GED
max
(g
r
, g
t
) is the maximal
graph edit distance.
III. INKBALL MODELS
The second structural method considered in this work is the
inkball model introduced by Howe in [14]. An inkball model
can be generated from a signature image through a procedure
similar to that used for keypoint graphs. After binarization
and thinning [20], inkballs are placed at each junction and
endpoint. Additional inkball nodes are added sequentially, as
close as possible to the existing positions while staying at
least distance
√
2 · D
inkball
away. When no more can be added
in this manner, additional inkballs are added that are as close
as possible to existing positions but at least distance D
inkball
away.
Once a set of inkballs have been identified, they are linked
in a tree structure by greedily adding edges between the closest
nodes that are not yet connected. The node nearest to the center
of mass is arbitrarily designated as the root, and each child
node is annotated with the Cartesian offset (denoted ~o
i
) of its
parent node relative to its own position.
A. Inkball Matching Energy
The degree of fit between an inkball model derived from
signature sample r to a second signature sample t can be
measured by a combination of deformation and proximity:
how closely the inkballs can be placed near the observed
ink, and how much the model structure must be deformed to
achieve that proximity (for a matching example, see Fig. 2).
Formally, define a configuration C of the model as a place-
ment ~v
i
for each inkball. These imply configuration offsets
~s
i
= ~v
i↑
−~v
i
for all nodes except the root. The configuration
energy then becomes the sum of squared differences between
the configuration offsets and the original the model offsets,
plus the square of the minimum distances Ω
t
(~v
i
) from each
inkball location to the target skeleton.
E(r, C, t) = E
ξ
(r, C) + λE
Ω
(C, t) (2)
E
ξ
(r, C) =
n
X
i=2
k~s
i
−~o
i
k
2
(3)
E
Ω
(C, t) =
n
X
i=1
Ω
t
(~v
i
)
2
(4)
In practice any mismatch larger than a certain threshold
should be treated as equally bad, and thus a truncated quadratic
Fig. 2. Inkball Model Matching
will better serve for the energy function. The truncated energy
is computed in a stucture-aware fashion:
E
0
i
(r, C, t) = min (E
i
(r, C, t), N
i
τ) (5)
E
i
(r, C, t) = Ω
T
(~v
i
)
2
+
X
j∈i↓
k~s
j
−~o
j
k
2
+ E
0
j
(r, C, t)
(6)
Here τ represents a maximal per-node energy contribution,
i ↓ denotes the children of node i, and N
i
is the number of
inkball nodes in the subtree with root at node i. For purposes
of signature verification, the important quantity is the minimal
configuration energy, which can be efficiently computed using
dynamic programming (for further algorithmic details, see
Howe et al. [15]).
E
∗
(r, t) ≡ min
C
E
0
1
(r, C, t) (7)
In the experiments, D
inkball
and τ are free parameters whose
values can be optimized on a validation set.
B. Normalization of the Inkball Deformation Energy
Using the equations above, two signature images may be
compared by converting the first one to an inkball model and
computing the optimal match E
∗
to the other image. The score
obtained is highly influenced by the number of inkballs in the
model, and thus is not independent of image scale. To address
this shortcoming it is preferable to apply a normalization
similar to that used with the graph edit distance. Instead of
measuring the amount of energy needed to match the inkball
model to a signature, we measure how much energy is needed
on average per inkball. Formally, we define the inkball based
dissimilarity of two signature images r and t as follows:
d
inkball
(r, t) =
E
∗
(r, t)
N
r
(8)
IV. SIGNATURE VERIFICATION SYSTEM
In the task of offline signature verification, an unseen sig-
nature image claiming to be from a specific user is compared
with known signatures from that user, so-called references.
Based on these comparisons a dissimilarity score between the
reference signatures and the questioned signature is calculated.
If this score is below a certain threshold, the signature is
accepted, otherwise it is rejected.
A. Reference-based Normalization
Each verification score (either d
GED
or d
inkball
) is divided by
the average dissimilarity score of the reference signatures of
the current user as suggested in [10]. Formally,
ˆ
d(r, t) =
d(r, t)
δ(R)
, (9)
where t is a questioned signature image, r ∈ R is a reference
signature image, R is the set of all reference signature images
of the current users, and
δ(R) =
1
|R|
X
r∈R
min
s∈R\r
d(r, s).
B. Signature Verification Score
We consider the minimum dissimilarity over all reference
signatures R of the claimed user for accepting or rejecting the
questioned signature t. Formally,
d(R, t) = min
r∈R
ˆ
d(r, t) (10)
C. Multiple Classifier System
Additionally to the signature verification score based on a
single dissimilarity measure, we propose a multiple classifier
system (MCS) using a linear combination of the two dissimi-
larity measures as our combined dissimilarity score. Before
the linear combination, each dissimilarity score is z-score
normalized based on all references signature images in the
current data set.
d
MCS,α
(R, t) = min
r∈R
α ·
ˆ
d
∗
GED
(r, t) + (1 − α) ·
ˆ
d
∗
inkball
(r, t)
,
(11)
where α ∈ [0, 1], and
ˆ
d
∗
(r, t) =
ˆ
d(r, t) − µ
R
σ
R
, (12)
considering the mean µ
R
and the standard deviation σ
R
calculated over the set R = {R
1
, . . . , R
n
} of all references
signature sets R
i
of all n users in the current data set. For
example, the mean µ
R
is calculated as
µ
R
=
1
|R|
X
R∈R
1
|R|
X
r∈R
min
s∈R\r
ˆ
d(r, s)
!
, (13)
V. EXPERIMENTAL EVALUATION
In this section, we evaluate the performance of the two
structural methods, individually as well as in combination, on
two publicly available benchmark data sets. The focus lies on
distinguishing genuine signatures from skilled forgeries (SF),
which are forgeries that have been created for each user with
knowledge about the user’s genuine signatures. Additionally,
we test how well genuine signatures can be distinguished from
random forgeries (RF), which are signatures of other users that
are used for a brute force attack on the verification system.
The performance on both is measured using the equal error
rate (EER). The EER is the point where the false rejection
rate is equal to the false acceptance rate in the detection error
tradeoff (DET) curve.
A. Data Sets
For our experimental evaluation, we consider the following
publicly available signature image data sets.
GPDSsynthetic-Offline: Ferrer et al. have introduced this
data set in [21]. It contains 24 genuine signatures and 30
simulated forgeries for each of the 4,000 synthetic users. We
have created two subsets of this data set using the first n users
(n ∈ {10, 75}). The subsets are called GPDS-10 and GPDS-75
respectively. Hence, GPDS-10 contains 10 · (30 + 24) = 540
signature images and GPDS-75 contains 75·(30+24) = 4, 050
signature images.
MCYT-75: Ortega-Garcia et al. introduced this data set as
part of the MCYT baseline corpus in [22], [23]. For each of
the 75 users, the data set offers 15 genuine signatures and
15 skilled forgeries. Thus, 75 · (15 + 15) = 2, 250 signature
images are included in MCYT-75. All available genuine sig-
natures and skilled forgeries are used in our experiments.
B. Tasks
We evaluate the following commonly used tasks:
– R5: First five genuine signatures are used as references.
– R10: First ten genuine signatures are used as references.
The remaining genuine signatures are used for testing for
both the skilled forgery (SF) and the random forgery (RF)
evaluation. All skilled forgeries are used for the SF evaluation.
For the RF evaluation, we used the first genuine signature
of all other users as random forgeries. This means that for
example for MCYT-75 R10, we have 75 ·10 = 750 reference
signatures, 75 ×5 = 375 genuine signatures, 75 ×15 = 1, 125
skilled forgeries, and 75 × 74 = 5, 550 random forgeries.
C. Setup
We performed a grid search on GPDS-10 R10 SF to
optimize the parameters of the inkball dissimilarity. The grid
search was performed over the following parameter range:
D
inkball
∈ {2, 4, 8, 16, 32} and τ ∈ {16, 32, 64, 128, ∞}.
Additionally, we tested all of these configurations with and
without inkball normalization (see Section III-B). The best
results have been achieved using s = 4, τ = 32, and with
normalization turned on. Note that decreasing the D
inkball
increases the size of the model and therefore the calculation
time.
For the graph edit distance based dissimilarity, we use
the configuration proposed in [10] where they optimized the
parameters on GPDS-75. Specifically, we used D
keypoint
= 25.
In an additional validation step on GPDS-10 R10, we tested
different weights for our proposed multiple classifier system
(see Section IV-C). The best results have been achieved using
α = 0.4 when investigating α ∈ {0, 0.1, 0.2, . . . , 1}.
D. Results on MCYT-75 and GPDS-75
Our EER results are shown in Table I for SF and in Table II
for RF. DET curves for the SF experiment are shown in Fig.3.
Consistently, the inkball dissimilarity performs better with
normalization than without. Interestingly, the performance of
the two dissimilarities differs depending on the data set. While
