Cronfa - Swansea University Open Access Repository
_____________________________________________________________
This is an author produced version of a paper published in :
Information Visualization
Cronfa URL for this paper:
http://cronfa.swan.ac.uk/Record/cronfa24663
_____________________________________________________________
Paper:
D., H., P., A., M., L., R., B., I., W., R., S. & M., C. (2013). Glyph sorting: Interactive visualization for multi-dimensional
data. Information Visualization, 14(1), 76-90.
http://dx.doi.org/10.1177/1473871613511959
_____________________________________________________________
This article is brought to you by Swansea University. Any person downloading material is agreeing to abide by the
terms of the repository licence. Authors are personally responsible for adhering to publisher restrictions or conditions.
When uploading content they are required to comply with their publisher agreement and the SHERPA RoMEO
database to judge whether or not it is copyright safe to add this version of the paper to this repository.
http://www.swansea.ac.uk/iss/researchsupport/cronfa-support/
Glyph Sorting: Interactive Visualization for
Multi-dimensional Data
David HS Chung
1,2
, Philip A Legg
1,2
, Matthew L Parry
1,2
, Rhodri Bown
3
, Iwan W Griffiths
2
, Robert S Laramee
1
and Min Chen
4
1
Department of Computer Science, Swansea University, UK.
2
College of Engineering, Swansea University, UK.
3
Centre of Excellence, Welsh Rugby Union, UK.
4
Oxford e-Research Centre, Oxford University, UK.
Abstract
Glyph-based visualization is an effective tool for depicting multivariate information. Since sorting is one of the
most common analytical tasks performed on individual attributes of a multi-dimensional data set, this motivates
the hypothesis that introducing glyph sorting would significantly enhance the usability of glyph-based visualiza-
tion. In this paper, we present a glyph-based conceptual framework as part of a visualization process for interactive
sorting of multivariate data. We examine several technical aspects of glyph sorting and provide design principles
for developing effective, visually sortable glyphs. Glyphs that are visually sortable provide two key benefits: 1)
performing comparative analysis of multiple attributes between glyphs and 2) to support multi-dimensional visual
search. We describe a system that incorporates focus and context glyphs to control sorting in a visually intuitive
manner and for viewing sorted results in an Interactive, Multi-dimensional Glyph (IMG) plot that enables users to
perform high-dimensional sorting, analyse and examine data trends in detail. To demonstrate the usability of glyph
sorting, we present a case study in rugby event analysis for comparing and analysing trends within matches. This
work is undertaken in conjunction with a national rugby team. From using glyph sorting, analysts have reported
the discovery of new insight beyond traditional match analysis.
1. Introduction
Sorting large, multi-dimensional data is a growing consen-
sus in modern data acquisition and processes where the or-
dering of data is an integral part of many applications and
disciplines, ranging from the analysis of scientific informa-
tion (e.g., using graphs and charts), to enhancing the effi-
ciency of algorithms. Such records are traditionally sorted
analytically in a data-driven manner (e.g., via spreadsheets),
where users perform sorting on individual attributes of a
multi-dimensional data set. This is a non-trivial task due
to the vast possible permutations of sorting which greatly
impacts the expressiveness in high dimensional visualiza-
tions [YPWR03]. When data must be ordered using a high
level of sorting, it reveals two important challenges: 1) how
the data is organised, and 2) the ordering of sort keys, which
can not be easily observed by viewing large tables of data.
Glyphs (sometimes known as icons) are graphical entities
that convey one or more data values using visual features
such as size, shape and colour. This significantly improves
perception of data characteristics and is well suited for de-
picting high-dimensional, multivariate data [War02]. Cher-
noff Faces [Che73] and Star Glyphs [SFGF72] are some
examples of multivariate glyphs where identifying glyphs
with similar features is effective, but cognitively challenging
when determining the ordering of glyphs. Thus, such glyphs
are not visually sortable in an obvious way. This becomes
a greater challenge when glyphs are unorganised. Figure 1
demonstrates how ordering such glyphs in a given spatial
configuration is more informative in revealing multivariate
trends. Glyph sorting is one approach for performing interac-
tive sorting of multivariate data as part of a visualization pro-
cess. As an data exploration mechanism, interactive sorting
in visualization provides the following additional objectives:
1) making observations about data patterns (e.g., clusters and
distributions) in relation to a sorted variable and stimulating
hypotheses about other variables. 2) performing analytical
tasks and visual evaluation of hypotheses, such as what vari-
ables may affect the ordering of a specific variable.
2 Chung et al. / Glyph Sorting: Interactive Visualization for Multi-dimensional Data
(a) (b)
Figure 1: Visual representation of two example multi-dimensional glyphs, namely (a) Star glyphs and (b) Bar chart glyphs
when glyphs on the left are unordered, in comparison to glyphs on the right which are ordered to two sorting parameters.
In this paper, we present a novel glyph-based sorting
framework to drive and facilitate interactive sorting of data
in a visual and intuitive manner. We describe a set of de-
sign principles (Section 4) for mapping attributes to visu-
ally sortable glyphs. This significantly enhances the usabil-
ity of glyph-based visualizations for both comparative anal-
ysis of multi-variate data and for supporting visual search.
In Section 5, we present an interactive system for the ex-
ploration of glyph-based visualization. Novel features of the
system include a focus and context glyph-based user inter-
face (Section 5.1) to control high-dimensional sorting and
viewing sorted results in a Interactive, Multi-dimensional
Glyph (IMG) plot (Section 5.2). We extend traditional axis
mapping using hierarchical axis binning (Section 5.3). This
enables visual depiction of multiple sort key parameters in
space, which is effective for reducing visual clutter in the
IMG plot view. To demonstrate the effectiveness of glyph-
sorting, we present a real-world case study of rugby event
analysis. The work is carried out in close collaboration with
an international rugby team, in which we developed a glyph-
sorting software tool for use by the coaching analysts. As
a result of glyph sorting, the analysts uncovers new insight
and knowledge for match analysis. The main contributions
of this paper are:
• The introduction and development of high-dimensional,
focus and context glyphs that are visually sortable to sup-
port sorting of multi-variate data.
• A novel glyph-based, interactive system for controlling
high-dimensional sorting and viewing sorted results.
• A hierarchical axis binning method for encoding multiple
dimensions onto a single axis. This effectively reduces vi-
sual clutter by relaxing the positioning of glyphs.
• An evaluation of the effectiveness of glyph sorting in a
real-world case study of sports event analysis.
2. Related Work
Sorting is the computational process of rearranging a se-
quence of items into ascending or descending order [Knu98].
Many sorting algorithms have been proposed, including bub-
ble sort by Demuth [Dem56], merge sort by von Neu-
mannr [Knu98], and quick sort by Hoare [Hoa62]. Since
best and worse case performance runtime can vary dras-
tically with such algorithms, further research continues to
propose new sorting techniques [BFCM06] and adaptive ap-
proaches that utilise ordered data [ECW92]. In a previous re-
lated work, we describe a knowledge-assisted event ranking
framework to convert tacit knowledge into a formal sorting
criteria for organising rugby match videos [CLP
∗
13]. The
sorting function here may be multivariate in nature [see sup-
plementary paper]. Our work is not focused on a faster sort-
ing algorithm per say, but combining the benefits of sorting
with glyph-based visualization.
Glyph-based visualization is an established technique
for depicting multi-dimensional data sets. The survey by
Ward [War02, War08a] provides a technical framework for
glyph-based visualization, covering aspects of visual map-
ping and layout methods, as well as addressing important is-
sues such as bias in mapping and interpretation. Ropinski et
al. [RP08] present an in-depth survey on the use of glyph-
based visualization for spatial multivariate medical data.
Glyphs are widely used in other application areas, such as
DT-MRI visualization [LAK
∗
98, WMM
∗
02], unsteady flow
visualization [HLNW11] and activity recognition [BBS
∗
08].
Lie et al. [LKH09] describe a general pipeline for visualiz-
ing scientific data in 3D using glyphs and introduce design
guidelines such as the orthogonality of individual attribute
mappings. Pearlman et al. [PRdJ07] use a glyph-based mul-
tivariate visualization to understand depth and diversity of
large data sets. Chlan and Rheingans [CR05] use 2D and
3D glyph-based multivariate visualization to show distribu-
tion within the data set. Jänicke et al. [JBMC10] introduce
SoundRiver, that depict audio/video events from movies us-
ing glyphs for visualization on a timeline. Previous to this
study, Legg et al. [LCP
∗
12] conducted a design study to
show the effective use of glyph-based visualization within
sports performance analysis. A fundamental difference here
is that we use glyphs that are visually sortable.
Chung et al. / Glyph Sorting: Interactive Visualization for Multi-dimensional Data 3
Interactive visualization studies the ability of human in-
teraction for exploring and understanding datasets through
visualization, which Zudilova et al. [ZSAL08] covers in a
state-of-the-art report. De Leeuw and Van Wijk [dLvW93]
is one earlier research which incorporates glyphs into in-
teractive visualization for analysing multiple flow charac-
teristics in selected regions using a probe glyph. Shaw et
al. [SEK
∗
98] describe an interactive glyph-based framework
for visualizing multi-dimensional data, where attributes are
mapped in order of data importance to visual cues such as lo-
cation, size, colour and shape. To our knowledge, this is the
first work of its kind to introduce focus and context glyphs
for visual sorting of high-dimensional data.
3. Sorting: Entities and Sort Keys
Sorting is the most common analytical task which is used for
re-organising entities consisting of single or multiple fields.
The objectives of sorting can be classified into the following:
• Ordering - arranging entities of the same type, or class
into some ordered sequence.
• Categorizing - grouping or labelling entities with similar
properties through sorting.
A sort operation can be performed based on one or more
attributes. We describe such attributes as sort keys. In more
general form, let us consider the set of objects or entities
E = (e
1
, e
2
, . . . , e
s
), each containing a set of attribute keys
K = (k
1
, k
2
, . . . , k
n
). This defines a n-dimensional attribute
space which governs the sorting process. Thus, e
i
is a n-tuple
or contains a n-tuple (as e
i
may have additional information
such as a video clip). For example, a group of entities E may
be classified as a pack of cards (52 entities) which is sortable
by keys K, such as card type (e.g., spades, clubs, diamond,
and hearts), colour (e.g., red or black) or by value (1-13).
In order theory, we can specify two types of ordering re-
lations: a weak (non-strict) order denoted by "", or a strict
ordering "≺". These two properties characterize the mathe-
matical concept of linear ordering [Knu98]. Given a subset
of keys κ ∈ K, the goal of sorting is to arrange the entities
e
i
into an ordered set (a list) such that e
κ
1
≺ e
κ
2
≺ ... ≺ e
κ
s
.
At the level of abstraction, sort keys as attributes can not
be directly compared (i.e., by arithmetic =, and <, >), as
they are essentially concepts. Hence, we introduce the no-
tion f
κ
: E 7→ R, that maps the object space with context
keys κ to a real value such that for any entity pair, e
i
, e
j
, the
ordering relation e
κ
i
≺ e
κ
j
implies:
f
κ
(e
i
) < f
κ
(e
j
) ∀i, j = 1, 2, . . . , n. i 6= j
With additional semantics, one can define such a function
f
κ
to sort data (e.g., events) into more practical, or mem-
orable orderings beyond common sorts (e.g., alphabetical),
since f
κ
could be an importance function. However, this
may cause data to lose its perceived ordering at the analyti-
cal level. We introduce glyph sorting as one solution for per-
forming interactive sorting in visualization, where one goal
is to use glyphs to sort the data.
4. Design Principles of Sortable Glyphs
The design of glyphs is the process of encoding attributes
of a data entity to a number of visual channels such as size,
colour, and texture that forms a small visual object. Build-
ing on previous works [Ber83, War08a, MPRSDC12], we
propose the following design principles for the creation of
sortable glyphs to be used in interactive sorting as part of a
visualization process.
Typedness: Each dimension in a multivariate dataset may
be of a different data type. Typically, these are classified us-
ing the theory of scales [Ste46] by: nominal, ordinal, in-
terval, and ratio. In addition, direction - a ’sign‘ that de-
notes the directionality of a component (e.g., a vector arrow)
should be considered as an important data type in visualiza-
tion [War08b]. Although hypothetically, we can map all data
types to one or a few visual channels, such as length and
size, it is more appropriate to use visual mappings that intu-
itively convey the underlying data type. For example, in Fig-
ure 2(a) it is clearer to determine the underlying data types
for each dimension in the glyph from the top row (that illus-
trates greater emphasis) than the bottom row (that illustrates
less emphasis). We can visually guess the first and fourth di-
mension (or attribute) to be of either ordinal or nominal type
more easily in the top glyph, since shape is perceived as a
discrete mapping. Similarly, the third and seventh attribute
is of interval type due to its length and position. This cannot
be distinguished from the bottom glyph.
Visual Orderability: Some visual channels (e.g., size,
greyscale intensity) naturally correspond to quantitative
measures that enable a viewer to order different glyphs per-
ceptually, while some others (e.g., an arbitrary set of shapes,
or textures) are much more difficult for viewers to establish
a consistent rule of ordering [War08b, War04]. Figure 2(b)
shows two example glyphs depicting 8 variables of the same
data type. It is easier to visually order the 8 variables in the
top glyph, than the bottom glyph. Additional semantics can
be attached to a visual channel such that it becomes visu-
ally orderable. For instance, scientists often make use of the
colour spectrum to determine the order of colours, which
may not be natural to a child who is unfamiliar with this
concept. In some cases, one may have to use a visual channel
with very poor orderability such as metaphoric pictograms.
The problem can be alleviated by accompanying such vi-
sual channels with an additional channel that is more vi-
sually orderable. For example, different pictograms can be
associated with a background of different greyscales, or a
regular polygonal boundary with different number of edges.
Alternatively, one may carefully design the pictogram set to
4 Chung et al. / Glyph Sorting: Interactive Visualization for Multi-dimensional Data
!"#$%$&'(')*+
,-%"+
("!!+
$."/0-/1
&$($/2"+
3-24!+5+
2-/)"6)+
7'!4$(+
-%8"%$&'(')*+
("$%/$&'(')*+)*#"8/"!!+
29$//"(+
2$#$2')*+
!"$%29$&'(')*+
:$;+ :&;+ :2;+ :8;+ :";+ :3;+ :<;+ :9;+
Figure 2: Variations of glyph design in accordance to the design principles of sortable glyph (a)-(h). For each principle, the
top row depicts a glyph with greater emphasis and the bottom row depicts a glyph with less emphasis.
make some components of pictograms orderable. For exam-
ple, Maguire et al. designs a set of 7 pictograms with incre-
mental number of components to encode levels of material
granularity in biology [MPRSDC12].
Channel Capacity: We adopt this term from information
theory to indicate the number of values that may be encoded
by a visual channel. It is necessary to note that such a capa-
bility value is not an absolute quality, as the number depends
on the size of a glyph as well as many other perceptual fac-
tors such as just noticeable difference [BF93], interference
from nearby visual objects, or from a co-channel in an in-
tegrated channel [She64, HI72]. From the glyph designs in
Figure 2(c), we can clearly observe that the top glyph has
a higher channel capacity since each bar can encode more
values visually (e.g., length, size and colour) than the radial
lines below in which size is not possible. It will always be de-
sirable to use a visual channel with a higher capacity, though
this is often in conflict with other requirements.
Separability: There have been many psychology studies on
the relative merits of separable and integrated visual chan-
nels (e.g., [She64, HI72]). Maguire et al. discuss this re-
quirement in the context of glyph design in [MPRSDC12].
We find that this requirement is particularly important to
glyph sorting. For example, in Figure 2(d), the glyph be-
low encodes 8 variables using 2 integrated channels. Each of
the 4 circles encodes two variables using size and greyscale
intensity. The constructive composition of integral visual
channels makes it more difficult to visually separate in com-
parison for example, the top glyph, where each variable is
mapped to radius length and position. Not only is the per-
ception of individual channel affected by another in an inte-
grated encoding, but also their ordering may demand more
cognitive load in order for a viewer to detach one channel
from another (e.g., intensity and size).
Searchability: For glyphs encoding high-dimensional mul-
tivariate data, it is necessary to help viewers to search
rapidly for a specific variable among many other vari-
ables [War08b]. In Figure 2(e), for example, it will be much
easier to search for a green variable than the 5th variable.
Searchability is affected by many factors [HE12]. One dom-
inant factor is the visual dissimilarity of individual channels.
Hence searchability is closely related to typedness and sep-
arability as mentioned above. It is also related to the spa-
tial organisation of different visual channels such as group-
ing and ordering, as well as design appearance of each vi-
sual channel. In many cases, one has to introduce an ad-
ditional visual channel, such as colour in the top glyph in
Figure 2(e) to help differentiate different variables. Another
factor is learnability, which is to be discussed below.
Learnability: While legends are usually essential to glyph-
based visualization systems, they cannot replace the need
for careful glyph designs to help viewers learn and mem-
orise the association between dimensions and visual chan-
nels without constantly consulting legends. It is desirable
for the appearance of a visual channel to be metaphorically
associated with the semantic meaning of the correspond-
ing dimension [War08b, SJAS05]. One of the most effec-
tive metaphoric designs is to use pictograms. This design
principle was demonstrated by Legg et al. [LCP
∗
12] through
the deployment of glyph-based visualization in sports. Fig-
ure 2(f) shows two different levels of learnability, when for
example one needs to encode the number of greeting cards
in different categories. The glyph on the top row is seman-
tically rich and is much easier to learn than that on the bot-
tom row. However, not all glyph-based visualization can af-
ford pictograms. These constraints can often be alleviated by
making abstract metaphoric association, such as green for
nature, renewable, safe, and so on.
Attention Balance: In multivariate visualization, one com-
mon task is to make observation of the “behaviour” of differ-
ent attributes in relation to the attribute(s) in a sorted order.
While it is helpful to make each individual attribute search-
able [TCW
∗
95, War08b], it is also necessary to avoid unbal-
