Visualization Methods for Personal Photo Collections:
Browsing and Searching in the PhotoFinder
Hyunmo
Kang,
Ben Shneideman
Department
of
Computer Science, Human-Computer Interaction Laboratory
University
of
Maryland
at
College
Park
College
Park,
MD 20742
USA
+1
301
405
2725
{
kang,
ben} @cs.umd.edu
ABSTRACT
Software tools for personal photo collection management
are
proliferating, but they usually have limited searching and
browsing functions. We implemented the PhotoFinder prototype
to enable non-technical users of personal photo collections to
search and browse easily. PhotoFinder provides a set of visual
Boolean query interfaces, coupled with dynamic query and query
preview features. It gives users powerful search capabilities.
Using a scatter plot thumbnail display and drag-and-drop
interface, PhotoFinder is designed to be easy to use for searching
and browsing photos.
Keywords
:
PhotoFinder, user interface, dynamic query, query
preview, search, browsing, Boolean query, digital photo library.
1.
INTRODUCTION
Digital cameras, scanners and personal computers are now
common. But as collections grow in size, the need to organize,
search, and browse digital photos increases
[l].
There are many
personal photo collection management tools available either
commercially or non-commercially.
A
search with terms "digital
photo" and "software download" on ZDNet.com
or
CNet.com
will retum many hits pointing to software entries. These software
products vary substantially in terms of features and quality. We
find what most
of
these products lack are easy-to-learn and
efficient searching and browsing capabilities, especially for
collections of photos. This has been a long-standing problem for
digital photo library tools. PhotoFinder was built as an attempt to
address some of these problems. We implemented some novel
widgets that can be used to do powerful Boolean queries,
A
variety
of
information visualization techniques, such as dynamic
queries and query previews
are
employed to provide facilities to
search and browse easily and effectively. We also designed
interfaces to cope with missing data. This paper describes the
features implemented in the PhotoFinder prototype and discusses
their advantages and disadvantages.
2.
DESIGN
OUTLINES
Influenced by some earlier projects at the Human Computer
Interaction Lab.(HCIL), University
of
Maryland at College Park,
our
design for PhotoFinder employs the following strategies:
powerful Boolean queries to select desired photos
dynamic queries to rapidly filter and view results
query previews
to
show cardinality of result sets
special fields
to
handle missing data for an attribute
0-7803-6536-4/00/$10.00
(c)
2000
IEEE
2.1
User Interface Strategies
We wanted to use a direct manipulation interface with visual
representation for the search space. This allows rapid,
incremental, and reversible exploratory actions. We wanted to
construct visual widgets to simplify the operations for searching
and browsing: e.g. pointing and selecting will be favored over
typing. The interface should produce immediate, continuous and
visual feedback of search/query results.
The PhotoFinder prototype has a three-window strategy:
a
search
tool window (top level), a thumbnail browser window
(intermediate level), a detail viewer window (bottom level). This
follows the tried and tested visual information seeking strategy:
"Overview first,
zoom
and filter, and then details on demand"[2].
We combined dynamic query techniques with query previews
that show the cardinality of result sets, thereby guiding users to
avoid zero-hit queries.
The three-window strategy tightly coupled displays to
form
a
coherent view to the data that users are searching and browsing.
2.2
Exploring PhotoFinder
PhotoFinder allows users to explore a library that consists of
multiple collections. Users begin with the collection browser to
select
a
single collection for detailed study. The collection
browser shows a representative photo for each collection (Figure
1).
There are three major parts
of
the PhotoFinder:
The
search
tool
window
contains a series of tabbed panels of
dialog boxes for setting query attributes. On each tabbed panel,
an attribute distribution chart shows the number of hits for those
attributes. Innovative multiple two-sided slider bars are used for
query range selections. The overview window gives users three
filtering mechanisms
to
display photo thumbnails: show all hits
for the current query (default behavior), show hits for each
independent attribute query (if "Show Thumbnail" button is
clicked), and show hits for the corresponding histogram
bar
(if a
histogram bar is clicked and highlighted).
There are two major display modes for the
thumbnail browser
window:
sequential display and scatter plot display. Zooming is
provided to better use the display space and for the ease of
navigation. Users can also just point and click to locate and
select an individual picture or a set of pictures in the thumbnail
browser window.
The
detail viewer window
displays the full-size picture selected
from thumbnail browser and its annotation.
A
slide show
function is also implemented to allow users to display
a
set of
selected pictures one after another. The detail viewer window
also
provides facilities for annotating a single picture.
1539
2.3
Implementation Environment
PhotoFinder is implemented using Microsoft Visual Basic
6.0.
The sample photo library was implemented and maintained as
a
Microsoft Access database. Through ODBC, PhotoFinder is able
to connect to and query the database.
3.
SYSTEM
FEATURES
3.1
Boolean
Query
and Search Interface
Enabling users to do Boolean queries visually is a major human-
computer interaction topic. Many approaches have been
suggested with different degrees of implementation difficulties
and different degrees of success
[5][8].
To provide full Boolean
query power in a visual form is difficult. In our experience, users
of digital photo libraries tend
to
perform Boolean queries for
searching pictures with some limited patterns
[6].
Namely, users
tend to use Boolean queries as follows:
For example, users may want to find pictures
“ofAnne
OR
Bob”
that
(AND)
“were taken in
1999“,
but not pictures
“of
Anne
OR
those taken in
Las
Vegas“.
make
OR
queries within an attribute;
make AND queries across different attributes;
Our approach for disjunctive
(OR)
queries within attributes is to
use MTS (Multiple-Two-sided) slider bars for each attribute.
The MTS slider bar enables disjunctive range selections (e.g. 2-4,
7-9,
14-21). Traditional double-sided slider bars give the users
the ability to select just one range. Using
MTS,
users can specify
any number
of
disjunctive ranges
of
one attribute (Figure
2).
Eleven tabbed panels are
used
to represent query for different
attributes
(10
attributes plus one for “All” attributes). The
relationship among these attributes in forming a query is by
default conjunctive (AND). The ”All Attributes” tabbed panel
shows all the query parameter settings in a compact display.
Histograms are used to show query results for each individual
attribute.
The yellow part of each histogram bar (bright
part
in black and
white) indicates the “hits” of query results. The gray part of the
histogram bar indicates the “non-selected photos”. Although the
relationship among query attributes defaults to be conjunctive,
users can still
form
complicated queries involving disjunctive
relations among attributes if they
so
desire. This is achieved by
sequentially performing different queries and merging the query
results.
Figure
1.
PhotoFinder (with search tool, thumbnail browser, and detail viewer windows)
plus the overlaid Collection Browser to select a set of photos to explore.
0-7803-6536-4/00/$10.00
(c)
2000
IEEE
1540
Missing data are displayed as an additional value in the
histogram (Figure
2,
left side). Therefore there is no loss of
information in visualizing the contents of the collection.
3.3 Multiple Displays in
a
Thumbnail Browser
The thumbnail browser is designed to hold and display the query
result for users to browse and select photos for detailed viewing.
By the nature of the complexity of the photo attributes, no single
way of displaying the thumbnails will fit all users' needs.
Therefore our browser has multiple displays of thumbnails.
In the sequential display of photos, one picture is placed after
another. Due to different sizes and orientations of photos, a large
portion
of
the display space
is
wasted white space. This is the
non-compact mode, which is the default mode. If "Compact"
mode is selected, our heuristics will re-order the thumbnails
so
that pictures of the same size and orientations will be placed
together to reduce the wasted white space. Figures 3a-b show the
effect of non-compact and compact mode of sequential display.
The compact mode is especially useful when the order of the
photos is not important to users
so
that the display space can be
better utilized.
Figure
2.
Examples of Dynamic Query Preview. The slider bars
under the histograms are MTS slider bars where multiple
disjunctive range selections can be specified by using multiple
range sliders. On the left, a separate attribute value is used to
represent data with missing a date.
3.2 Dynamic Queries
&
Query Previews
Dynamic query techniques have proved to be comprehensible
and effective
[2][4].
Immediate feedback often provides more
information than statically displayed query results. In
PhotoFinder, dynamic queries are also used to enhance the
searching capabilities. Query results are updated in real time for
previews in the search panel, and the thumbnail browser window
is
also
updated in real time for each query setting change.
Dynamic filtering
is
made possible by clicking on individual
histogram bars to further filter the query results. When this
happens, only pictures with that particular histogram bar value
will be shown. In all attribute panels, query result histograms are
updated to reflect the effect of query changes.
There are also many advantages
of
query previews which show
the distribution of data
[7].
In
PhotoFinder, we provided a
histogram chart for each attribute as a query preview tool. The
users can see the distribution of pictures for each attribute in
advance and therefore gain a better understanding of the photo
collection. The histogram chart also gives users immediate
feedback of the number
of
hits for the query they are performing.
Query parameters can then be adjusted to refine the query. It
presumably will prevent users from performing zero-hit queries.
0-7803-6536-4/00/$10.00 (c)
2000
IEEE
Figure
3a.
Non-Compact Mode of Sequential Disdav
..
..
Figure
3b.
Compact Mode of Sequential Display
Users can choose to have a scatter plot display for the thumbnail
pictures. Like Spotfire (www.spotfire.com), such a scatter plot
display, coupled with the flexibility of choosing attributes for the
axes, often provides users with quick and insightful
understanding of the pictures displayed (Figure 4). By choosing
different axes, the distribution of pictures can reveal patterns that
are
difficult to find in a textual tabular display.
3.4 Drag and
Drop
to the Detail Viewer Window
From the thumbnail browser window, users can point and click
to
select a single picture or use a rubber rectangle to select a
group
of
pictures. After the selection is made, users can drag and
drop the selection into the detail viewer window for viewing
individual photos or a slide show. Such an interface should be
easy to use, and we believe it makes browsing photos more
enjoyable. Figure
5
shows how drag and drop works for the
detail viewer window.
1541
Figure
4.
Scatter Plot Display of Thumbnail Browser with
Zooming Function. The x-axis is number of people in a picture
and the y-axis is the picture rating (1-9, 9 is best). While most
pictures in this collection have only a few people in it, the one in
the northeast comer has 7 people. The distribution of pictures in-
this scatter plot also shows that pictures of multiple people tend
to
have higher ratings.
Figure
5.
Drag and Drop to the Detail Viewer Window: one
picture is shown in the detail viewer window, while a group
selection is performed in the thumbnail browser. The selection is
being dragged to the detail viewer window to create a slide show.
4.
FUTUREWORK
We feel our design and implementation of PhotoFinder have
many advantages over most of the personal photo collection
management products currently available. Still, to make a really
good product for home users to organize, browse and search their
growing digital photo library is hard. We realize PhotoFinder has
many limitations. There is still a lot of work needed in terms of
features and interfaces. Currently, we have not integrated
PhotoFinder with a proper Collection Browser.
To
combine
PhotoFinder with
a
Collection Browser will give users a more
complete solution to their personal photo library organization
problem. Planned future work includes the implementation of a
Q-78Q3-6536-4/00/$10.00 (c) 2000
IEEE
histogram display in the thumbnail browser. We also have plans
to implement an automatic annotation mechanism
so
that the
users can just use a mouse to click, drag and drop to make
annotations instead of repetitively typing in text fields. These
techniques can be further combined with functionality
for
annotation of single photos, groups of selected photos, or entire
collections of photos. Our future work includes user studies of
PhotoFinder and controlled experiments [3].
5.
CONCLUSION
We implemented PhotoFinder to provide better searching and
browsing capabilities than in existing software tools. By using a
novel attribute selection query interface, PhotoFinder gives users
nearly full Boolean query power with high ease of use.
Information visualization techniques, such as dynamic query and
query preview, further strengthen the search capabilities of
PhotoFinder. We believe that multiple forms of thumbnail
display and a drag-and-drop interface make PhotoFinder an
enjoyable
and
powerful photo-browsing tool.
Acknowledgements:
We appreciate the partial support
of
Intel, and
input
from
colleagues: Ben Bederson, Hilary Browne,
Todd
Carlough,
Tammara Combs, Hatim Khan, Manav
Kher,
Catherine Plaisant, and Jin
Tong.
6.
REFERENCES
[l]
Allan Kuchinsky, Celine Pering, Michael L. Creech, Dennis
Freeze, Bill Serra, Jacek Gwizdka, "FotoFile: A Consumer
Multimedia Organization and Retrieval System",
Proc.
ACM CHI99 Conference
on
Human Factors in Computing
Systems,
pp. 496-503, May, 1999.
[2] C. North,
B.
Shneiderman, C. Plaisant, "User Controlled
Overviews of an Image Library: A Case Study of the Visible
Human",
Proc. ACM Digital Libraries
'96
Conference,
ACM Press, New York, pp. 74-82, 1996.
[3] Joemon M.Jose, Jonathan Fumer, and David J.Harper,
"Spatial querying for image retrieval: a user-oriented
evaluation",
Pr0.e 21st Annual International ACM SIGIR88
Conference
on
Research and Development in Information
Retrieval,
pp. 232-240, August, 1998.
[4] Ken Fishkin, Maureen C. Stone, "Enhanced Dynamic
queries via Movable Filters",
Proc. ACM CHI95 Conference
on
Human Factors in Computing Systems,
pp. 415-420,
May, 1995.
[
51 Michael Chui, "Pattem, Procedurality
&
Pictures: Factors
Affecting Boolean Query Interface Design for the Web",
Proc. ACM SIGIR99.
htt~:Nwww.cs.indiana.edul-mchuilsinir99/
[6] Sharon
R.
Garber, Mitch B. Grunes, "The
Art
of Search:
A
Study
of
Art
Directors",
Proc. ACM CHI92 Conference
on
Human Factors in Computing Systems,
pp. 157-163, 1992.
[7] Stephan Greene, Gary Marchionini, Catherine Plaisant, and
Ben Shneiderman, "Previews and Overviews in Digital
Libraries: Designing Surrogates to Support Visual
Information Seeking",
University of Maryland Technical
Report CS-TR-3838,
1997.
[8] Sylvia Willie, "Query Context: Would a graphical interface
help?",
School
of
Information Systems, Queensland
University
of
Technology
http://www.icis.sut.edu.au/-wjllie/My
Pa~ers/ozchi-94.html
1542