Provided by the author(s) and University College Dublin Library in accordance with publisher
policies. Please cite the published version when available.
Title Coping with noisy search experiences
Authors(s) Champin, Pierre-Antoine; Briggs, Peter; Coyle, Maurice; Smyth, Barry
Publication date 2010-05
Publication information Knowledge-Based Systems, 23 (4): 287-294
Publisher Elsevier
Link to online version http://dx.doi.org/10.1016/j.knosys.2009.11.011
Item record/more information http://hdl.handle.net/10197/1999
Publisher's version (DOI) 10.1016/j.knosys.2009.11.011
Downloaded 2022-08-09T21:56:00Z
The UCD community has made this article openly available. Please share how this access
benefits you. Your story matters! (@ucd_oa)
© Some rights reserved. For more information, please see the item record link above.
Coping with Noisy Search Experiences
Pierre-Antoine Champin
1,1
, Peter Briggs
1
, Maurice Coyle
1
, Barry Smyth
1
a
LIRIS, Universit´e de Lyon, CNRS, UMR5205, Universit´e Claude Bernard Lyon 1,
F-69622, Villeurbanne, France,
b
CLARITY: Centre for Sensor Web Technologies, School of Computer Science and
Informatics, University College Dublin, Ireland
Abstract
The so-called Social Web has helped to change the very nature of the Internet
by emphasising the role of our online experiences as new forms of content and
service knowledge. In this paper we describe an approach to improving main-
stream Web search by harnessing the search experiences of groups of like-minded
searchers. We focus on the HeyStaks system (www.heystaks.com) and look in
particular at the experiential knowledge that drives its search recommendations.
Specifically we describe how this knowledge can be noisy, and we describe and
evaluate a recommendation technique for coping with this noise and discuss how
it may be incorporated into HeyStaks as a useful feature.
Experience is the name everyone gives to their mistakes. —Oscar Wilde
Key words: experience, web search, recommender system
1. Introduction
The now familiar Social Web reflects an important change in the nature of
the Web and its content. The development since 1999 of blogs, as a simple way
for users to express their views and opinions, ushered in this new era of user-
generated content (UGC) as many sites quickly began to offer a whole host of
UGC alternatives including the ability to leave comments and write reviews, as
well as the ability to rate or vote on the comments/opinions of others. The result
has been an evolution of the Web from a repository of information to a repository
of experiences, and an increased emphasis on people rather than content. In
combination with social networking services, this has precipitated the growth
of the Social Web as a platform for communication, sharing, recommendation,
and collaboration.
Web search has continued to play a vital role in this evolving online world
and there is no doubting the success of the mainstream Web search engines as
Based on works supported by Science Foundation Ireland, Grant No. 07/CE/I1147, the
French National Center for Scientific Research (CNRS), and HeyStaks Technologies Ltd.
Preprint submitted to Elsevier December 3, 2009
a key information tool for hundreds of millions of users everyday. Given the
importance of Web search it is no surprise that researchers continue to look for
new ways to improve upon the mainstream search engines. However, new tools
are also needed to gather, harness, reuse and share, in the most efficient and
enjoyable way, the experiences captured by UGC [? ? ]. One particular line
of research has focused on using recommendation technologies in an effort to
make Web search more personal: by learning about the preferences and interests
of individual searchers, personalized Web search systems can influence search
results in a manner that better suits the individual searcher [? ? ]. Recently,
another complementary research direction has seen researchers explore the col-
laborative potential of Web search by proposing that the conventional solitary
nature of Web search can be enhanced in many search scenarios by recognising
and supporting the sharing of search experiences to facilitate synchronous or
asynchronous collaboration among searchers [? ? ]. Indeed, the work of [?
? ] has shown that collaborative Web search can lead to a more personalized
search experience by harnessing recommendations from the search experiences
of communities of like-minded searchers.
Our recent work [? ] has led to the development of a new system to support
collaborative Web search. This system is called HeyStaks (heystaks.com) and it
benefits from providing a collaborative search experience that is fully integrated
with mainstream search engines such as Google. HeyStaks comes in the form of
a browser toolbar and, as users search as normal, HeyStaks captures their search
experiences and promotes results based on their past search experiences and the
experiences of friends, colleagues, and other like-minded searchers. HeyStaks
introduces the key concept of a search stak which serves as a repository for search
experiences. Users can create search staks to represent their search interests
and they can share their staks with others to create pools of focused search
experiences.
The key contribution of this paper is to focus on an important challenge
faced by HeyStaks and to propose a recommendation solution to meet this
challenge. The challenge concerns the basic stak selection task: prior to a search,
a HeyStaks user must select an active stak so that their search experiences can
be correctly stored and so that they can receive appropriate recommendations.
Many users have built this into their search workflow and HeyStaks does contain
some simple techniques for automatically switching to the right search stak at
search time. However, many users forget to choose a stak before they search
and, as a result, search experiences are often mis-filed in an incorrect stak.
Ultimately this limits the effectiveness of HeyStaks and contributes significant
experience noise to search staks.
In what follows we will briefly introduce the HeyStaks system. Then we
will describe the development of a stak recommendation technique as part of
HeyStaks’ stak maintenance features, which allow stak owners to review and
edit stak content. In brief, our stak recommender is capable of highlighting
potentially mis-filed experiences and offers the user a suggested target stak that
is expected to provide a better fit. We will describe an evaluation on real-user
search data to demonstrate the effectiveness of this technique.
2
2. HeyStaks: an overview
HeyStaks is a collarative search systems, similar to those presented in [?
? ]. Our primary goal in designing HeyStaks is to help improve upon the
search experience offered by mainstream search engines, while at the same time
allowing searchers to search as normal with their favourite engine. In this section
we will outline the basic HeyStaks system architecture and summarize how
result recommendations are made during search. In addition we will make this
discussion more concrete by briefly summarizing a worked example of HeyStaks
in action.
2.1. Concepts and Architecture
HeyStaks adds two important collaboration features to any mainstream
search engine. First, it captures users’ experiences in using the search engine,
and store them in search staks. Staks are a type a of folder that users can create
to store search experiences related to a given topic of interest. Staks can also
be shared with others so that their own searches will also be added to the stak.
Second, HeyStaks uses staks to generate recommendations that are added to the
underlying search results that come from the mainstream search engine. These
recommendations are results that stak members have previously found to be
relevant for similar queries in the context of this stak, and help the searcher to
discover results that friends or colleagues have found relevant, results that may
otherwise be buried deep within the engine’s result-list.
HeyStaks takes the form of two basic components: a client-side browser
toolbar and a back-end server. The toolbar allows users to create and share
staks and provides a range of ancillary services, such as the ability to tag or
vote for pages. The toolbar also captures search result click-thrus and man-
ages the integration of HeyStaks recommendations with the default result-list.
The back-end server manages the individual stak indexes (indexing individual
pages against query/tag terms and positive/negative votes), the stak database
(stak titles, members, descriptions, status, etc.), the HeyStaks social networking
service and the recommendation engine.
2.2. Running Example
To make things more concrete, consider the following example. Pierre, Mau-
rice and some colleagues are using the LaTeX typesetting system on a regular
basis, and Web search as a source of information about how to use it. Pierre cre-
ated a search stak called “LaTeX” and shared this with Maurice and colleagues,
encouraging them to use this stak for their LaTeX-related searches.
Fig. ?? shows Maurice selecting this stak as he embarks on a new search
about the tabular environment, and Fig. ?? shows the results of this search.
The usual Google results are shown, but in addition HeyStaks has made one
promotion. This was promoted because other members of the “LaTeX” stak
had recently found these results to be relevant; perhaps they selected them for
similar queries, or voted for them, or tagged them with related terms. These
recommendations may have been promoted from much deeper within the Google
3
The Stak−List
The HeyStaks Toolbar
Tag, Share, Vote Actions
Create, Share, Remove Staks
Figure 1: Selecting a new active stak.
result-list, or they may not even be present in Google’s default results. Other
relevant results may also be highlighted by HeyStaks, but left in their default
Google position. In this way Pierre and Maurice benefit from promotions that
are based on their previous similar searches. In addition, HeyStaks can recom-
mend results from Pierre and Maurice’s other staks, helping them to benefit
from the search knowledge that other groups and communities have created.
Separately from the toolbar, HeyStaks users also benefit from the HeyStaks
search portal, which provides a social networking service built around people’s
search histories. For example, Fig. ?? shows the portal page for the “LaTeX”
stak. It presents an activity feed of recent search history and a query cloud that
makes it easy for the user to find out about what others have been searching
for. The search portal also provides users with a wide range of features such as
stak maintenance (e.g., editing, moving, copying results in staks and between
staks), various search and filtering tools, and a variety of features to manage
their own search profiles and find new search partners.
2.3. Generating Recommendations
In HeyStaks each search stak (S) serves as a profile of the search activities of
the stak members. Each stak is made up of a set of result pages (S = {p
1
, ..., p
k
})
and each page is anonymously associated with a number of implicit and explicit
interest indicators, including the total number of times a result has been selected
(sel), the query terms (q
1
, ..., q
n
) that led to its selection, the number of times a
result has been tagged (tag), the terms used to tag it (t
1
, ..., t
m
), the votes it has
received (v
+
, v
−
), and the number of people it has been shared with (share).
In this way, each page is associated with a set of term data (query terms
and/or tag terms) and a set of usage data (the selection, tag, share, and voting
counts). The term data is stored as a Lucene (lucene.apache.org) index, with
each page indexed under its associated query and tag terms, and provides the
basis for retrieving and ranking promotion candidates. The usage data provides
4
