The major change in the second edition of this book is the addition of a new chapter on probabilistic retrieval. This chapter has been included because I think this is one of the most interesting and active areas of research in information retrieval. There are still many problems to be solved so I hope that this particular chapter will be of some help to those who want to advance the state of knowledge in this area. All the other chapters have been updated by including some of the more recent work on the topics covered. In preparing this new edition I have benefited from discussions with Bruce Croft, The material of this book is aimed at advanced undergraduate information (or computer) science students, postgraduate library science students, and research workers in the field of IR. Some of the chapters, particularly Chapter 6 * , make simple use of a little advanced mathematics. However, the necessary mathematical tools can be easily mastered from numerous mathematical texts that now exist and, in any case, references have been given where the mathematics occur. I had to face the problem of balancing clarity of exposition with density of references. I was tempted to give large numbers of references but was afraid they would have destroyed the continuity of the text. I have tried to steer a middle course and not compete with the Annual Review of Information Science and Technology. Normally one is encouraged to cite only works that have been published in some readily accessible form, such as a book or periodical. Unfortunately, much of the interesting work in IR is contained in technical reports and Ph.D. theses. For example, most the work done on the SMART system at Cornell is available only in reports. Luckily many of these are now available through the National Technical Information Service (U.S.) and University Microfilms (U.K.). I have not avoided using these sources although if the same material is accessible more readily in some other form I have given it preference. I should like to acknowledge my considerable debt to many people and institutions that have helped me. Let me say first that they are responsible for many of the ideas in this book but that only I wish to be held responsible. My greatest debt is to Karen Sparck Jones who taught me to research information retrieval as an experimental science. Nick Jardine and Robin …

Information retrieval

Recommender systems are now popular both commercially and in the research community, where many algorithms have been suggested for providing recommendations. These algorithms typically perform differently in various domains and tasks. Therefore, it is important from the research perspective, as well as from a practical view, to be able to decide on an algorithm that matches the domain and the task of interest. The standard way to make such decisions is by comparing a number of algorithms offline using some evaluation metric. Indeed, many evaluation metrics have been suggested for comparing recommendation algorithms. The decision on the proper evaluation metric is often critical, as each metric may favor a different algorithm. In this paper we review the proper construction of offline experiments for deciding on the most appropriate algorithm. We discuss three important tasks of recommender systems, and classify a set of appropriate well known evaluation metrics for each task. We demonstrate how using an improper evaluation metric can lead to the selection of an improper algorithm for the task of interest. We also discuss other important considerations when designing offline experiments.

/pdf/a-survey-of-accuracy-evaluation-metrics-of-recommendation-5eusrb5kdb.pdf

A Survey of Accuracy Evaluation Metrics of Recommendation Tasks

The way a choice is presented influences what a decision-maker chooses. This paper outlines the tools available to choice architects, that is anyone who present people with choices. We divide these tools into two categories: those used in structuring the choice task and those used in describing the choice options. Tools for structuring the choice task address the idea of what to present to decision-makers, and tools for describing the choice options address the idea of how to present it. We discuss implementation issues in using choice architecture tools, including individual differences and errors in evaluation of choice outcomes. Finally, this paper presents a few applications that illustrate the positive effect choice architecture can have on real-world decisions.

/pdf/beyond-nudges-tools-of-a-choice-architecture-4h6586rt6m.pdf

Beyond nudges: Tools of a choice architecture

The way a choice is presented influences what a decision-maker chooses.This paper outlines the tools available to choice architects, that is anyone who present people with choices. We divide these tools into two categories: those used in structuring the choice task and those used in describing the choice options. Tools for structuring the choice task address the idea of what to present to decision-makers, and tools for describing the choice options address the idea of how to present it. We discuss implementation issues in using choice architecture tools, including individual differences and errors in evaluation of choice outcomes. Finally, this paper presents a few applications that illustrate the positive effect choice architecture can have on real-world decisions.

Beyond Nudges: Tools of a Choice Architecture

Regression test prioritization is often performed in a time constrained execution environment in which testing only occurs for a fixed time period. For example, many organizations rely upon nightly building and regression testing of their applications every time source code changes are committed to a version control repository. This paper presents a regression test prioritization technique that uses a genetic algorithm to reorder test suites in light of testing time constraints. Experiment results indicate that our prioritization approach frequently yields higher average percentage of faults detected (APFD) values, for two case study applications, when basic block level coverage is used instead of method level coverage. The experiments also reveal fundamental trade offs in the performance of time-aware prioritization. This paper shows that our prioritization technique is appropriate for many regression testing environments and explains how the baseline approach can be extended to operate in additional time constrained testing circumstances.

/pdf/timeaware-test-suite-prioritization-4dx9v2z1i4.pdf

TimeAware test suite prioritization

We propose an approach to recommendation systems that optimizes over possible sets of recommended alternatives in a decision-theoretic manner. Our approach selects the alternative set that maximizes the expected valuation of the user's choice from the recommended set. The set-based optimization explicitly recognizes the opportunity for passing residual uncertainty about preferences back to the user to resolve. Implicitly, the approach chooses a set with a diversity of alternatives that optimally covers the uncertainty over possible user preferences. The approach can be used with several preference representations, including utility theory, qualitative preferences models, and informal scoring. We develop a specific formulation for multi-attribute utility theory, which we call maximization of expected max (MEM). We go on to show that this optimization is NP-complete (when user preferences are described by discrete distributions) and suggest two efficient methods for approximating it. These approximations have complexity of the same order as the traditional k-max operator and, for both synthetic and real-world data, perform better than the approach of recommending the k-individually best alternatives (which is not a surprise) and very close to the optimum set (which is less expected).

/pdf/optimal-recommendation-sets-covering-uncertainty-over-user-56bg6shf4u.pdf

Optimal recommendation sets: covering uncertainty over user preferences

Interface personalization can improve a user's performance and subjective impression of interface quality and responsiveness. Personalization is difficult to implement as it requires an accurate model of a user's intentions and a formal model of how an interface meets a user's need. We present a novel model for tractable inference of consumer intentions in the context of grocery shopping. The model makes unique use of a priori temporal relations to simplify inference. We then present a simple interface generation framework that was inspired by viewing user interface interaction as a channel coding problem. The resulting model defines a simplified but clear notion of a user's utility for an interface. We demonstrate the effectiveness of the research prototype on some simple data, and explain how the model can be augmented with richer user modeling to create a deployable application.

/pdf/automatic-construction-of-personalized-customer-interfaces-4ekv7gpgta.pdf

Automatic construction of personalized customer interfaces

We introduce our research on learning browsing behavior models for inferring a user's information need (corresponding to a set of words) based on the actions he has taken during his current web session This information is then used to find relevant pages, from essentially anywhere on the web The models, learned from over one hundred users during a fiveweek user study, are session-specific but independent of both the user and website Our empirical results suggest that these models can identify and satisfy the current information needs of users, even if they browse previously unseen pages containing unfamiliar words

/pdf/using-learned-browsing-behavior-models-to-recommend-relevant-uwdkfwcdt4.pdf

Using learned browsing behavior models to recommend relevant web pages

This paper introduces a novel method for predicting the current information need of a web user from the content of the pages the user has visited and the actions the user has applied to these pages. This inference is based on a parameterized model of how the sequence of actions chosen by the user indicates the degree to which page content satisfies the user's information need. We show that the model parameters can be estimated using standard methods from a labelled corpus. Data from lab experiments demonstrate that the prediction model can effectively identify the information needs of new users, browsing previously unseen pages. The paper concludes with an overview of our complete-web recommendation system, WebIC, which uses the prediction model to recommend useful pages to the user, from anywhere on the Web.

Bob Price

Papers

Optimal recommendation sets: covering uncertainty over user preferences

Automatic construction of personalized customer interfaces

Using learned browsing behavior models to recommend relevant web pages

Predicting web information content

Behavior-Based Recommender Systems for Web Content : Workshop on the Next Stage of Recommender Systems Research (Beyond Personalization 2005)