Looking for Information -- A Survey of Research on Information Seeking, Needs, and Behavior by Donald Case

Evaluation is central in research and development of information retrieval (IR). In addition to designing and implementing new retrieval mechanisms, one must also show through rigorous evaluation that they are effective. A major focus in IR is IR mechanisms’ capability of ranking relevant documents optimally for the users, given a query. Searching for information in practice involves searchers, however, and is highly interactive. When human searchers have been incorporated in evaluation studies, the results have often suggested that better ranking does not necessarily lead to better search task, or work task, performance. Therefore, it is not clear which system or interface features should be developed to improve the effectiveness of human task performance. In the present article, we focus on the evaluation of task-based information interaction (TBII). We give special emphasis to learning tasks to discuss TBII in more concrete terms. Information interaction is here understood as behavioral and cognitive activities related to task planning, searching information items, selecting between them, working with them, and synthesizing and reporting. These five generic activities contribute to task performance and outcome and can be supported by information systems. In an attempt toward task-based evaluation, we introduce program theory as the evaluation framework. Such evaluation can investigate whether a program consisting of TBII activities and tools works and how it works and, further, provides a causal description of program (in)effectiveness. Our goal in the present article is to structure TBII on the basis of the five generic activities and consider the evaluation of each activity using the program theory framework. Finally, we combine these activity-based program theories in an overall evaluation framework for TBII. Such an evaluation is complex due to the large number of factors affecting information interaction. Instead of presenting tested program theories, we illustrate how the evaluation of TBII should be accomplished using the program theory framework in the evaluation of systems and behaviors, and their interactions, comprehensively in context.

Task-Based Information Interaction Evaluation: The Viewpoint of Program Theory

The field of human information behavior runs the gamut of processes from the realization of a need or gap in understanding, to the search for information from one or more sources to fill that gap, to the use of that information to complete a task at hand or to satisfy a curiosity, as well as other behaviors such as avoiding information or finding information serendipitously. Designers of mechanisms, tools, and computer-based systems to facilitate this seeking and search process often lack a full knowledge of the context surrounding the search. This context may vary depending on the job or role of the person; individual characteristics such as personality, domain knowledge, age, gender, perception of self, etc.; the task at hand; the source and the channel and their degree of accessibility and usability; and the relationship that the seeker shares with the source. Yet researchers have yet to agree on what context really means. While there have been various research studies incorporatin context, and biennial conferences on context in information behavior, there lacks a clear definition of what context is, what its boundaries are, and what elements and variables comprise context. In this book, we look at the many definitions of and the theoretical and empirical studies on context, and I attempt to map the conceptual space of context in information behavior. I propose theoretical frameworks to map the boundaries, elements, and variables of context. I then discuss how to incorporate these frameworks and variables in the design of research studies on context. We then arrive at a unified definition of context. This book should provide designers of search systems a better understanding of context as they seek to meet the needs and demands of information seekers. It will be an important resource for researchers in Library and Information Science, especially doctoral students looking for one resource that covers an exhaustive range of the most current literature related to ontext, the best selection of classics, and a synthesis of these into theoretical frameworks and a unified definition. The book should help to move forward research in the field by clarifying the elements, variables, and views that are pertinent. In particular, the list of elements to be considered, and the variables associated with each element will be extremely useful to researchers wanting to include the influences of context in their studies.

Exploring Context in Information Behavior: Seeker, Situation, Surroundings, and Shared Identities

Evaluation of information during information problem-solving processes already starts when trying to select the appropriate search result on a search engine results page SERP. Up to now, research has mainly focused on the evaluation of webpages, while the evaluation of SERPs received less attention. Furthermore, task complexity is often not taken into account. A within-subjects design was used to study the influence of task complexity on search query formulation, evaluation of search results, and task performance. Three search tasks were used: a fact-finding, cause-effect, and a controversial topic task. To measure perceptual search processes, we used a combination of log files, eye-tracking data, answer forms, and think-aloud protocols. The results reveal that an increase in task complexity results in more search queries and used keywords, more time to formulate search queries, and more considered search results on the SERPs. Furthermore, higher ranked search results were considered more often than lower ranked results. However, not all the results for the most complex task were in line with expectations. These conflicting results can be explained by a lack of prior knowledge and the possible interference of prior attitudes.

Effects of task complexity on online search behavior of adolescents

This study aims to explore the relationships between user interaction and digital libraries (DLs) evaluation. User interaction is a multi-dimensional construct and recognized as three dimensions in this study, as user interaction with: information resource; interface; and, tasks. DL evaluation is considered from the user's perspective and defined as users’ perception of DL performance from different perspectives, including the support of DL's interaction design to user interaction (labeled as interaction-design-based (IDB) evaluation), the support of task completion (labeled as task-based evaluation), and a DL's overall performance (labeled as overall evaluation). An experiment with 48 participants was conducted using the China National Knowledge Infrastructure (CNKI ( http://cnki.net/ ), the most widely used digital library in China). Participants searched for four simulated work tasks and one real work task during the experiment, subsequently evaluating their interaction with information resource, interface, and tasks, and DL performance from different perspectives before or after the search. Correlation analysis and stepwise regression analysis were conducted to examine the relationships. The results indicate that a list of factors related to different dimensions of user interaction can significantly predict or be correlated to users’ evaluation of DL performance from different perspectives, including appropriateness, rich and valid links, reasonable page layout, salience of topics, search task difficulty, well-organized web site, easy to learn, accessibility, usefulness, familiarity with task procedure, etc. These factors surface as the most critical criteria for DL evaluation. Based on the results, an integrated DL evaluation framework is developed. The study adds new knowledge about how tasks affect DL evaluation. It has implications for improving the efficiency of DL evaluation and helping DL developers design DLs to better support users’ interaction, task completion, and their overall experience with DLs.

Information Resource, Interface, and Tasks as User Interaction Components for Digital Library Evaluation

The aim of the study was to test whether query expansion by approximate string matching methods is beneficial in retrieval from historical newspaper collections in a language rich with compounds and inflectional forms Finnish. First, approximate string matching methods were used to generate lists of index words most similar to contemporary query terms in a digitized newspaper collection from the 1800s. Top index word variants were categorized to estimate the appropriate query expansion ranges in the retrieval test. Second, the effectiveness of approximate string matching methods, automatically generated inflectional forms, and their combinations were measured in a Cranfield-style test. Finally, a detailed topic-level analysis of test results was conducted. In the index of historical newspaper collection the occurrences of a word typically spread to many linguistic and historical variants along with optical character recognition OCR errors. All query expansion methods improved the baseline results. Extensive expansion of around 30 variants for each query word was required to achieve the highest performance improvement. Query expansion based on approximate string matching was superior to using the inflectional forms of the query words, showing that coverage of the different types of variation is more important than precision in handling one type of variation.

Information retrieval from historical newspaper collections in highly inflectional languages: A query expansion approach

Information searching in practice seldom is an end in itself. In work, work task WT performance forms the context, which information searching should serve. Therefore, information retrieval IR systems development/evaluation should take the WT context into account. The present paper analyzes how WT features: task complexity and task types, affect information searching in authentic work: the types of information needs, search processes, and search media. We collected data on 22 information professionals in authentic work situations in three organization types: city administration, universities, and companies. The data comprise 286 WTs and 420 search tasks STs. The data include transaction logs, video recordings, daily questionnaires, interviews. and observation. The data were analyzed quantitatively. Even if the participants used a range of search media, most STs were simple throughout the data, and up to 42% of WTs did not include searching. WT's effects on STs are not straightforward: different WT types react differently to WT complexity. Due to the simplicity of authentic searching, the WT/ST types in interactive IR experiments should be reconsidered.

/pdf/search-task-features-in-work-tasks-of-varying-types-and-316saf1600.pdf

Search task features in work tasks of varying types and complexity

In task-based information searching, the task at hand is a central factor affecting information search. Task complexity, in particular, has been discovered to affect searching. In the present study, we shadowed the tasks of seven people working in city administration. The data consist of shadowing field notes, voice recordings, photographs and forms. We study, how task complexity affects information searching and information resource use. Task complexity was defined through the task performer's own experience (perceived task complexity) and her estimates of her a priori knowledge concerning the task. We analyzed the data both qualitatively and quantitatively, focusing on the links between task complexity and the use of information resources, information searching and problems encountered. We found that task complexity has a central but ambiguous relationship to task performance. The clearest differences were found between simple and complex tasks. In addition, perceived task complexity seems to affect the ways of performing the task more than a priori knowledge. The more complex a task is perceived, the more searches are performed and the more they concentrate on networked resources instead of information systems provided by the organization (SPOs). The use of resources on the task performer's PC and the SPOs decreases when complexity increases. In proportion, the use of networked resources and communication resources increases. The total number of information resources used is somewhat greater in complex and semi-complex tasks than in simple tasks; and each resource is used for a longer time on average. Our study shows that task context and especially task complexity seems to affect information searching and the selection of sources.

Task complexity and information searching in administrative tasks revisited

Information seeking research often reports about types of information resources, ways of acquiring them and opinions on their importance in various professions. Based on self-reporting, these findi...

Miamaria Saastamoinen

Papers

Task-Based Information Interaction Evaluation: The Viewpoint of Program Theory

Information retrieval from historical newspaper collections in highly inflectional languages: A query expansion approach

Search task features in work tasks of varying types and complexity

Task complexity and information searching in administrative tasks revisited

Relationships between work task types, complexity and dwell time of information resources: