Interactive Information Retrieval in Digital Environments

Abstract: The present paper reports on the initial study and the preliminary findings of how the concept of simulated work task situation is reported used in the research literature. The overall objective of the study is in a systematic manner to learn how and for what types of evaluations the concept is applied. In particular we are interested to learn whether the recommendations for how to apply simulated work task situations are followed.The preliminary findings indicate a need for clarifications of the recommendations of how to use simulated work task situations. Particularly with respect to 'realism' of the simulated work task situations, which is emphasised through the need for tailoring of the simulated work task situations towards the group of study participant to ensure the depicted situations are realistic and interesting from the participants' point of view. Likewise it seems that the recommendation to involve the study participants' own information needs (to function as baseline of search interaction) is generally neglected in the reported studies.

Abstract: Evaluation is central in research and development of information retrieval (IR). In addition to designing and implementing new retrieval mechanisms, one must also show through rigorous evaluation that they are effective. A major focus in IR is IR mechanisms’ capability of ranking relevant documents optimally for the users, given a query. Searching for information in practice involves searchers, however, and is highly interactive. When human searchers have been incorporated in evaluation studies, the results have often suggested that better ranking does not necessarily lead to better search task, or work task, performance. Therefore, it is not clear which system or interface features should be developed to improve the effectiveness of human task performance. In the present article, we focus on the evaluation of task-based information interaction (TBII). We give special emphasis to learning tasks to discuss TBII in more concrete terms. Information interaction is here understood as behavioral and cognitive activities related to task planning, searching information items, selecting between them, working with them, and synthesizing and reporting. These five generic activities contribute to task performance and outcome and can be supported by information systems. In an attempt toward task-based evaluation, we introduce program theory as the evaluation framework. Such evaluation can investigate whether a program consisting of TBII activities and tools works and how it works and, further, provides a causal description of program (in)effectiveness. Our goal in the present article is to structure TBII on the basis of the five generic activities and consider the evaluation of each activity using the program theory framework. Finally, we combine these activity-based program theories in an overall evaluation framework for TBII. Such an evaluation is complex due to the large number of factors affecting information interaction. Instead of presenting tested program theories, we illustrate how the evaluation of TBII should be accomplished using the program theory framework in the evaluation of systems and behaviors, and their interactions, comprehensively in context.

Abstract: The present study reports on the information seeking processes in a visual context, referred to throughout as visual information seeking. This study synthesizes research throughout different, yet complementary, areas, each capable of contributing findings and understanding to visual information seeking. Methods previously applied for examining the visual information seeking process are reviewed, including interactive experiments, surveys, and various qualitative approaches. The methods and resulting findings are presented and structured according to generalized phases of existing information seeking models, which include the needs, actions, and assessments of users. A review of visual information needs focuses on need and thus query formulation; user actions, as reviewed, centers on search and browse behaviors and the observed trends, concluded by a survey of users' assessments of visual information as part of the interactive process. This separate examination, specific to a visual context, is significant; visual information can influence outcomes in an interactive process and presents variations in the types of needs, tasks, considerations, and decisions of users, as compared to information seeking in other contexts.

Abstract: Informetrics and information retrieval (IR) represent fundamental areas of study within information science. Historically, researchers have not fully capitalized on the potential research synergies that exist between these two areas. Data sources used in traditional informetrics studies have their analogues in IR, with similar types of empirical regularities found in IR system content and use. Methods for data collection and analysis used in informetrics can help to inform IR system development and evaluation. Areas of application have included automatic indexing, index term weighting and understanding user query and session patterns through the quantitative analysis of user transaction logs. Similarly, developments in database technology have made the study of informetric phenomena less cumbersome, and recent innovations used in IR research, such as language models and ranking algorithms, provide new tools that may be applied to research problems of interest to informetricians. Building on the author's previous work (Wolfram in Applied informetrics for information retrieval research, Libraries Unlimited, Westport, 2003), this paper reviews a sample of relevant literature published primarily since 2000 to highlight how each area of study may help to inform and benefit the other.

Abstract: The present paper reports on the initial study and the preliminary findings of how the concept of simulated work task situation is reported used in the research literature. The overall objective of the study is in a systematic manner to learn how and for what types of evaluations the concept is applied. In particular we are interested to learn whether the recommendations for how to apply simulated work task situations are followed.The preliminary findings indicate a need for clarifications of the recommendations of how to use simulated work task situations. Particularly with respect to 'realism' of the simulated work task situations, which is emphasised through the need for tailoring of the simulated work task situations towards the group of study participant to ensure the depicted situations are realistic and interesting from the participants' point of view. Likewise it seems that the recommendation to involve the study participants' own information needs (to function as baseline of search interaction) is generally neglected in the reported studies.

Abstract: Evaluation is central in research and development of information retrieval (IR). In addition to designing and implementing new retrieval mechanisms, one must also show through rigorous evaluation that they are effective. A major focus in IR is IR mechanisms’ capability of ranking relevant documents optimally for the users, given a query. Searching for information in practice involves searchers, however, and is highly interactive. When human searchers have been incorporated in evaluation studies, the results have often suggested that better ranking does not necessarily lead to better search task, or work task, performance. Therefore, it is not clear which system or interface features should be developed to improve the effectiveness of human task performance. In the present article, we focus on the evaluation of task-based information interaction (TBII). We give special emphasis to learning tasks to discuss TBII in more concrete terms. Information interaction is here understood as behavioral and cognitive activities related to task planning, searching information items, selecting between them, working with them, and synthesizing and reporting. These five generic activities contribute to task performance and outcome and can be supported by information systems. In an attempt toward task-based evaluation, we introduce program theory as the evaluation framework. Such evaluation can investigate whether a program consisting of TBII activities and tools works and how it works and, further, provides a causal description of program (in)effectiveness. Our goal in the present article is to structure TBII on the basis of the five generic activities and consider the evaluation of each activity using the program theory framework. Finally, we combine these activity-based program theories in an overall evaluation framework for TBII. Such an evaluation is complex due to the large number of factors affecting information interaction. Instead of presenting tested program theories, we illustrate how the evaluation of TBII should be accomplished using the program theory framework in the evaluation of systems and behaviors, and their interactions, comprehensively in context.

Abstract: The present study reports on the information seeking processes in a visual context, referred to throughout as visual information seeking. This study synthesizes research throughout different, yet complementary, areas, each capable of contributing findings and understanding to visual information seeking. Methods previously applied for examining the visual information seeking process are reviewed, including interactive experiments, surveys, and various qualitative approaches. The methods and resulting findings are presented and structured according to generalized phases of existing information seeking models, which include the needs, actions, and assessments of users. A review of visual information needs focuses on need and thus query formulation; user actions, as reviewed, centers on search and browse behaviors and the observed trends, concluded by a survey of users' assessments of visual information as part of the interactive process. This separate examination, specific to a visual context, is significant; visual information can influence outcomes in an interactive process and presents variations in the types of needs, tasks, considerations, and decisions of users, as compared to information seeking in other contexts.

Abstract: Informetrics and information retrieval (IR) represent fundamental areas of study within information science. Historically, researchers have not fully capitalized on the potential research synergies that exist between these two areas. Data sources used in traditional informetrics studies have their analogues in IR, with similar types of empirical regularities found in IR system content and use. Methods for data collection and analysis used in informetrics can help to inform IR system development and evaluation. Areas of application have included automatic indexing, index term weighting and understanding user query and session patterns through the quantitative analysis of user transaction logs. Similarly, developments in database technology have made the study of informetric phenomena less cumbersome, and recent innovations used in IR research, such as language models and ranking algorithms, provide new tools that may be applied to research problems of interest to informetricians. Building on the author's previous work (Wolfram in Applied informetrics for information retrieval research, Libraries Unlimited, Westport, 2003), this paper reviews a sample of relevant literature published primarily since 2000 to highlight how each area of study may help to inform and benefit the other.