The history and development of transaction log analysis
TL;DR: An overview of the history and development of transaction log analysis (TLA) in library and information science research is presented to provide an accurate sense of the development of all applications.
Abstract: The purpose of this article is to present an overview of the history and development of transaction log analysis (TLA) in library and information science research. Organizing a literature review of the first twenty‐five years of TLA poses some challenges and requires some decisions. The primary organizing principle could be a strict chronology of the published research, the research questions addressed, the automated information retrieval (IR) systems that generated the data, the results gained, or even the researchers themselves. The group of active transaction log analyzers remains fairly small in number, and researchers who use transaction logs tend to use this method more than once, so tracing the development and refinement of individuals' uses of the methodology could provide insight into the progress of the method as a whole. For example, if we examine how researchers like W. David Penniman, John Tolle, Christine Borgman, Ray Larson, and Micheline Hancock‐Beaulieu have modified their own understandings and applications of the method over time, we may get an accurate sense of the development of all applications.
Citations
More filters
TL;DR: A failure analysis was conducted, identifying trends among user mistakes, and a summary of findings and a discussion of the implications of these findings were concluded.
Abstract: We analyzed transaction logs containing 51,473 queries posed by 18,113 users of Excite, a major Internet search service. We provide data on: (i) sessions — changes in queries during a session, number of pages viewed, and use of relevance feedback; (ii) queries — the number of search terms, and the use of logic and modifiers; and (iii) terms — their rank/frequency distribution and the most highly used search terms. We then shift the focus of analysis from the query to the user to gain insight to the characteristics of the Web user. With these characteristics as a basis, we then conducted a failure analysis, identifying trends among user mistakes. We conclude with a summary of findings and a discussion of the implications of these findings. # 2000 Elsevier Science Ltd. All rights reserved.
1,414 citations
01 Jan 2006
TL;DR: In this paper, the authors report results from research that examines characteristics and changes in Web searching from nine studies of five Web search engines based in the US and Europe and find that users are viewing fewer result pages, searchers on US-based web search engines use more query operators, and there are statistically significant differences in the use of Boolean operators and result pages viewed, and one cannot necessary apply results from studies of one particular Web search engine to another web search engine.
Abstract: The Web and especially major Web search engines are essential tools in the quest to locate online information for many people. This paper reports results from research that examines characteristics and changes in Web searching from nine studies of five Web search engines based in the US and Europe. We compare interactions occurring between users and Web search engines from the perspectives of session length, query length, query complexity, and content viewed among the Web search engines. The results of our research shows (1) users are viewing fewer result pages, (2) searchers on US-based Web search engines use more query operators than searchers on European-based search engines, (3) there are statistically significant differences in the use of Boolean operators and result pages viewed, and (4) one cannot necessary apply results from studies of one particular Web search engine to another Web search engine. The wide spread use of Web search engines, employment of simple queries, and decreased viewing of result pages may have resulted from algorithmic enhancements by Web search engine companies. We discuss the implications of the findings for the development of Web search engines and design of online content.
810 citations
Book•
28 Apr 2009TL;DR: In this paper, the authors provide an overview and instruction regarding the evaluation of interactive information retrieval systems with users and present core instruments and data collection techniques and measures, as well as a discussion of outstanding challenges and future research directions.
Abstract: This paper provides overview and instruction regarding the evaluation of interactive information retrieval systems with users The primary goal of this article is to catalog and compile material related to this topic into a single source This article (1) provides historical background on the development of user-centered approaches to the evaluation of interactive information retrieval systems; (2) describes the major components of interactive information retrieval system evaluation; (3) describes different experimental designs and sampling strategies; (4) presents core instruments and data collection techniques and measures; (5) explains basic data analysis techniques; and (4) reviews and discusses previous studies This article also discusses validity and reliability issues with respect to both measures and methods, presents background information on research ethics and discusses some ethical issues which are specific to studies of interactive information retrieval (IIR) Finally, this article concludes with a discussion of outstanding challenges and future research directions
565 citations
TL;DR: The results from published studies of Web searching are reviewed and the searching characteristics of Web users are compared and contrasted with users of traditional information retrieval and online public access systems to discover if there is a need for more studies that focus predominantly or exclusively on Web searching.
Abstract: Research on Web searching is at an incipient stage. This aspect provides a unique opportunity to review the current state of research in the field, identify common trends, develop a methodological framework, and define terminology for future Web searching studies. In this article, the results from published studies of Web searching are reviewed to present the current state of research. The analysis of the limited Web searching studies available indicates that research methods and terminology are already diverging. A framework is proposed for future studies that will facilitate comparison of results. The advantages of such a framework are presented, and the implications for the design of Web information retrieval systems studies are discussed. Additionally, the searching characteristics of Web users are compared and contrasted with users of traditional information retrieval and online public access systems to discover if there is a need for more studies that focus predominantly or exclusively on Web searching. The comparison indicates that Web searching differs from searching in other environments.
466 citations
Cites background from "The history and development of tran..."
...However, there have been surprising few detailed studies on Web or Internet searching ( Peters, 1993 )....
[...]
...In this respect, TLA is limited ( Peters, 1993 ); however, TLA can provide some necessary data....
[...]
...However, studies show that users enter null queries during the normal searching process ( Peters, 1993 )....
[...]
...For a historical review of TLA, see Peters (1993) ....
[...]
TL;DR: A review and foundation for conducting Web search transaction log analysis consisting of three stages, which are collection, preparation, and analysis is presented and suggestions are provided on ways to leverage the strengths of, while addressing the limitations of, transaction logs for Web-searching research.
285 citations
Cites background from "The history and development of tran..."
...Peters (1993) views 2 of 42 transaction logs as electronically recorded interactions between on-line information retrieval systems and the persons who search for the information found in those systems....
[...]
...Peters (1993) views...
[...]
...Peters (1993) provides a review of TLA in library and experimental IR systems....
[...]
...In other views, Peters (1993) describes TLA as the study of electronically recorded interactions between on-line information retrieval systems and the persons who search for information found in those systems....
[...]
References
More filters
TL;DR: Suggestions include the development of a comprehensive set of synthetic programs on an industry-wide basis for selection evaluation purposes and the use of simulation as a suitable technique for performance projection.
Abstract: Three major purposes for evaluating the hardware and software performance of computer systems--selection evaluation, performance projection, and performance monitoring--are described. Eight techniques that have been used or suggested for evaluating performance are discussed. Each of these techniques is rated on its suitability for the three purposes of evaluation. Recommendations are made on the most appropriate technique for each evaluation purpose. These suggestions include the development of a comprehensive set of synthetic programs on an industry-wide basis for selection evaluation purposes. Simulation is recommended as the most suitable technique for performance projection. Finally, a number of hardware and software monitors are available for performance monitoring.
163 citations
TL;DR: Analysis of transaction data suggested a number of factors contributing to the decline in subject search frequency are user difficulties in formulating subject queries with Library of Congress Subject Headings, leading to search failure, and the problem of “information overload” as database size increases.
Abstract: Search index usage in a large university online catalog system over a six-year period (representing about 15.3 million searches) was investigated using transaction monitor data. Mathematical models of trends and patterns In the data were developed and tested using regression techniques. The results of the analyses show a consistent decline in the frequency of subject index use by online catalog users, with a corresponding increase in the frequency of title keyword searching. Significant annual patterns in index usage were also identified. Analysis of the transaction data, and related previous studies of online catalog users, suggest a number of factors contributing to the decline in subject search frequency. Chief among these factors are user difficulties in formulating subject queries with Library of Congress Subject Headings, leading to search failure, and the problem of “information overload” as database size increases. This article presents the models and results of the transaction log analysis, discusses the underlying problems with subject searching contributing to the observed decline, and reviews some proposed improvements to online catalog systems to aid in overcoming these problems.
95 citations
TL;DR: To improve retrieval effectiveness it is suggested that online catalogues should cater for both matching and contextual approaches to searching, and a holistic approach is adopted encompassing both catalogue use and browsing at the shelves for catalogue users and non‐users.
Abstract: The second half of a ‘before and after’ study to evaluate the impact of an online catalogue on subject searching behaviour is reported. A holistic approach is adopted encompassing both catalogue use and browsing at the shelves for catalogue users and non‐users. Verbal and non‐verbal data were elicited from searchers using a combined methodology including talk‐aloud technique, observation and a screen logging facility. An extensive qualitative analysis was carried out correlating expressed topics, search formulation strategies and documents retrieved at the shelves. The online catalogue environment does not appear to have increased the extent of subject searching nor the use of the bibliographic tool. The manual precis index supported a contextual approach for broad and more interactive search formulations whereas the opac encouraged a matching approach and narrow formulations with fewer but user generated formulations. The success rate of the online catalogue was slightly better than that of the manual tools but fewer items were retrieved at the shelves. Non‐users of the bibliographic tools seemed to be just as successful. To improve retrieval effectiveness it is suggested that online catalogues should cater for both matching and contextual approaches to searching. Recent research indicates that a more interactive process could be promoted by providing query expansion through a combination of searching aids for matching, for search formulation assistance and for structured contextual retrieval.
91 citations
Journal Article•
TL;DR: A vehicle-type thermostat having a housing provided with a valve seat that is controlled by a movable valve member which is interconnected to a temperature responsive device which is carried by the housing.
89 citations
Book•
01 Jun 1984
85 citations