Speaking on the record: Combining interviews with search log analysis in user research
01 Jul 2022-Vol. 44, Iss: 3, pp 101179-101179
TL;DR: In this paper , a novel sequential mixed methods approach was developed to more holistically study user search behavior within a library discovery system, where customized semi-structured interview protocols, using the critical incident technique, were based on each participant's search logs.
Abstract: The use of mixed methods in library and information science research is more effective in generating insightful results when the methods are fully integrated with each other. A novel sequential mixed methods approach was developed to more holistically study user search behavior within a library discovery system. Customized semi-structured interview protocols, using the critical incident technique , were based on each participant's search logs. Findings from the interviews informed statistical log analysis, which identified features of search sessions that made accessing resources more likely. Individual interviews provide more precise data when protocols are created using the participants' logs. Similarly, statistical log analyses are enhanced with users' descriptions of their behaviors in library discovery systems. While prior studies have employed both interviews and log analysis, using the methods to inform one another reduces the limitations and enhances the benefits of each. • A novel sequential mixed methods approach provides a more holistic view of user behavior in library discovery systems. • Individual interviews provide more precise data when protocols are created using participants' logs. • Statistical log analyses are enhanced with users' descriptions of their behaviors in library discovery systems.
Citations
More filters
TL;DR: Li et al. as mentioned in this paper conducted a pilot study using questionnaires, user patterns, user interviews, and a user journey map to determine the user needs for electric minivan user service guarantees.
Abstract: With the introduction of “carbon neutrality”, promoting carbon peaking and carbon neutrality has become contingent upon the vigorous development of pure electric vehicles. The development of electric minivans is receiving substantial backing from the government as China’s logistics vehicle market undergoes a transformation. The drivers of electric minivans in China’s Shanxi Province are the objects of this research, whose purpose was to establish a service model based on the four elements of service design and grounded theory to clarify the key factors of user guarantee for electric minivans. We conducted a pilot study using questionnaires, user patterns, user interviews, and a user journey map to determine the user needs for electric minivan user service guarantees. NVivo was used to analyze and code the user interview data qualitatively. The research findings can serve as a resource for entrepreneurs, service providers, managers, and designers of electric minivans.
References
More filters
8,493 citations
TL;DR: A review and foundation for conducting Web search transaction log analysis consisting of three stages, which are collection, preparation, and analysis is presented and suggestions are provided on ways to leverage the strengths of, while addressing the limitations of, transaction logs for Web-searching research.
Abstract: The use of data stored in transaction logs of Web search engines, Intranets, and Web sites can provide valuable insight into understanding the information-searching process of online searchers. This understanding can enlighten information system design, interface development, and devising the information architecture for content collections. This article presents a review and foundation for conducting Web search transaction log analysis. A methodology is outlined consisting of three stages, which are collection, preparation, and analysis. The three stages of the methodology are presented in detail with discussions of goals, metrics, and processes at each stage. Critical terms in transaction log analysis for Web searching are defined. The strengths and limitations of transaction log analysis as a research method are presented. An application to log client-side interactions that supplements transaction logs is reported on, and the application is made available for use by the research community. Suggestions are provided on ways to leverage the strengths of, while addressing the limitations of, transaction log analysis for Web-searching research. Finally, a complete flat text transaction log from a commercial search engine is available as supplementary material with this manuscript.
285 citations
03 Jul 2018
TL;DR: Mixed methods combine the counterfactual analysis from a RCT with factual analysis using quantitative and qualitative data to analyze the causal chain this paper, where the factual analysis will address such issues as the quality of implementation, targeting, barriers to participation or adoption by intended beneficiaries.
Abstract: Mixed methods combine the counterfactual analysis from a RCT with factual analysis using quantitative and qualitative data to analyze the causal chain…
The factual analysis will address such issues as the quality of implementation, targeting, barriers to participation or adoption by intended beneficiaries.
227 citations
TL;DR: In this paper, the authors conducted a transaction log analysis on user activity in the Computer Science Technical Reports Collection of the New Zealand Digital Library, and report insights gained and identify resulting search interface design issues.
Abstract: As experimental digital library testbeds gain wider acceptance and develop significant user bases, it becomes important to investigate the ways in which users interact with the systems in practice. Transaction logs are one source of usage information, and the information on user behavior can be culled from them both automatically (through calculation of summary statistics) and manually (by examining query strings for semantic clues on search motivations and searching strategy). We have conducted a transaction log analysis on user activity in the Computer Science Technical Reports Collection of the New Zealand Digital Library, and report insights gained and identify resulting search interface design issues. Specifically, we present the user demographics available with our library, discuss the use of operators and search options in queries, and examine patterns in query construction and refinement. We also describe common mistakes in searching, and examine the distribution of query terms appearing in the logs.
181 citations
TL;DR: An overview of 10 years of research on log analysis is presented, which presents an overview of two main themes: Web search engine log analysis and Digital Library System log analysis.
Abstract: In the last decade, the importance of analyzing information management systems logs has grown, because log data constitute a relevant aspect in evaluating the quality of such systems. A review of 10 years of research on log analysis is presented in this paper. About 50 papers and posters from five major conferences and about 30 related journal papers have been selected to trace the history of the state-of-the-art in this field. The paper presents an overview of two main themes: Web search engine log analysis and Digital Library System log analysis. The problem of the analysis of different sources of log data and the distribution of data are investigated.
109 citations