User's relevance criteria in image retrieval in American history

TLDR: The study found that the user's perception of topicality was still the most important factor across the information-seeking stages, however, the users decided on retrieved items according to a variety of criteria other than topicality.

Abstract: A large number of digital images are available and accessible due to recent advances in technology. Since image retrieval systems are designed to meet user information needs, it seems apparent that image retrieval system design and implementation should take into account user-based aspects such as information use patterns and relevance judgments. However, little is known about what criteria users employ when making relevance judgments and which textual representations of the image help them make relevance judgments in their situational context.Thus, this study attempted to investigate the criteria which image users apply when making judgments about the relevance of an image. This research was built on prior work by Barry, Schamber and others which examined relevance criteria for textual and non-textual documents, exploring the extent to which these criteria apply to visual documents and the extent to which new and different criteria apply. Data were collected from unstructured interviews and questionnaires. Quantitative statistical methods were employed to analyze the importance of relevance criteria to see how much each criterion affected the user's judgments. The study involved 38 faculty and graduate students of American history in 1999 in a local setting, using the Library of Congress American memory photo archives.The study found that the user's perception of topicality was still the most important factor across the information-seeking stages. However, the users decided on retrieved items according to a variety of criteria other than topicality. Image quality and clarity was important. Users also searched for relevant images on the basis of title, date, subject descriptors, and notes provided. The conclusions of this study will be useful in image database design to assist users in conducting image searches. This study can be helpful to future relevance studies in information system design and evaluation.