Users' relevance criteria for evaluating answers in a social Q&A site
01 Apr 2009-Journal of the Association for Information Science and Technology (John Wiley & Sons, Ltd)-Vol. 60, Iss: 4, pp 716-727
TL;DR: The authors examined the criteria questioners use to select the best answers in a social QA Answers within the theoretical framework of relevance research and found that socio-emotional criteria are popular in discussion-oriented categories, content-oriented criteria in topicoriented categories and utility criteria in self-help categories.
Abstract: This study examines the criteria questioners use to select the best answers in a social QA Answers) within the theoretical framework of relevance research. A social QA Answers, the questioner selects the answer that best satisfies his or her question and leaves comments on it. Under the assumption that the comments reflect the reasons why questioners select particular answers as the best, this study analyzed 2,140 comments collected from Yahoo! Answers during December 2007. The content analysis identified 23 individual relevance criteria in six classes: Content, Cognitive, Utility, Information Sources, Extrinsic, and Socioemotional. A major finding is that the selection criteria used in a social Q&A site have considerable overlap with many relevance criteria uncovered in previous relevance studies, but that the scope of socio-emotional criteria has been expanded to include the social aspect of this environment. Another significant finding is that the relative importance of individual criteria varies according to topic categories. Socioemotional criteria are popular in discussion-oriented categories, content-oriented criteria in topic-oriented categories, and utility criteria in self-help categories. This study generalizes previous relevance studies to a new environment by going beyond an academic setting. © 2009 Wiley Periodicals, Inc.
The authors contributed equally to this work.
Citations
More filters
TL;DR: Findings show that altruism is the most influential motivation, while personal gain is the least, and enjoyment and efficacy are more influential than other social motivations, although there are some variations across different groups of answerers.
Abstract: In Web 2.0 environments, people commonly share their knowledge and personal experiences with others, but little is known about their background characteristics and motivations. Thus, the current study examines some of the characteristics and motivations common among answerers, who produce health-related answers to questions asked by anonymous others in a social Q&A site, Yahoo! Answers. An online survey questionnaire was distributed to top and recent answerers to investigate their demographics, areas of health expertise, and other characteristics related to answering behaviors online. Also, 10 motivation factors are proposed and tested in the survey: enjoyment, efficacy, learning, personal gain, altruism, community interest, social engagement, empathy, reputation, and reciprocity. Findings show that altruism is the most influential motivation, while personal gain is the least. Enjoyment and efficacy are more influential than other social motivations, such as reputation or reciprocity, although there are some variations across different groups of answerers. Motivational factors among top answerers or health experts are further analyzed. The findings of this study have practical implications for promoting health answerers to share knowledge and experiences in social contexts. Furthermore, the study design of the current study can be used to examine motivations of answerers in other topic areas as well as other social contexts. © 2012 Wiley Periodicals, Inc.
191 citations
TL;DR: The authors developed a research agenda for social Q&A, reviewing recent studies and identifying core issues, questions, and challenges, and identified core issues and challenges in social question and answer (Q&A).
163 citations
TL;DR: For instance, this article found that over 60% of respondents had shared misinformation on social media, and the top reasons were related to the information's perceived characteristics, as well as self-expression and socializing.
162 citations
Additional excerpts
...…on perception and use of social media (e.g., Kim, Sin, & Tsai, 2014; Lim, 2009; Morris, Teevan, & Panovich, 2010; Zhang, 2012), aswell as on the criteria and strategies used in evaluating the credibility of social media information (e.g., Kim & Oh, 2009; Kim& Sin, 2014a; Lim, 2013;Walsh, 2010)....
[...]
[...]
TL;DR: A review and analysis of the research literature in social Q&A (SQA), a term describing systems where people ask, answer, and rate content while interacting around it, and how trust and expertise are both operationalized by and emerge from SQA sites is presented.
Abstract: This article presents a review and analysis of the research literature in social Q&A (SQA), a term describing systems where people ask, answer, and rate content while interacting around it. The growth of SQA is contextualized within the broader trend of user-generated content from Usenet to Web 2.0, and alternative definitions of SQA are reviewed. SQA sites have been conceptualized in the literature as simultaneous examples of tools, collections, communities, and complex sociotechnical systems. Major threads of SQA research include user-generated and algorithmic question categorization, answer classification and quality assessment, studies of user satisfaction, reward structures, and motivation for participation, and how trust and expertise are both operationalized by and emerge from SQA sites. Directions for future research are discussed, including more refined conceptions of SQA site participants and their roles, unpacking the processes by which social capital is achieved, managed, and wielded in SQA sites, refining question categorization, conducting research within and across a wider range of SQA sites, the application of economic and game-theoretic models, and the problematization of SQA itself. © 2011 Wiley Periodicals, Inc.
134 citations
TL;DR: Because many students are actively using social media platforms for a variety of information-seeking purposes, it is suggested that IL programs embrace social media as potential information sources and offer effective strategies for using and evaluating these increasingly popular social media sources.
116 citations
References
More filters
TL;DR: A general statistical methodology for the analysis of multivariate categorical data arising from observer reliability studies is presented and tests for interobserver bias are presented in terms of first-order marginal homogeneity and measures of interob server agreement are developed as generalized kappa-type statistics.
Abstract: This paper presents a general statistical methodology for the analysis of multivariate categorical data arising from observer reliability studies. The procedure essentially involves the construction of functions of the observed proportions which are directed at the extent to which the observers agree among themselves and the construction of test statistics for hypotheses involving these functions. Tests for interobserver bias are presented in terms of first-order marginal homogeneity and measures of interobserver agreement are developed as generalized kappa-type statistics. These procedures are illustrated with a clinical diagnosis example from the epidemiological literature.
64,109 citations
TL;DR: In this article, the authors present a procedure for having two or more judges independently categorize a sample of units and determine the degree, significance, and significance of the units. But they do not discuss the extent to which these judgments are reproducible, i.e., reliable.
Abstract: CONSIDER Table 1. It represents in its formal characteristics a situation which arises in the clinical-social-personality areas of psychology, where it frequently occurs that the only useful level of measurement obtainable is nominal scaling (Stevens, 1951, pp. 2526), i.e. placement in a set of k unordered categories. Because the categorizing of the units is a consequence of some complex judgment process performed by a &dquo;two-legged meter&dquo; (Stevens, 1958), it becomes important to determine the extent to which these judgments are reproducible, i.e., reliable. The procedure which suggests itself is that of having two (or more) judges independently categorize a sample of units and determine the degree, significance, and
34,965 citations
21 Apr 2008
TL;DR: This paper analyzes YA's forum categories and cluster them according to content characteristics and patterns of interaction among the users, finding that lower entropy correlates with receiving higher answer ratings, but only for categories where factual expertise is primarily sought after.
Abstract: Yahoo Answers (YA) is a large and diverse question-answer forum, acting not only as a medium for sharing technical knowledge, but as a place where one can seek advice, gather opinions, and satisfy one's curiosity about a countless number of things. In this paper, we seek to understand YA's knowledge sharing and activity. We analyze the forum categories and cluster them according to content characteristics and patterns of interaction among the users. While interactions in some categories resemble expertise sharing forums, others incorporate discussion, everyday advice, and support. With such a diversity of categories in which one can participate, we find that some users focus narrowly on specific topics, while others participate across categories. This not only allows us to map related categories, but to characterize the entropy of the users' interests. We find that lower entropy correlates with receiving higher answer ratings, but only for categories where factual expertise is primarily sought after. We combine both user attributes and answer characteristics to predict, within a given category, whether a particular answer will be chosen as the best answer by the asker.
799 citations
TL;DR: This article presents the history of relevance through an exhaustive review of the literature under seven different aspects (methodological foundations, different kinds of relevance, beyond-topical criteria adopted by users, modes for expression of the relevance judgment, dynamic nature of relevance), and types of document representation.
Abstract: Relevance is a fundamental, though not completely understood, concept for documentation, information science, and information retrieval. This article presents the history of relevance through an exhaustive review of the literature. Such history being very complex (about 160 papers are discussed), it is not simple to describe it in a comprehensible way. Thus, first of all a framework for establishing a common ground is defined, and then the history itself is illustrated via the presentation in chronological order of the papers on relevance. The history is divided into three periods (“Before 1958,” “1959–1976,” and “1977–present”) and, inside each period, the papers on relevance are analyzed under seven different aspects (methodological foundations, different kinds of relevance, beyond-topical criteria adopted by users, modes for expression of the relevance judgment, dynamic nature of relevance, types of document representation, and agreement among different judges). © 1997 John Wiley & Sons, Inc.
564 citations
TL;DR: The concept of relevance as viewed and applied in the context of IR evaluation is introduced, by presenting an overview of the multidimensional and dynamic nature of the concept.
Abstract: This article introduces the concept of relevance as viewed and applied in the context of IR evaluation, by presenting an overview of the multidimensional and dynamic nature of the concept. The literature on relevance reveals how the relevance concept, especially in regard to the multidimensionality of relevance, is many faceted, and does not just refer to the various relevance criteria users may apply in the process of judging relevance of retrieved information objects. From our point of view, the multidimensionality of relevance explains why some will argue that no consensus has been reached on the relevance concept. Thus, the objective of this article is to present an overview of the many different views and ways by which the concept of relevance is used--leading to a consistent and compatible understanding of the concept. In addition, special attention is paid to the type of situational relevance. Many researchers perceive situational relevance as the most realistic type of user relevance, and therefore situational relevance is discussed with reference to its potential dynamic nature, and as a requirement for interactive information retrieval (IIR) evaluation.
520 citations