Topic
Crowdsourcing
About: Crowdsourcing is a research topic. Over the lifetime, 12889 publications have been published within this topic receiving 230638 citations.
Papers published on a yearly basis
Papers
More filters
TL;DR: Crowd information quality crowd IQ is defined, empirically examines implications of class-based modeling approaches for crowd IQ, and a path for improving crowd IQ using instance-and-attribute based modeling is offered.
Abstract: User-generated content UGC is becoming a valuable organizational resource, as it is seen in many cases as a way to make more information available for analysis To make effective use of UGC, it is necessary to understand information quality IQ in this setting Traditional IQ research focuses on corporate data and views users as data consumers However, as users with varying levels of expertise contribute information in an open setting, current conceptualizations of IQ break down In particular, the practice of modeling information requirements in terms of fixed classes, such as an Entity-Relationship diagram or relational database tables, unnecessarily restricts the IQ of user-generated data sets This paper defines crowd information quality crowd IQ, empirically examines implications of class-based modeling approaches for crowd IQ, and offers a path for improving crowd IQ using instance-and-attribute based modeling To evaluate the impact of modeling decisions on IQ, we conducted three experiments Results demonstrate that information accuracy depends on the classes used to model domains, with participants providing more accurate information when classifying phenomena at a more general level In addition, we found greater overall accuracy when participants could provide free-form data compared to a condition in which they selected from constrained choices We further demonstrate that, relative to attribute-based data collection, information loss occurs when class-based models are used Our findings have significant implications for information quality, information modeling, and UGC research and practice
117 citations
TL;DR: It is found that dedicated promoter roles strongly contribute to a successful implementation of crowdsourcing, turning pilot projects into an organizational routine, and suggestions for organizational interventions to overcome barriers and sources of resistance.
Abstract: Crowdsourcing has been demonstrated to be an effective strategy to enhance the efficiency of a firm’s innovation process. In this paper, we focus on tournament-based crowdsourcing (also referred to as “broadcast search”), a method to solve technical problems in form of an open call for solutions to a large network of experts. Based on a longitudinal study of six companies piloting this application of crowdsourcing, we identify barriers and sources of resistance that hinder its implementation in firms. Our paper contributes to the state of research by analyzing crowdsourcing on the level of pilot projects, hence providing a workflow perspective that considers the creation of dedicated processes and operations of crowdsourcing. This project level analysis enables the identification of specific challenges managers face when implementing crowdsourcing within an established R&D organization. Following a design science approach, we derive suggestions for organizational interventions to overcome these barriers. We find that dedicated promoter roles strongly contribute to a successful implementation of crowdsourcing, turning pilot projects into an organizational routine.
117 citations
TL;DR: An overall picture of the current state of the art techniques in general-purpose crowdsourcing is offered and a real scenario on how the different techniques are used in implementing a location-based crowdsourcing platform, gMission is presented.
Abstract: Since Jeff Howe introduced the term Crowdsourcing in 2006, this human-powered problem-solving paradigm has gained a lot of attention and has been a hot research topic in the field of computer science. Even though a lot of work has been conducted on this topic, so far we do not have a comprehensive survey on most relevant work done in the crowdsourcing field. In this paper, we aim to offer an overall picture of the current state of the art techniques in general-purpose crowdsourcing. According to their focus, we divide this work into three parts, which are: incentive design, task assignment, and quality control. For each part, we start with different problems faced in that area followed by a brief description of existing work and a discussion of pros and cons. In addition, we also present a real scenario on how the different techniques are used in implementing a location-based crowdsourcing platform, gMission. Finally, we highlight the limitations of the current general-purpose crowdsourcing techniques and present some open problems in this area.
117 citations
05 May 2012
TL;DR: It is found that Umati was able to grade exams with 2% higher accuracy or at 33% lower cost than traditional single-expert grading, indicating that communitysourcing can successfully elicit high-quality expert work from specific communities.
Abstract: Online labor markets, such as Amazon's Mechanical Turk, have been used to crowdsource simple, short tasks like image labeling and transcription. However, expert knowledge is often lacking in such markets, making it impossible to complete certain classes of tasks. In this work we introduce an alternative mechanism for crowdsourcing tasks that require specialized knowledge or skill: communitysourcing --- the use of physical kiosks to elicit work from specific populations. We investigate the potential of communitysourcing by designing, implementing and evaluating Umati: the communitysourcing vending machine. Umati allows users to earn credits by performing tasks using a touchscreen attached to the machine. Physical rewards (in this case, snacks) are dispensed through traditional vending mechanics. We evaluated whether communitysourcing can accomplish expert work by using Umati to grade Computer Science exams. We placed Umati in a university Computer Science building, targeting students with grading tasks for snacks. Over one week, 328 unique users (302 of whom were students) completed 7771 tasks (7240 by students). 80% of users had never participated in a crowdsourcing market before. We found that Umati was able to grade exams with 2% higher accuracy (at the same price) or at 33% lower cost (at equivalent accuracy) than traditional single-expert grading. Mechanical Turk workers had no success grading the same exams. These results indicate that communitysourcing can successfully elicit high-quality expert work from specific communities.
117 citations
Posted Content•
TL;DR: A large user study on the ability of humans to detect today's Sybil accounts is conducted, using a large corpus of ground-truth Sybils from the Facebook and Renren networks and finds that while turkers vary significantly in their effectiveness, experts consistently produce near-optimal results.
Abstract: As popular tools for spreading spam and malware, Sybils (or fake accounts) pose a serious threat to online communities such as Online Social Networks (OSNs). Today, sophisticated attackers are creating realistic Sybils that effectively befriend legitimate users, rendering most automated Sybil detection techniques ineffective. In this paper, we explore the feasibility of a crowdsourced Sybil detection system for OSNs. We conduct a large user study on the ability of humans to detect today's Sybil accounts, using a large corpus of ground-truth Sybil accounts from the Facebook and Renren networks. We analyze detection accuracy by both "experts" and "turkers" under a variety of conditions, and find that while turkers vary significantly in their effectiveness, experts consistently produce near-optimal results. We use these results to drive the design of a multi-tier crowdsourcing Sybil detection system. Using our user study data, we show that this system is scalable, and can be highly effective either as a standalone system or as a complementary technique to current tools.
117 citations