scispace - formally typeset
Search or ask a question
Topic

Crowdsourcing

About: Crowdsourcing is a research topic. Over the lifetime, 12889 publications have been published within this topic receiving 230638 citations.


Papers
More filters
Proceedings Article
03 Nov 2013
TL;DR: Using data from Galaxy Zoo, a prominent citizen science project, statistical models are designed and constructed that provide predictions about the forthcoming engagement of volunteers and characterize the accuracy of predictions with respect to different sets of features that describe user behavior.
Abstract: We present studies of the attention and time, or engagement, invested by crowd workers on tasks. Consideration of worker engagement is especially important in volunteer settings such as online citizen science. Using data from Galaxy Zoo, a prominent citizen science project, we design and construct statistical models that provide predictions about the forthcoming engagement of volunteers. We characterize the accuracy of predictions with respect to different sets of features that describe user behavior and study the sensitivity of predictions to variations in the amount of data and retraining. We design our model for guiding system actions in real-time settings, and discuss the prospect for harnessing predictive models of engagement to enhance user attention and effort on volunteer tasks.

65 citations

Proceedings ArticleDOI
18 Apr 2015
TL;DR: This work proposes a data-driven effort metric, ETA (error-time area), that can be used to determine a task's fair price and validate the ETA metric on ten common crowdsourcing tasks, finding that ETA closely tracks how workers would rank these tasks by effort.
Abstract: Crowdsourcing systems lack effective measures of the effort required to complete each task. Without knowing how much time workers need to execute a task well, requesters struggle to accurately structure and price their work. Objective measures of effort could better help workers identify tasks that are worth their time. We propose a data-driven effort metric, ETA (error-time area), that can be used to determine a task's fair price. It empirically models the relationship between time and error rate by manipulating the time that workers have to complete a task. ETA reports the area under the error-time curve as a continuous metric of worker effort. The curve's 10th percentile is also interpretable as the minimum time most workers require to complete the task without error, which can be used to price the task. We validate the ETA metric on ten common crowdsourcing tasks, including tagging, transcription, and search, and find that ETA closely tracks how workers would rank these tasks by effort. We also demonstrate how ETA allows requesters to rapidly iterate on task designs and measure whether the changes improve worker efficiency. Our findings can facilitate the process of designing, pricing, and allocating crowdsourcing tasks.

65 citations

Journal ArticleDOI
TL;DR: This work analytically derives an all-pay auction-based mechanism to incentivize agents to act in the principal's interest, i.e., maximizing profit, while allowing agents to reap strictly positive utility, and demonstrates that this approach remarkably outperforms its counterparts in terms of the principal’s profit, agent’“ utility,” and social welfare.
Abstract: Crowdsourcing can be modeled as a principal-agent problem in which the principal (crowdsourcer) desires to solicit a maximal contribution from a group of agents (participants) while agents are only motivated to act according to their own respective advantages. To reconcile this tension, we propose an all-pay auction approach to incentivize agents to act in the principal’s interest, i.e., maximizing profit, while allowing agents to reap strictly positive utility. Our rationale for advocating all-pay auctions is based on two merits that we identify, namely all-pay auctions (i) compress the common, two-stage “bid-contribute” crowdsourcing process into a single “bid-cum-contribute” stage, and (ii) eliminate the risk of task nonfulfillment. In our proposed approach, we enhance all-pay auctions with two additional features: an adaptive prize and a general crowdsourcing environment. The prize or reward adapts itself as per a function of the unknown winning agent’s contribution, and the environment or setting generally accommodates incomplete and asymmetric information, risk-averse (and risk-neutral) agents, and a stochastic (and deterministic) population. We analytically derive this all-pay auction-based mechanism and extensively evaluate it in comparison to classic and optimized mechanisms. The results demonstrate that our proposed approach remarkably outperforms its counterparts in terms of the principal’s profit, agent’s utility, and social welfare.

65 citations

Journal ArticleDOI
TL;DR: This study introduces hyperlocal spatial crowdsourcing, where all workers who are located within the spatiotemporal vicinity of a task are eligible to perform the task (e.g., reporting the precipitation level at their area and time), and studies the hardness of the task assignment problem in the offline setting and proposes online heuristics which exploit the spatial and temporal knowledge acquired over time.
Abstract: Spatial Crowdsourcing (SC) is a novel platform that engages individuals in the act of collecting various types of spatial data. This method of data collection can significantly reduce cost and turnover time and is particularly useful in urban environmental sensing, where traditional means fail to provide fine-grained field data. In this study, we introduce hyperlocal spatial crowdsourcing, where all workers who are located within the spatiotemporal vicinity of a task are eligible to perform the task (e.g., reporting the precipitation level at their area and time). In this setting, there is often a budget constraint, either for every time period or for the entire campaign, on the number of workers to activate to perform tasks. The challenge is thus to maximize the number of assigned tasks under the budget constraint despite the dynamic arrivals of workers and tasks. We introduce a taxonomy of several problem variants, such as budget-per-time-period vs. budget-per-campaign and binary-utility vs. distance-based-utility. We study the hardness of the task assignment problem in the offline setting and propose online heuristics which exploit the spatial and temporal knowledge acquired over time. Our experiments are conducted with spatial crowdsourcing workloads generated by the SCAWG tool, and extensive results show the effectiveness and efficiency of our proposed solutions.

65 citations

Journal ArticleDOI
TL;DR: In this paper, the authors investigate how the characteristics (i.e., direction, size, and strength) of customers' online peer-to-peer and peerto-firm interactions, moderated by customers' past efforts to post ideas, influence their likelihood of generating ideas in an idea crowdsourcing community.

65 citations


Network Information
Related Topics (5)
Social network
42.9K papers, 1.5M citations
87% related
User interface
85.4K papers, 1.7M citations
86% related
Deep learning
79.8K papers, 2.1M citations
85% related
Cluster analysis
146.5K papers, 2.9M citations
85% related
The Internet
213.2K papers, 3.8M citations
85% related
Performance
Metrics
No. of papers in the topic in previous years
YearPapers
2023637
20221,420
2021996
20201,250
20191,341
20181,396