scispace - formally typeset
Search or ask a question
Topic

Crowdsourcing

About: Crowdsourcing is a research topic. Over the lifetime, 12889 publications have been published within this topic receiving 230638 citations.


Papers
More filters
Journal ArticleDOI
10 Jun 2016-Science
TL;DR: In May, 23,000 people voluntarily took part in thousands of social science experiments without ever visiting a lab by logging on to Amazon Mechanical Turk, an online crowdsourcing service run by the Seattle, Washington–based company better known for its massive internet-based retail business.
Abstract: In May, 23,000 people voluntarily took part in thousands of social science experiments without ever visiting a lab. All they did was log on to Amazon Mechanical Turk (MTurk), an online crowdsourcing service run by the Seattle, Washington–based company better known for its massive internet-based retail business. Those research subjects completed 230,000 tasks on their computers in 3.3 million minutes—more than 6 years of effort in total. The prodigious output demonstrates the popularity of an online platform that scientists had only begun to exploit 5 years ago. But the growing use of MTurk has raised concerns, as researchers discussed at the Association for Psychological Science meeting in Chicago, Illinois, last month. Some worry that they are becoming too dependent on a commercial platform. Others question whether the research volunteers are paid fairly and treated ethically. And looming over it all are questions about who these anonymous volunteers actually are, and concerns that they are less numerous and diverse than researchers hope.

71 citations

Journal ArticleDOI
TL;DR: In this paper, the authors proposed a max-margin majority voting (M$^3$3V) method to improve the discriminative ability of majority voting and further presented a Bayesian generalization to incorporate the flexibility of generative methods on modeling noisy observations with worker confusion matrices for different application settings.
Abstract: Learning-from-crowds aims to design proper aggregation strategies to infer the unknown true labels from the noisy labels provided by ordinary web workers. This paper presents max-margin majority voting (M$^3$3V) to improve the discriminative ability of majority voting and further presents a Bayesian generalization to incorporate the flexibility of generative methods on modeling noisy observations with worker confusion matrices for different application settings. We first introduce the crowdsourcing margin of majority voting, then we formulate the joint learning as a regularized Bayesian inference (RegBayes) problem, where the posterior regularization is derived by maximizing the margin between the aggregated score of a potential true label and that of any alternative label. Our Bayesian model naturally covers the Dawid-Skene estimator and M$^3$3V as its two special cases. Due to the flexibility of our model, we extend it to handle crowdsourced labels with an ordinal structure with the main ideas about the crowdsourcing margin unchanged. Moreover, we consider an online learning-from-crowds setting where labels coming in a stream. Empirical results demonstrate that our methods are competitive, often achieving better results than state-of-the-art estimators.

71 citations

Journal ArticleDOI
TL;DR: Happy New Year to Everyone!
Abstract: Happy New Year to Everyone! This is the year of the sheep according to the Chinese lunar calendar, which implies a year of happiness in every day, almost! I wish our journal to be a better one in the year of the sheep with your help. So please check @IEEE-TITS (http://www.weibo.com/u/3967923931) in Weibo (an extended Chinese version of Twitter), IEEE ITS Facebook (https://www.facebook.com/IEEEITS), and our Twitter account @IEEEITS (https://twitter.com/IEEEITS) for upcoming news in the IEEE Intelligent Transportation Systems Society, the IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, and the IEEE INTELLIGENT TRANSPORTATION SYSTEMS MAGAZINE. The three sites are still under development, and your participation and any suggestions for their operation are extremely welcome.

71 citations

Proceedings ArticleDOI
13 Sep 2014
TL;DR: Camera monitoring of several parking lots as 105 PocketParker users generated 10;827 events over 45 days shows that Pocket parker was able to correctly predict lot availability 94% of the time.
Abstract: Searching for parking spots generates frustration and pollution. To address these parking problems, we present PocketParker, a crowdsourcing system using smartphones to predict parking lot availability. PocketParker is an example of a subset of crowdsourcing we call pocketsourcing. Pocketsourcing applications require no explicit user input or additional infrastructure, running effectively without the phone leaving the user's pocket. PocketParker detects arrivals and departures by leveraging existing activity recognition algorithms. Detected events are used to maintain per-lot availability models and respond to queries. By estimating the number of drivers not using PocketParker, a small fraction of drivers can generate accurate predictions. Our evaluation shows that PocketParker quickly and correctly detects parking events and is robust to the presence of hidden drivers. Camera monitoring of several parking lots as 105 PocketParker users generated 10;827 events over 45 days shows that PocketParker was able to correctly predict lot availability 94% of the time.

71 citations

Journal ArticleDOI
TL;DR: This paper addresses this evolution of AI&EdAIED by identifying six trends, which depict the evolution of learning technologies as a whole.
Abstract: How does AIE today compare to 25 years ago? This paper addresses this evolution by identifying six trends. The trends are ongoing and will influence learning technologies going forward. First, the physicality of interactions and the physical space of the learner became genuine components of digital education. The frontier between the digital and the physical has faded out. Similarly, the opposition between individual and social views on cognition has been subsumed by integrated learning scenarios, which means that AIED pays more attention today to social interactions than it did at its outset. Another trend is the processing of learners' behavioural particles, which do not carry very many semantics when considered individually, but are predictive of knowledge states when large data sets are processed with machine learning methods. The development of probabilistic models and the integration of crowdsourcing methods has produced another trend: the design of learning environments has become less deterministic than before. The notion of learning environment evolved from a rather closed box to an open ecosystem in which multiple components are distributed over multiple platforms and where multiple stakeholders interact. Among these stakeholders, it is important to notice that teachers play a more important role than before: they interact not only at the design phase (authoring) but also in the runtime phase (orchestration). These trends are not specific to AIED; they depict the evolution of learning technologies as a whole.

71 citations


Network Information
Related Topics (5)
Social network
42.9K papers, 1.5M citations
87% related
User interface
85.4K papers, 1.7M citations
86% related
Deep learning
79.8K papers, 2.1M citations
85% related
Cluster analysis
146.5K papers, 2.9M citations
85% related
The Internet
213.2K papers, 3.8M citations
85% related
Performance
Metrics
No. of papers in the topic in previous years
YearPapers
2023637
20221,420
2021996
20201,250
20191,341
20181,396