Journal ArticleDOI
Inside the Turk Understanding Mechanical Turk as a Participant Pool
Gabriele Paolacci,Jesse Chandler +1 more
TLDR
The characteristics of Mechanical Turk as a participant pool for psychology and other social sciences, highlighting the traits of the MTurk samples, why people become Mechanical Turk workers and research participants, and how data quality on Mechanical Turk compares to that from other pools and depends on controllable and uncontrollable factors as mentioned in this paper.Abstract:
Mechanical Turk (MTurk), an online labor market created by Amazon, has recently become popular among social scientists as a source of survey and experimental data. The workers who populate this market have been assessed on dimensions that are universally relevant to understanding whether, why, and when they should be recruited as research participants. We discuss the characteristics of MTurk as a participant pool for psychology and other social sciences, highlighting the traits of the MTurk samples, why people become MTurk workers and research participants, and how data quality on MTurk compares to that from other pools and depends on controllable and uncontrollable factors.read more
Citations
More filters
Journal ArticleDOI
Beyond the Turk: Alternative platforms for crowdsourcing behavioral research
TL;DR: This article found that participants on both platforms were more naive and less dishonest compared to MTurk participants, and ProA and CrowdFlower participants produced data quality that was higher than CF's and comparable to M-Turk's.
Journal ArticleDOI
Prolific.ac—A subject pool for online experiments
TL;DR: This article presents www.prolific.ac and lays out its suitability for recruiting subjects for social and economic science experiments, and traces the platform’s historical development, present its features, and contrast them with requirements for different types of social andEconomic experiments.
Journal ArticleDOI
Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants
David J. Hauser,Norbert Schwarz +1 more
TL;DR: In three online studies, participants from MTurk and collegiate populations participated in a task that included a measure of attentiveness to instructions (an instructional manipulation check: IMC), and MTurkers were more attentive to the instructions than were college students, even on novel IMCs.
Journal ArticleDOI
Conducting Clinical Research Using Crowdsourced Convenience Samples
TL;DR: This article addresses methodological issues with using MTurk--many of which are common to other nonprobability samples but unfamiliar to clinical science researchers--and suggests concrete steps to avoid these issues or minimize their impact.
Journal ArticleDOI
“Who are these people?” Evaluating the demographic characteristics and political preferences of MTurk survey respondents:
Connor Huff,Dustin Tingley +1 more
TL;DR: As Amazon's Mechanical Turk (MTurk) has surged in popularity throughout political science, scholars have increasingly challenged the external validity of inferences made drawing upon MTurk samples as mentioned in this paper.
References
More filters
Journal ArticleDOI
Amazon's Mechanical Turk A New Source of Inexpensive, Yet High-Quality, Data?
TL;DR: Findings indicate that MTurk can be used to obtain high-quality data inexpensively and rapidly and the data obtained are at least as reliable as those obtained via traditional methods.
Running experiments on Amazon Mechanical Turk
TL;DR: The authors presented new demographic data about the Mechanical Turk subject population, reviewed the strengths of Mechanical Turk relative to other online and offline methods of recruiting subjects, and compared the magnitude of effects obtained using Mechanical Turk and traditional subject pools.
Journal ArticleDOI
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
TL;DR: It is shown that respondents recruited in this manner are often more representative of the U.S. population than in-person convenience samples but less representative than subjects in Internet-based panels or national probability samples.
Posted Content
Running experiments on Amazon Mechanical Turk
TL;DR: The authors presented new demographic data about the Mechanical Turk subject population, reviewed the strengths of Mechanical Turk relative to other online and offline methods of recruiting subjects, and compared the magnitude of effects obtained using Mechanical Turk and traditional subject pools.
Posted Content
Conducting Behavioral Research on Amazon's Mechanical Turk
TL;DR: In this article, the authors demonstrate how to use Mechanical Turk for conducting behavioral research and lower the barrier to entry for researchers who could benefit from this platform, and illustrate the mechanics of putting a task on Mechanical Turk including recruiting subjects, executing the task, and reviewing the work submitted.