Open AccessPosted Content
Running experiments on Amazon Mechanical Turk
Reads0
Chats0
TLDR
The authors presented new demographic data about the Mechanical Turk subject population, reviewed the strengths of Mechanical Turk relative to other online and offline methods of recruiting subjects, and compared the magnitude of effects obtained using Mechanical Turk and traditional subject pools.Abstract:
Although Mechanical Turk has recently become popular among social scientists as a source of experimental data, doubts may linger about the quality of data provided by subjects recruited from online labor markets. We address these potential concerns by presenting new demographic data about the Mechanical Turk subject population, reviewing the strengths of Mechanical Turk relative to other online and offline methods of recruiting subjects, and comparing the magnitude of effects obtained using Mechanical Turk and traditional subject pools. We further discuss some additional benefits such as the possibility of longitudinal, cross cultural and prescreening designs, and offer some advice on how to best manage a common subject pool.read more
Citations
More filters
Journal ArticleDOI
Collaborative learning in networks
Winter Mason,Duncan J. Watts +1 more
TL;DR: This paper reports on a series of 256 Web-based experiments in which groups of 16 individuals collectively solved a complex problem and shared information through different communication networks, finding that collective exploration improved average success over independent exploration because good solutions could diffuse through the network.
Journal ArticleDOI
Introducing the Open Affective Standardized Image Set (OASIS).
TL;DR: The Open Affective Standardized Image Set (OASIS), an open-access online stimulus set containing 900 color images depicting a broad spectrum of themes, along with normative ratings on two affective dimensions—valence and arousal, is introduced.
Journal ArticleDOI
A functional and perceptual signature of the second visual area in primates
Jeremy Freeman,Jeremy Freeman,Corey M. Ziemba,David J. Heeger,David J. Heeger,Eero P. Simoncelli,J. Anthony Movshon,J. Anthony Movshon +7 more
TL;DR: A synthetic stimuli replicating the higher-order statistical dependencies found in natural texture images was constructed and used to stimulate macaque V1 and V2 neurons, revealing a particular functional role for V2 in the representation of natural image structure.
Journal ArticleDOI
The evolution of antisocial punishment in optional public goods games
David G. Rand,Martin A. Nowak +1 more
TL;DR: The standard theory of optional public goods games is extended to include the full set of punishment strategies and finds that punishment no longer increases cooperation, and that selection favours substantial levels of antisocial punishment for a wide range of parameters.
Journal ArticleDOI
Recruiting large online samples in the United States and India: Facebook, Mechanical Turk, and Qualtrics
TL;DR: This article examined online recruitment via Facebook, Mechanical Turk (MTurk), and Qualtrics panels in India and the United States and found that online convenience samples often provide valid inferences into how partisanship moderates treatment effects, yet they are typically unrepresentative on such political variables, which has implications for the external validity of sample average treatment effects.
References
More filters
Journal ArticleDOI
The Framing of Decisions and the Psychology of Choice
Amos Tversky,Daniel Kahneman +1 more
TL;DR: The psychological principles that govern the perception of decision problems and the evaluation of probabilities and outcomes produce predictable shifts of preference when the same problem is framed in different ways.
Journal ArticleDOI
Amazon's Mechanical Turk A New Source of Inexpensive, Yet High-Quality, Data?
TL;DR: Findings indicate that MTurk can be used to obtain high-quality data inexpensively and rapidly and the data obtained are at least as reliable as those obtained via traditional methods.
Journal ArticleDOI
On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications.
TL;DR: In this paper, the authors focus on some of the qualities peculiar to psychological experiments and point out that the demand characteristics perceived in any particular experiment will vary with the sophistication, intelligence, and previous experience of each experimental subject.
Journal ArticleDOI
Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment.
Amos Tversky,Daniel Kahneman +1 more
TL;DR: The conjunction rule as mentioned in this paper states that the probability of a conjunction cannot exceed the probabilities of its constituents, P (A) and P (B), because the extension (or the possibility set) of the conjunction is included in the extension of their constituents.
Journal ArticleDOI
Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires.
TL;DR: Internet data collection methods, with a focus on self-report questionnaires from self-selected samples, are evaluated and compared with traditional paper-and-pencil methods and it is concluded that Internet methods can contribute to many areas of psychology.