Open AccessPosted Content
Running experiments on Amazon Mechanical Turk
Reads0
Chats0
TLDR
The authors presented new demographic data about the Mechanical Turk subject population, reviewed the strengths of Mechanical Turk relative to other online and offline methods of recruiting subjects, and compared the magnitude of effects obtained using Mechanical Turk and traditional subject pools.Abstract:
Although Mechanical Turk has recently become popular among social scientists as a source of experimental data, doubts may linger about the quality of data provided by subjects recruited from online labor markets. We address these potential concerns by presenting new demographic data about the Mechanical Turk subject population, reviewing the strengths of Mechanical Turk relative to other online and offline methods of recruiting subjects, and comparing the magnitude of effects obtained using Mechanical Turk and traditional subject pools. We further discuss some additional benefits such as the possibility of longitudinal, cross cultural and prescreening designs, and offer some advice on how to best manage a common subject pool.read more
Citations
More filters
Journal ArticleDOI
Intellectual humility and prosocial values: Direct and mediated effects
TL;DR: This paper found that intellectual humility is associated with higher levels of empathy, gratitude, altruism, benevolence, and universalism, and lower levels of power seeking, leaving open the possibility that intellectual humbleness may be a precursor to links previously established between empathy and gratitude and prosocial outcomes.
Journal ArticleDOI
The expression and interpretation of uncertain forensic science evidence: verbal equivalence, evidence strength, and the weak evidence effect.
TL;DR: Two experiments designed to investigate how much decision makers change their beliefs when presented with evidence in the form of verbal or numeric LRs found a reasonable degree of correspondence in observed belief change, but belief change was considerably smaller than Bayesian calculations would predict.
Journal ArticleDOI
Time, money, and happiness: How does putting a price on time affect our ability to smell the roses?
Sanford E. DeVoe,Julian House +1 more
TL;DR: In this article, the authors investigate how the impatience that results from placing a price on time impairs individuals' ability to derive happiness from pleasurable experiences and find that the deleterious effect on happiness caused by impatience is attenuated by offering participants monetary compensation in exchange for time spent listening to music, suggesting that a sensation of unprofitably wasted time underlay the induced impatience.
Proceedings ArticleDOI
Comparing the Reliability of Amazon Mechanical Turk and Survey Monkey to Traditional Market Research Surveys
TL;DR: This study presents a study investigating the reliability of fast survey platforms such as Amazon Mechanical Turk and Survey Monkey as compared to larger market research studies for technology behavior research and shows that results can be obtained in hours for much smaller costs with accuracy within 10% of traditional larger surveys.
Journal ArticleDOI
Gender Bias Produces Gender Gaps in STEM Engagement
TL;DR: This article explored whether the existence of gender bias causes gender gaps in STEM engagement and found that women projected less sense of belonging, positivity toward, and aspirations to participate in STEM than did men when exposed to the reality of STEM gender bias.
References
More filters
Journal ArticleDOI
The Framing of Decisions and the Psychology of Choice
Amos Tversky,Daniel Kahneman +1 more
TL;DR: The psychological principles that govern the perception of decision problems and the evaluation of probabilities and outcomes produce predictable shifts of preference when the same problem is framed in different ways.
Journal ArticleDOI
Amazon's Mechanical Turk A New Source of Inexpensive, Yet High-Quality, Data?
TL;DR: Findings indicate that MTurk can be used to obtain high-quality data inexpensively and rapidly and the data obtained are at least as reliable as those obtained via traditional methods.
Journal ArticleDOI
On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications.
TL;DR: In this paper, the authors focus on some of the qualities peculiar to psychological experiments and point out that the demand characteristics perceived in any particular experiment will vary with the sophistication, intelligence, and previous experience of each experimental subject.
Journal ArticleDOI
Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment.
Amos Tversky,Daniel Kahneman +1 more
TL;DR: The conjunction rule as mentioned in this paper states that the probability of a conjunction cannot exceed the probabilities of its constituents, P (A) and P (B), because the extension (or the possibility set) of the conjunction is included in the extension of their constituents.
Journal ArticleDOI
Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires.
TL;DR: Internet data collection methods, with a focus on self-report questionnaires from self-selected samples, are evaluated and compared with traditional paper-and-pencil methods and it is concluded that Internet methods can contribute to many areas of psychology.