Mechanical Turk (MTurk), an online labor market created by Amazon, has recently become popular among social scientists as a source of survey and experimental data. The workers who populate this market have been assessed on dimensions that are universally relevant to understanding whether, why, and when they should be recruited as research participants. We discuss the characteristics of MTurk as a participant pool for psychology and other social sciences, highlighting the traits of the MTurk samples, why people become MTurk workers and research participants, and how data quality on MTurk compares to that from other pools and depends on controllable and uncontrollable factors.

/pdf/inside-the-turk-understanding-mechanical-turk-as-a-33lzlsoyyp.pdf

Inside the Turk Understanding Mechanical Turk as a Participant Pool

where A represents the magnetic vector potential, is an integral of the hydromagnetic equations. This -integral made it possible to formulate a variational principle for the force-free magnetic fields. The integral expresses the fact that motions cannot transform a given field in an entirely arbitrary different field, if the conductivity of the medium isconsidered infinite. In this paper we shall show that the full set of hydromagnetic equations admit five more integrals, besides the energy integral, if dissipative processes are absent. These integrals, as we shall presently verify, are I2 =fbHvdV, (2)

Proceedings of the National Academy of Sciences

Amazon Mechanical Turk (AMT) is an online crowdsourcing service where anonymous online workers complete web-based tasks for small sums of money. The service has attracted attention from experimental psychologists interested in gathering human subject data more efficiently. However, relative to traditional laboratory studies, many aspects of the testing environment are not under the experimenter's control. In this paper, we attempt to empirically evaluate the fidelity of the AMT system for use in cognitive behavioral experiments. These types of experiment differ from simple surveys in that they require multiple trials, sustained attention from participants, comprehension of complex instructions, and millisecond accuracy for response recording and stimulus presentation. We replicate a diverse body of tasks from experimental psychology including the Stroop, Switching, Flanker, Simon, Posner Cuing, attentional blink, subliminal priming, and category learning tasks using participants recruited using AMT. While most of replications were qualitatively successful and validated the approach of collecting data anonymously online using a web-browser, others revealed disparity between laboratory results and online results. A number of important lessons were encountered in the process of conducting these replications that should be of value to other researchers.

/pdf/evaluating-amazon-s-mechanical-turk-as-a-tool-for-1acmhyqzqj.pdf

Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research

Prolific.ac—A subject pool for online experiments

Economic games are used to investigate the cognitive mechanisms underlying cooperative behaviour, and show that intuition supports cooperation in social dilemmas, whereas reflection can undermine these cooperative impulses. Many people are willing to make sacrifices for the common good, but little is known about the cognitive mechanisms that underlie such cooperative behaviour. In economic experiments subjects often contribute cooperatively against what rational self-interest should dictate. This study uses a series of ten varied experimental designs, including both one-shot and repeated games, to establish whether we are intuitively predisposed to cooperate or to act selfishly. And it seems our gut response is to cooperate — but given more time to think the logic of self-interest undermines collective action and we become less generous. Cooperation is central to human social behaviour1,2,3,4,5,6,7,8,9. However, choosing to cooperate requires individuals to incur a personal cost to benefit others. Here we explore the cognitive basis of cooperative decision-making in humans using a dual-process framework10,11,12,13,14,15,16,17,18. We ask whether people are predisposed towards selfishness, behaving cooperatively only through active self-control; or whether they are intuitively cooperative, with reflection and prospective reasoning favouring ‘rational’ self-interest. To investigate this issue, we perform ten studies using economic games. We find that across a range of experimental designs, subjects who reach their decisions more quickly are more cooperative. Furthermore, forcing subjects to decide quickly increases contributions, whereas instructing them to reflect and forcing them to decide slowly decreases contributions. Finally, an induction that primes subjects to trust their intuitions increases contributions compared with an induction that promotes greater reflection. To explain these results, we propose that cooperation is intuitive because cooperative heuristics are developed in daily life where cooperation is typically advantageous. We then validate predictions generated by this proposed mechanism. Our results provide convergent evidence that intuition supports cooperation in social dilemmas, and that reflection can undermine these cooperative impulses.

/pdf/spontaneous-giving-and-calculated-greed-10q2b8pxf2.pdf

Spontaneous giving and calculated greed

Online labor markets such as Amazon Mechanical Turk (MTurk) off er an unprecedented opportunity to run economic game experiments quickly and inexpensively. Using Mturk, we recruited 756 subjects and examined their behavior in four canonical economic games, with two payoff conditions each: a stakes condition, in which subjects' earnings were based on the outcome of the game (maximum earnings of $1); and a no-stakes condition, in which subjects' earnings are una ffected by the outcome of the game. Our results demonstrate that economic game experiments run on MTurk are comparable to those run in laboratory settings, even when using very low stakes.

/pdf/economic-games-on-the-internet-the-effect-of-1-stakes-16ghtp1b2w.pdf

Economic Games on the Internet: The Effect of $1 Stakes

Online labor markets such as Amazon Mechanical Turk (MTurk) offer an unprecedented opportunity to run economic game experiments quickly and inexpensively. Using Mturk, we recruited 756 subjects and examined their behavior in four canonical economic games, with two payoff conditions each: a stakes condition, in which subjects' earnings were based on the outcome of the game (maximum earnings of $1); and a no-stakes condition, in which subjects' earnings are unaffected by the outcome of the game. Our results demonstrate that economic game experiments run on MTurk are comparable to those run in laboratory settings, even when using very low stakes.

/pdf/economic-games-on-the-internet-the-effect-of-1-stakes-56s1ergixq.pdf

Economic games on the internet: the effect of $1 stakes.

Semitic languages pose a problem to Natural Language Processing since most of the vowels are omitted from written prose, resulting in considerable ambiguity at the word level. However, while reading text, native speakers can generally vocalize each word based on their familiarity with the lexicon and the context of the word. Methods for vowel restoration in previous work involving morphological analysis concentrated on a single language and relied on a parsed corpus that is difficult to create for many Semitic languages. We show that Hidden Markov Models are a useful tool for the task of vowel restoration in Semitic languages. Our technique is simple to implement, does not require any language specific knowledge to be embedded in the model and generalizes well to both Hebrew and Arabic. Using a publicly available version of the Bible and the Qur'an as corpora, we achieve a success rate of 86% for restoring the exact vowel pattern in Arabic and 81% in Hebrew. For Hebrew, we also report on 87% success rate for restoring the correct phonetic value of the words.

/pdf/an-hmm-approach-to-vowel-restoration-in-arabic-and-hebrew-3xmi9m2nab.pdf

An HMM Approach to Vowel Restoration in Arabic and Hebrew

https://dash.harvard.edu/bitstream/handle/1/4726287/gal-2010-adm-distrib.pdf?sequence=1

Agent decision-making in open mixed networks

This paper presents Networks of Influence Diagrams (NID), a compact, natural and highly expressive language for reasoning about agents' beliefs and decision-making processes. NIDs are graphical structures in which agents' mental models are represented as nodes in a network; a mental model for an agent may itself use descriptions of the mental models of other agents. NIDs are demonstrated by examples, showing how they can be used to describe conflicting and cyclic belief structures, and certain forms of bounded rationality. In an opponent modeling domain, NIDs were able to outperform other computational agents whose strategies were not known in advance. NIDs are equivalent in representation to Bayesian games but they are more compact and structured than this formalism. In particular, the equilibrium definition for NIDs makes an explicit distinction between agents' optimal strategies, and how they actually behave in reality.

/pdf/networks-of-influence-diagrams-a-formalism-for-representing-2py0pdw4zt.pdf

Ya'akov Gal

Papers

Economic Games on the Internet: The Effect of $1 Stakes

Economic games on the internet: the effect of $1 stakes.

An HMM Approach to Vowel Restoration in Arabic and Hebrew

Agent decision-making in open mixed networks

Networks of influence diagrams: a formalism for representing agents' beliefs and decision-making processes