Field Experiments in Markets

TLDR: In this article, a review of the literature of field experimental studies of markets is presented, and the main results covered by the review are as follows: (1) Generally speaking, markets organize the efficient exchange of commodities; (2) There are some behavioral anomalies that impede efficient exchange; (3) Many behavioral anomalies disappear when traders are experienced.

Abstract: This is a review of the literature of field experimental studies of markets. The main results covered by the review are as follows: (1) Generally speaking, markets organize the efficient exchange of commodities; (2) There are some behavioral anomalies that impede efficient exchange; (3) Many behavioral anomalies disappear when traders are experienced.

Figures

Figure 1: Demand/Supply Schedules in the Symmetric Markets in List (2004)

Full text

NBER WORKING PAPER SERIES
FIELD EXPERIMENTS IN MARKETS
Omar Al-Ubaydli
John A. List
Working Paper 22113
http://www.nber.org/papers/w22113
NATIONAL BUREAU OF ECONOMIC RESEARCH
1050 Massachusetts Avenue
Cambridge, MA 02138
March 2016
We would like to thank Andrew Simon for excellent research assistance. The views expressed
herein are those of the authors and do not necessarily reflect the views of the National Bureau of
Economic Research.
NBER working papers are circulated for discussion and comment purposes. They have not been
peer-reviewed or been subject to the review by the NBER Board of Directors that accompanies
official NBER publications.
© 2016 by Omar Al-Ubaydli and John A. List. All rights reserved. Short sections of text, not to
exceed two paragraphs, may be quoted without explicit permission provided that full credit,
including © notice, is given to the source.

Field Experiments in Markets
Omar Al-Ubaydli and John A. List
NBER Working Paper No. 22113
March 2016
JEL No. C93,D01,D03
ABSTRACT
This is a review of the literature of field experimental studies of markets. The main results
covered by the review are as follows: (1) Generally speaking, markets organize the efficient
exchange of commodities;(2) There are some behavioral anomalies that impede efficient
exchange; (3) Many behavioral anomalies disappear when traders are experienced.
Omar Al-Ubaydli
Department of Economics and Mercatus Center
George Mason University
omar@omar.ec
John A. List
Department of Economics
University of Chicago
1126 East 59th
Chicago, IL 60637
and NBER
jlist@uchicago.edu

1
FIELD EXPERIMENTS IN MARKETS
Omar Al-Ubaydli and John A. List
1
September 2015
ABSTRACT
This is a review of the literature of field experimental studies of markets. The main results covered
by the review are as follows: (1) Generally speaking, markets organize the efficient exchange of
commodities; (2) There are some behavioral anomalies that impede efficient exchange; (3) Many
behavioral anomalies disappear when traders are experienced.
1. INTRODUCTION
Traditionally, the study of economics is virtually synonymous with the study of markets, with the
most notable illustration being Adam Smith’s Wealth of Nations—arguably the discipline’s
inaugural contribution. The Scottish economist’s treatise was followed by seminal contributions
from numerous luminaries such as Alfred Marshall, John Maynard Keynes, Friedrich Von Hayek,
Kenneth Arrow, Gary Becker, and Robert Lucas. While the range of topics studied by economists
has undoubtedly expanded in the years following the Freakonomics revolution, markets remain
the centerpiece of the discipline’s intellectual mission. For example, in the 14 years since 2001,
eight Nobel prizes in economics have been explicitly for research on markets, and of the remaining
six, two were for econometric methods that are most frequently applied to the study of markets.
The methodological tools deployed by economists have evolved from the narrative and deductive
arguments of the likes of John Stuart Mill, going on to introduction of elementary mathematical
methods by the likes of Leon Walras, followed by the formal decision-theoretic mathematical
machinery used by the likes of John Hicks and Gerard Debreu, the game-theoretic analysis of
scholars such as James Mirrlees and George Akerlof, and most recently, the arrival of agent-based
modeling. These theoretical contributions have been complemented by a huge volume of empirical
work, with some of the most notable studies relating to international trade and financial markets.
Without doubt, our understanding of how markets function has advanced immeasurably due to the
efforts of the aforementioned scholars.
1
We would like to thank Andrew Simon for excellent research assistance. Affiliations: Al-Ubaydli: Bahrain Center
for Strategic, International and Energy Studies and Department of Economics and Mercatus Center, George Mason
University. List: University of Chicago and NBER.

2
Until the 2002 Nobel prize, one of the most important contributions to our understanding of
markets—Vernon Smith’s (1962, 1965) real-stakes double oral auctions—remained under the
radar of most mainstream economists. Smith’s experiments, which followed in the footsteps of
Chamberlin (1948), spawned a massive experimental literature investigating market processes.
Subsequent scholars have examined alternative institutions, such as conventional auctions
(Coppinger et al. 1980), decentralized bilateral bargaining (Hong and Plott, 1982), and posted
prices (Plott, 1986). They have also varied the information structure to study important phenomena
such as asset bubbles (Smith et al., 1988), while other studies have examined the possibility of
social preferences interfering with the market-clearing process (Fehr et al., 1993). The single most
important conclusion emerging from the early experimental literature was one that no theoretical
or non-experimental study had ever convincingly demonstrated: markets lead to the efficient
exchange of commodities, and that this occurs even when many of the traditional assumptions of
“perfect markets” break down, i.e., when there is a small number of price-setting traders who have
incomplete information, and in the absence of a centralized orchestrator such as the nebulous
“Walrasian auctioneer” (Hayek, 1945; Smith, 1982).
Compared to conventional naturally-occurring data, the key advantage offered by the laboratory
experimental methods pioneered by Vernon Smith was the ability to artificially control how much
traders valued the commodities being traded (known as inducing values), as this allowed
researchers to accurately estimate demand and supply schedules, thereby permitting precise
welfare analysis. This point should not be understated: while mainstream economics embraced
Adam Smith’s theory of the “invisible hand,” it took almost 200 years for economists to present
plausible evidence of the phenomenon. Despite the fundamental importance of this result,
laboratory experiments in general (let alone ones used to study markets) remained a niche tool.
One of the reasons for the relative lack of enthusiasm among many mainstream economists was
the perceived artificiality of laboratory experiments: many scholars were concerned that the small
stakes, the restricted strategy space, the inexperience of subjects, and the scrutiny of the
experimenter all contributed to empirical insights that were of limited value to understanding
naturally-occurring markets (Levitt and List, 2007). In particular, many behavioral anomalies that
were regularly detected in laboratory experiments, such as the endowment effect (Knetsch, 1989)
and reciprocity in zero-reputation environments (Fehr and Gachter, 2002), were inconsistent with
both the premises and the conclusions of the neoclassical model, and were being used as an
intellectual platform to challenge mainstream economics.
Just as the theoretical and empirical literatures on markets developed in line with the advancement
of the available methodological tools, so too did the sub-literature on experimental studies of
markets with the advent of field experimental techniques in the early 2000s. This change was a
fundamental step towards unifying empirical methods as in many situations, field experimental
techniques offered the advantages of both conventional naturally-occurring data and laboratory
experimental data with neither set of disadvantages. Empirical economists who would previously

3
reject laboratory experimental data due to a perceived lack of realism embraced natural field
experiments as an elixir for the endogeneity problem associated with naturally-occurring data.
In particular, field experiments allowed researchers to investigate real traders operating in real
markets without the researcher having to surrender control over the environment. Consequently,
the literature offered more sophisticated answers to questions such as: “how prevalent is ethnic
discrimination in rental markets?,” (Ahmed and Hammarstedt, 2008), “what are the productivity
consequences of increasing wage rates?,” (Gneezy and List, 2006), and “what is the impact of
business training on the microfinance market?,” (Karlan and Valdivia, 2011).
Markets are a large and extremely diverse area of research and many of the other studies in this
volume cover important components of the literature. In this review, we focus on the overarching
conclusions. Moreover, in light of the literature’s relative youth, we also discuss some of
methodological issues associated with the literature.
In our review of the literature on field experimental studies of markets, we present three main
conclusions. The first is the most important: generally speaking, markets organize the efficient
exchange of commodities. Second, consistent with much of the laboratory experimental literature,
there exist behavioral anomalies that impede markets’ ability to organize the efficient exchange of
commodities. Third, many behavioral anomalies disappear when traders are sufficiently
experienced in their roles, rehabilitating markets’ ability to organize the efficient exchange of
commodities.
This review is organized as follows. Section 2 is the preamble, where we discuss: how markets are
defined; the studies that will be covered by this review; the classification of field experimental
studies of markets; and the advantages and disadvantages of field experiments when studying
markets. Section 3 is a presentation of the main results in the literature. Section 4 is a discussion
of the key methodological insights for scholars considering the use of field experiments in an
empirical market study. Section 5 provides closing remarks, including a discussion of possible
future field experimental studies of markets.
2. PREAMBLE
The importance of markets to academic economics is self-evident, and many of the earliest
contributions to the experimental literature in general (Chamberlin, 1948; Smith, 1965) and to the
field-experimental sub-literature in particular (List, 2004) were studies of markets. Accordingly,
when the editors of this volume approached us with the idea of writing a literature review on field
experiments in markets, in addition to being delighted and honored by the proposal, we regarded
it as a very logical component of a volume dedicated to field experiments.
As any author of a literature review is well aware, deciding on which studies to include can be
somewhat problematic as one attempts to balance the desire for parsimony with efforts at being

Frequently asked questions

Q1. What are the contributions in this paper?

This is a review of the literature of field experimental studies of markets. The main results covered by the review are as follows: ( 1 ) Generally speaking, markets organize the efficient exchange of commodities ; 

Q2. What are the common methods for eliciting trader values?

One of the most commonly used methods for eliciting trader values is surveys (also known as contingent valuation); in spite of the deployment of a variety of complex schemes, they remain costly, unwieldy, and unreliable methods of estimating values (Diamond and Hausman, 1994). 

Q3. What was the key advantage of the laboratory experimental methods?

Compared to conventional naturally-occurring data, the key advantage offered by the laboratory experimental methods pioneered by Vernon Smith was the ability to artificially control how much traders valued the commodities being traded (known as inducing values), as this allowed researchers to accurately estimate demand and supply schedules, thereby permitting precise welfare analysis. 

Q4. What is the recent study that uses the written approach?

One recent study that uses the written approach is due to Bertrand and Mullainathan (2002), who manipulate perception of race by randomly assigning white-sounding or black-sounding names to resumes sent to various prospective employers in Boston and Chicago. 

Q5. What is the main reason for the abundance of reduced-form models?

The consequent abundance of reduced-form models reflects researchers’ efforts at inductively learning about markets with any method that comes to hand. 

Q6. Why is the literature so difficult to produce the kind of overarching conclusions that the authors are interested?

Due to the expanded opportunity for generating testable hypotheses, and the relative youth of the literature on field experiments, in their opinion, the literature is yet to produce the sort of overarching conclusions that the authors are interested in for the purposes of this paper. 

Q7. What is the alternative route taken by some studies?

(An alternative route taken by some studies, such as Gjerstad and Dickhaut (1998), is to relax the strict rationality assumption and use agent-based modeling to derive testable predictions.) 

Q8. Why is the field experimentation of auctions so exciting?

This is particularly exciting for field experimentalists because in the case of conventional markets, scholars require knowledge of the values of market participants to be able to investigate the limited range of testable predictions. 

Q9. What is the reason for the comparative ease of field experimentation with auctions?

A further reason for the comparative ease of field experimentation with auctions is that auctions are inherently one-off exercises, meaning that the researcher can oversee and potentially control the entirety of the process. 

Q10. How can they explore the neoclassical theory of price endings?

By systematically changing a product’s price by varying the presence or absence of a $9 price ending, they can explore various features of neoclassical theory.