Why Economists Should Conduct Field Experiments and 14 Tips for Pulling One Off
01 Aug 2011-Journal of Economic Perspectives (American Economic Association)-Vol. 25, Iss: 3, pp 3-16
TL;DR: In this paper, an overview of the spectrum of experimental methods in economics, from laboratory experiments to the field experiments that are the subject of this symposium, is presented, along with a mental checklist of 14 steps to improve the chances of carrying out an economics field experiment successfully.
Abstract: In this introduction to the symposium, I first offer an overview of the spectrum of experimental methods in economics, from laboratory experiments to the field experiments that are the subject of this symposium. I then offer some thoughts about the potential gains from doing economic research using field experiments and my own mental checklist of 14 steps to improve the chances of carrying out an economics field experiment successfully.
Citations
More filters
TL;DR: The authors review current research on the three main decisions that shape the corporate information environment in capital market settings: (1) managers' voluntary disclosure decisions, (2) disclosures mandated by regulators, and (3) reporting decisions by analysts.
1,387 citations
TL;DR: In this paper, the authors present a set of guidelines for stated preference studies that are more comprehensive than those of the original National Oceanic and Atmospheric Administration (NOAA) Blue Ribbon Panel on contingent valuation, and reflect the two decades of research since that time.
Abstract: This article proposes contemporary best-practice recommendations for stated preference (SP) studies used to inform decision making, grounded in the accumulated body of peer-reviewed literature. These recommendations consider the use of SP methods to estimate both use and non-use (passive-use) values, and cover the broad SP domain, including contingent valuation and discrete choice experiments. We focus on applications to public goods in the context of the environment and human health but also consider ways in which the proposed recommendations might apply to other common areas of application. The recommendations recognize that SP results may be used and reused (benefit transfers) by governmental agencies and nongovernmental organizations, and that all such applications must be considered. The intended result is a set of guidelines for SP studies that is more comprehensive than that of the original National Oceanic and Atmospheric Administration (NOAA) Blue Ribbon Panel on contingent valuation, is more germane to contemporary applications, and reflects the two decades of research since that time. We also distinguish between practices for which accumulated research is sufficient to support recommendations and those for which greater uncertainty remains. The goal of this article is to raise the quality of SP studies used to support decision making and promote research that will further enhance the practice of these studies worldwide.
896 citations
TL;DR: It is the hope that this Review will inspire more interesting, robust, multi-method, comparative, interdisciplinary and impactful research that will accelerate the contribution that energy social science can make to both theory and practice.
Abstract: A series of weaknesses in creativity, research design, and quality of writing continue to handicap energy social science. Many studies ask uninteresting research questions, make only marginal contributions, and lack innovative methods or application to theory. Many studies also have no explicit research design, lack rigor, or suffer from mangled structure and poor quality of writing. To help remedy these shortcomings, this Review offers suggestions for how to construct research questions; thoughtfully engage with concepts; state objectives; and appropriately select research methods. Then, the Review offers suggestions for enhancing theoretical, methodological, and empirical novelty. In terms of rigor, codes of practice are presented across seven method categories: experiments, literature reviews, data collection, data analysis, quantitative energy modeling, qualitative analysis, and case studies. We also recommend that researchers beware of hierarchies of evidence utilized in some disciplines, and that researchers place more emphasis on balance and appropriateness in research design. In terms of style, we offer tips regarding macro and microstructure and analysis, as well as coherent writing. Our hope is that this Review will inspire more interesting, robust, multi-method, comparative, interdisciplinary and impactful research that will accelerate the contribution that energy social science can make to both theory and practice.
670 citations
TL;DR: In this paper, the authors present a mnemonic, Mindspace, which gathers up the nine most robust effects that influence our behaviour in mostly automatic (rather than deliberate) ways.
571 citations
TL;DR: In this article, the authors describe what economists have learned about differences between group and individual decision-making, and compare individual decision making to group decision making, and to individual to individual.
Abstract: In this paper, we describe what economists have learned about differences In this paper, we describe what economists have learned about differences between group and individual decision-making. This literature is still young, and in between group and individual decision-making. This literature is still young, and in this paper, we will mostly draw on experimental work (mainly in the laboratory) that this paper, we will mostly draw on experimental work (mainly in the laboratory) that has compared individual decision-making to group decision-making, and to individual has compared individual decision-making to group decision-making, and to individual
360 citations
References
More filters
TL;DR: The authors discusses the central role of propensity scores and balancing scores in the analysis of observational studies and shows that adjustment for the scalar propensity score is sufficient to remove bias due to all observed covariates.
Abstract: : The results of observational studies are often disputed because of nonrandom treatment assignment. For example, patients at greater risk may be overrepresented in some treatment group. This paper discusses the central role of propensity scores and balancing scores in the analysis of observational studies. The propensity score is the (estimated) conditional probability of assignment to a particular treatment given a vector of observed covariates. Both large and small sample theory show that adjustment for the scalar propensity score is sufficient to remove bias due to all observed covariates. Applications include: matched sampling on the univariate propensity score which is equal percent bias reducing under more general conditions than required for discriminant matching, multivariate adjustment by subclassification on balancing scores where the same subclasses are used to estimate treatment effects for all outcome variables and in all subpopulations, and visual representation of multivariate adjustment by a two-dimensional plot. (Author)
23,744 citations
Emory University1, University of California, Los Angeles2, University of Siena3, California Institute of Technology4, University of Zurich5, Central European University6, University of California, Davis7, Texas A&M University8, University of Oxford9, University of New Mexico10, University of Pennsylvania11, University of California, Santa Barbara12, Harvard University13, California State University, Fullerton14, University of Colorado Denver15
TL;DR: A cross-cultural study of behavior in ultimatum, public goods, and dictator games in a range of small-scale societies exhibiting a wide variety of economic and cultural conditions found the canonical model – based on self-interest – fails in all of the societies studied.
Abstract: Researchers from across the social sciences have found consistent deviations from the predictions of the canonical model of self-interest in hundreds of experiments from around the world. This research, however, cannot determine whether the uniformity re- sults from universal patterns of human behavior or from the limited cultural variation available among the university students used in virtually all prior experimental work. To address this, we undertook a cross-cultural study of behavior in ultimatum, public goods, and dictator games in a range of small-scale societies exhibiting a wide variety of economic and cultural conditions. We found, first, that the canonical model - based on self-interest - fails in all of the societies studied. Second, our data reveal substantially more behavioral vari- ability across social groups than has been found in previous research. Third, group-level differences in economic organization and the structure of social interactions explain a substantial portion of the behavioral variation across societies: the higher the degree of market integration and the higher the payoffs to cooperation in everyday life, the greater the level of prosociality expressed in experimental games. Fourth, the available individual-level economic and demographic variables do not consistently explain game behavior, either within or across groups. Fifth, in many cases experimental play appears to reflect the common interactional patterns of everyday life.
1,589 citations
TL;DR: The authors show how experiments sometimes serve as instrumental variables to identify program impacts and the most favorable case for experiments ignores variability across persons in response to treatments received.
Abstract: Rs ecent academic debates pit two alternative approaches to policy evaluation against one another. The first is the "experimental" approach, based on the random assignment of accepted program applicants to a recipient, or treatment, group and a non-recipient, or control, group. The second is the "nonexperimental," or "econometric," approach that uses a variety of microdata sources, statistical methods, and behavioral models to compare the outcomes of participants in social programs with those of nonparticipants. The central question addressed in this paper is whether or not randomized social experiments aid in securing answers to basic questions about the evaluation of social programs. There are many distinct and complementary approaches to the study of the impact of public policy, including full general equilibrium analysis of policy impacts (Tinbergen, 1956; Auerbach and Kotlikoff, 1987; Shoven and Whalley, 1992; Kydland and Prescott, 1991) and less ambitious partial equilibrium microeconomic structural research programs, such as those designed to estimate the impact of taxes on labor supply. Both approaches offer answers to many interesting counterfactual policy questions, but their credibility rests critically on the quality of the empirical input used to generate their answers
960 citations
TL;DR: In this paper, the authors present a mnemonic, Mindspace, which gathers up the nine most robust effects that influence our behaviour in mostly automatic (rather than deliberate) ways.
571 citations
TL;DR: William Vickrey's predicted equivalences between first- price sealed-bid and Dutch auctions, and between second-price sealed- bid and English auctions, are tested using field experiments that auctioned off collectible trading cards over the Internet.
Abstract: William Vickrey's predicted equivalences between first-price sealed-bid and Dutch auctions, and between second-price sealed-bid and English auctions, are tested using field experiments that auctioned off collectible trading cards over the Internet. The results indicate that the Dutch auction produces 30-percent higher revenues than the first-price auction format, a violation of the theoretical prediction and a reversal of previous laboratory results, and that the English and second-price formats produce roughly equivalent revenues.
476 citations