Regression discontinuity designs in economics

TLDR: In this paper, the authors provide an introduction and user guide to regression discontinuity (RD) designs for empirical researchers, and discuss the advantages and disadvantages of estimating RD designs and the limitations of interpreting these estimates.

Abstract: This paper provides an introduction and “user guide” to Regression Discontinuity (RD) designs for empirical researchers. It presents the basic theory behind the research design, details when RD is likely to be valid or invalid given economic incentives, explains why it is considered a “quasi-experimental” design, and summarizes different ways (with their advantages and disadvantages) of estimating RD designs and the limitations of interpreting these estimates. Concepts are discussed using examples drawn from the growing body of empirical research using RD. ( JEL C21, C31)

Figures

Table 3: Optimal Bandwidth for Local Linear Regressions, Voting Example

Table 4: Regression Discontinuity Applications in Economics

Table 1: Choice of Bandwidth in Graph for Voting Example

Figure 5. Treatment, Observables, and Unobservables in four research designs.

Full text

WORKING PAPER # 548
INDUSTRIAL RELATIONS SECTION
PRINCETON UNIVERSITY
FEBRUARY 2009
NBER WORKING PAPER SERIES
REGRESSION DISCONTINUITY DESIGNS IN ECONOMICS
David S. Lee
Thomas Lemieux
Working Paper 14723
http://www.nber.org/papers/w14723
NATIONAL BUREAU OF ECONOMIC RESEARCH
1050 Massachusetts Avenue
Cambridge, MA 02138
February 2009
We thank David Autor, David Card, John DiNardo, Guido Imbens, and Justin McCrary for
suggestions for this article, as well as for numerous illuminating discussions on the various topics we
cover in this review. We also thank two anonymous referees for their helpful suggestions and
comments. Emily Buchsbaum, Elizabeth Debraggio, Enkeleda Gjeci, Ashley Hodgson, Xiaotong
Niu, and Zhuan Pei provided excellent research assistance.¸ The views expressed herein are those of
the author(s) and do not necessarily reflect the views of the National Bureau of Economic Research.
© 2009 by David S. Lee and Thomas Lemieux. All rights reserved. Short sections of text, not to
exceed two paragraphs, may be quoted without explicit permission provided that full credit,
including © notice, is given to the source.

Regression Discontinuity Designs in Economics
David S. Lee and Thomas Lemieux
NBER Working Paper No. 14723
February 2009
JEL No. C1,H0,I0,J0
ABSTRACT
This paper provides an introduction and "user guide" to Regression Discontinuity (RD) designs for
empirical researchers. It presents the basic theory behind the research design, details when RD is likely
to be valid or invalid given economic incentives, explains why it is considered a "quasi-experimental"
design, and summarizes different ways (with their advantages and disadvantages) of estimating RD
designs and the limitations of interpreting these estimates. Concepts are discussed using using examples
drawn from the growing body of empirical research using RD.
David S. Lee
Industrial Relations Section
Princeton University
Firestone Library A-16-J
Princeton, NJ 08544
and NBER
davidlee@princeton.edu
Thomas Lemieux
Department of Economics
University of British Columbia
#997-1873 East Mall
Vancouver, BC V6T 1Z1
Canada
and NBER
tlemieux@interchange.ubc.ca

1 Introduction
Regression Discontinuity (RD) designs were first introduced by Thistlethwaite and Campbell (1960) as a
way of estimating treatment effects in a non-experimental setting where treatment is determined by whether
an observed “forcing” variable exceeds a known cutoff point. In their initial application of RD designs,
Thistlethwaite and Campbell (1960) analyzed the impact of merit awards on future academic outcomes,
using the fact that the allocation of these awards was based on an observed test score. The main idea behind
the research design was that individuals with scores just below the cutoff (who did not receive the award)
were good comparisons to those just above the cutoff (who did receive the award). Although this evaluation
strategy has been around for almost fifty years, it did not attract much attention in economics until relatively
recently.
Since the late 1990s, a growing number of studies have relied on RD designs to estimate program effects
in a wide variety of economic contexts. Like Thistlethwaite and Campbell (1960), early studies by Van der
Klaauw (2002) and Angrist and Lavy (1999) exploited threshold rules often used by educational institutions
to estimate the effect of financial aid and class size, respectively, on educational outcomes. Black (1999)
exploited the presence of discontinuities at the geographical level (school district boundaries) to estimate the
willingness to pay for good schools. Following these early papers in the area of education, the past five years
have seen a rapidly growing literature using RD designs to examine a range of questions. Examples include:
the labor supply effect of welfare, unemployment insurance, and disability programs; the effects of Medicaid
on health outcomes; the effect of remedial education programs on educational achievement; the empirical
relevance of median voter models; and the effects of unionization on wages and employment.
One important impetus behind this recent flurry of research is a recognition, formalized by Hahn et
al. (2001), that RD designs require seemingly mild assumptions compared to those needed for other non-
experimental approaches. Another reason for the recent wave of research is the belief that the RD design is
not “just another” evaluation strategy, and that causal inferences from RD designs are potentially more cred-
ible than those from typical “natural experiment” strategies (e.g. difference-in-differences or instrumental
variables), which have been heavily employed in applied research in recent decades. This notion has a the-
oretical justification: Lee (2008) formally shows that one need not assume the RD design isolates treatment
variation that is “as good as randomized”; instead, such randomized variation is a consequence of agents’
inability to precisely control the forcing variable near the known cutoff.
1

So while the RD approach was initially thought to be “just another” program evaluation method with
relatively little general applicability outside of a few specific problems, recent work in economics has shown
quite the opposite.
1
In addition to providing a highly credible and transparent way of estimating program
effects, RD designs can be used in a wide variety of contexts covering a large number of important economic
questions. These two facts likely explain why the RD approach is rapidly becoming a major element in the
toolkit of empirical economists.
Despite the growing importance of RD designs in economics, there is no single comprehensive summary
of what is understood about RD designs – when they succeed, when they fail, and their strengths and weak-
nesses.
2
Furthermore, the “nuts and bolts” of implementing RD designs in practice are not (yet) covered in
standard econometrics texts, making it difficult for researchers interested in applying the approach to do so.
Broadly speaking, the main goal of this paper is to fill these gaps by providing an up-to-date overview of RD
designs in economics and creating a guide for researchers interested in applying the method.
A reading of the most recent research reveals a certain body of “folk wisdom” regarding the applicability,
interpretation, and recommendations of practically implementing RD designs. This article represents our
attempt at identifying what we believe are the most important of these pieces of wisdom, while also dispelling
misconceptions that could potentially (and understandably) arise for those new to the RD approach.
We will now briefly summarize what we see as the main points in the “folk wisdom” about RD designs
to set the stage for the rest of the paper where we systematically discuss identification, interpretation, and
estimation issues. Here, and throughout the paper, we refer to the forcing variable as X. Treatment is, thus,
assigned to individuals (or “units”) with a value of X greater than or equal to a cutoff value c.
• RD designs can be invalid if individuals can precisely manipulate the “forcing variable”.
When there is a payoff or benefit to receiving a treatment, it is natural for an economist to consider how
an individual may behave to obtain such benefits. For example, if students could effectively “choose”
their test score X through effort, those who chose a score c (and hence received the merit award) could
be somewhat different from those who chose scores just below c. The important lesson here is that
the existence of a treatment being a discontinuous function of a forcing variable is not sufficient to
justify the validity of an RD design. Indeed, if anything, discontinuous rules may generate incentives,
1
See Cook (2008) for an interesting history of the RD design in education research, psychology, statistics, and economics. Cook
argues the resurgence of the RD design in economics is unique as it is still rarely used in other disciplines.
2
See, however, two recent overview papers by Van der Klaauw (2008b) and Imbens and Lemieux (2008) that have begun bridging
this gap.
2

causing behavior that would invalidate the RD approach.
• If individuals – even while having some influence – are unable to precisely manipulate the forcing
variable, a consequence of this is that the variation in treatment near the threshold is random-
ized as though from a randomized experiment.
This is a crucial feature of the RD design, since it is the reason RD designs are often so compelling.
Intuitively, when individuals have imprecise control over the forcing variable, even if some are espe-
cially likely to have values of X near the cutoff, every individual will have approximately the same
probability of having an X that is just above (receiving the treatment) or just below (being denied the
treatment) the cutoff – similar to a coin-flip experiment. This result clearly differentiates the RD and
IV approaches. When using IV for causal inference, one must assume the instrument is exogenously
generated as if by a coin-flip. Such an assumption is often difficult to justify (except when an actual
lottery was run, as in Angrist (1990), or if there were some biological process, e.g. gender determina-
tion of a baby, mimicking a coin-flip). By contrast, the variation that RD designs isolate is randomized
as a consequence of individuals having imprecise control over the forcing variable.
• RD designs can be analyzed – and tested – like randomized experiments.
This is the key implication of the local randomization result. If variation in the treatment near the
threshold is approximately randomized, then it follows that all “baseline characteristics” – all those
variables determined prior to the realization of the forcing variable – should have the same distribution
just above and just below the cutoff. If there is a discontinuity in these baseline covariates, then at a
minimum, the underlying identifying assumption of individuals’ inability to precisely manipulate the
forcing variable is unwarranted. Thus, the baseline covariates are used to test the validity of the RD
design. By contrast, when employing an IV or a matching/regression-control strategy, assumptions
typically need to be made about the relationship of these other covariates to the treatment and outcome
variables.
3
• Graphical presentation of an RD design is helpful and informative, but the visual presentation
should not be tilted toward either finding an effect or finding no effect.
It has become standard to summarize RD analyses with a simple graph showing the relationship be-
3
Typically, one assumes that conditional on the covariates, the treatment (or instrument) is essentially “as good as” randomly
assigned.
3

Frequently asked questions

Q1. What are the contributions in this paper?

The authors thank David Autor, David Card, John DiNardo, Guido Imbens, and Justin McCrary for suggestions for this article, as well as for numerous illuminating discussions on the various topics they cover in this review. The views expressed herein are those of the author ( s ) and do not necessarily reflect the views of the National Bureau of Economic Research. The authors also thank two anonymous referees for their helpful suggestions and comments. 

Q2. What are the future works in this paper?

The authors believe a “ state-of-the-art ” RD analysis today will consider carefully the possibility of endogenous sorting. Consequently, when the authors control on x as in the multiple regression, z has no explanatory power with respect to y [ the outcome measured with error ]. They also provide evidence that those families in schools just to the left and right of the thresholds are systematically different in family income, suggesting some degree of sorting. 41 From a practitioner ’ s perspective, an important recent development is the notion that the authors can empirically examine the degree of sorting, and one way of doing so is suggested in McCrary ( 2008 ). 

Q3. How can the treatment effect be recovered in an instrumental variable setting?

As in an instrumental variable setting however, the treatment effect can be recovered by dividing the jump in the relationship between Y and X at c by the fraction induced to take-up the treatment at the threshold – in other words, the discontinuity jump in the relation between D and X . 

Q4. What is the simplest way of estimating the treatment effect?

Assuming that the relationship between Y and X is otherwise linear, a simple way of estimating the treatment effect τ is by fitting the linear regressionY = α + Dτ + Xβ + ε (1)where ε is the usual error term that can be viewed as a purely random error generating variation in the value of Y around the regression line α + 

Q5. What is the continuity assumption for the RD estimates of the treatment effect?

Since the authors know the monetary compensation is a continuous function of X , the authors also know the continuity assumption required for the RD estimates of the treatment effect to be consistent is also satisfied. 

Q6. What is the direct evidence of a sorting around the threshold?

A jump in the density at the threshold is probably the most direct evidence of some degree of sorting around the threshold, and should provoke serious skepticism about the appropriateness of the RD design. 

Q7. Why are both W and U systematically related to X?

Both W and U may be systematically related to X , perhaps due to the actions taken by units to increase their probability of receiving treatment. 

Q8. What is the main reason for the recent flurry of research using RD designs?

In addition to providing a highly credible and transparent way of estimating program effects, RD designs can be used in a wide variety of contexts covering a large number of important economic questions. 

Q9. What is the main reason behind the recent flurry of research using RD designs?

One important impetus behind this recent flurry of research is a recognition, formalized by Hahn et al. (2001), that RD designs require seemingly mild assumptions compared to those needed for other nonexperimental approaches. 

Q10. What is the resulting estimate of the true discontinuity gap?

In this example, if the slope β were (erroneously) restricted to equal zero, it is clear the resulting OLS coefficient on D would be a biased estimate of the true discontinuity gap. 

Q11. What is the difference between the probability of treatment and the threshold?

Since the probability of treatment jumps by less than one at the threshold, the jump in the relationship between Y and X can no longer be interpreted as an average treatment effect.