Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique
TLDR
Freedman [Adv. in Appl. Math. as mentioned in this paper argued that adjustment can lead to worsened asymptotic precision, invalid measures of precision, and small-sample bias, and in sufficiently large samples, those problems are either minor or easily fixed.Abstract:
Freedman [Adv. in Appl. Math. 40 (2008) 180–193; Ann. Appl. Stat. 2 (2008) 176–196] critiqued ordinary least squares regression adjustment of estimated treatment effects in randomized experiments, using Neyman’s model for randomization inference. Contrary to conventional wisdom, he argued that adjustment can lead to worsened asymptotic precision, invalid measures of precision, and small-sample bias. This paper shows that in sufficiently large samples, those problems are either minor or easily fixed. OLS adjustment cannot hurt asymptotic precision when a full set of treatment–covariate interactions is included. Asymptotically valid confidence intervals can be constructed with the Huber–White sandwich standard error estimator. Checks on the asymptotic approximations are illustrated with data from Angrist, Lang, and Oreopoulos’s [Am. Econ. J.: Appl. Econ. 1:1 (2009) 136–163] evaluation of strategies to improve college students’ achievement. The strongest reasons to support Freedman’s preference for unadjusted estimates are transparency and the dangers of specification search.read more
Citations
More filters
Journal ArticleDOI
How Conditioning on Posttreatment Variables Can Ruin Your Experiment and What to Do about It
TL;DR: The authors demonstrate the severity of experimental post-treatment bias analytically and document the magnitude of the potential distortions it induces using visualizations and reanalyses of real-world data.
Journal ArticleDOI
Regression Discontinuity Designs Using Covariates
TL;DR: In this paper, the authors examined local polynomial estimators that include discrete or continuous covariates in an additive separable separable regression discontinuity design, where covariates are included in the estimation.
Journal ArticleDOI
How Much Should We Trust Estimates from Multiplicative Interaction Models? Simple Tools to Improve Empirical Practice
TL;DR: This paper found that a large portion of findings across all political science subfields based on interaction models are fragile and model dependent, and proposed a checklist of simple diagnostics to assess the validity of these assumptions and offer flexible estimation strategies that allow for nonlinear interaction effects.
Book ChapterDOI
The Econometrics of Randomized Experiments
TL;DR: In this paper, the authors present econometric and statistical methods for analyzing randomized experiments, and stress the general efficiency gains from stratification, and contrast intention to treat analyses with instrumental variables analyses allowing for general treatment effect heterogeneity.
Journal ArticleDOI
Sources of Authoritarian Responsiveness: A Field Experiment in China
TL;DR: Li et al. as mentioned in this paper conducted an online field experiment among 2,103 Chinese counties to examine factors that affect officials' incentives to respond to citizens in an authoritarian context, finding that approximately one third of county governments respond to citizen demands expressed online.
References
More filters
Journal ArticleDOI
A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity
TL;DR: In this article, a parameter covariance matrix estimator which is consistent even when the disturbances of a linear regression model are heteroskedastic is presented, which does not depend on a formal model of the structure of the heteroSkewedness.
Posted Content
Mostly Harmless Econometrics: An Empiricist's Companion
TL;DR: The core methods in today's econometric toolkit are linear regression for statistical control, instrumental variables methods for the analysis of natural experiments, and differences-in-differences methods that exploit policy changes.
The behavior of maximum likelihood estimates under nonstandard conditions
TL;DR: In this paper, the authors prove consistency and asymptotic normality of maximum likelihood estimators under weaker conditions than usual, such that the true distribution underlying the observations belongs to the parametric family defining the estimator, and the regularity conditions do not involve the second and higher derivatives of the likelihood function.
Journal ArticleDOI
Recent developments in the econometrics of program evaluation
TL;DR: In the last two decades, much research has been done on the econometric and statistical analysis of such causal effects as discussed by the authors, which has reached a level of maturity that makes it an important tool in many areas of empirical research in economics, including labor economics, public finance, development economics, industrial organization, and other areas in empirical microeconomics.
Journal ArticleDOI
Multivariate regression models for panel data
TL;DR: In this article, the authors examined the relationship between heterogeneity bias and strict exogeneity in a distributed lag regression of y on x, and showed that the relationship is very strong when x is continuous, weaker when X is discrete, and non-existent as the order of the distributed lag becomes infinite.