We investigate conditions sufficient for identification of average treatment effects using instrumental variables. First we show that the existence of valid instruments is not sufficient to identify any meaningful average treatment effect. We then establish that the combination of an instrument and a condition on the relation between the instrument and the participation status is sufficient for identification of a local average treatment effect for those who can be induced to change their participation status by changing the value of the instrument. Finally we derive the probability limit of the standard IV estimator under these conditions. It is seen to be a weighted average of local average treatment effects.

Evolution and Rationality Some Recent Game-Theoretic Results. Identification and Estimation of Local Average Treatment Effects

Economics Of Discrimination

Difference-in-Differences with multiple time periods

This paper reviews a theory of causal inference based on the Structural Causal Model (SCM) described in (Pearl, 2000a). The theory unifies the graphical, potential-outcome (Neyman-Rubin), decision analytical, and structural equation approaches to causation, and provides both a mathematical foundation and a friendly calculus for the analysis of causes and counterfactuals. In particular, the paper establishes a methodology for inferring (from a combination of data and assumptions) the answers to three types of causal queries: (1) queries about the effect of potential interventions, (2) queries about counterfactuals, and (3) queries about the direct (or indirect) effect of one event on another.

What is Causal Inference

A clinician or a medical laboratory worker who turns over the 250 pages of Professor R. A. FISHER's new book' and discovers that more than 200 pages are devoted to the discussion of the yield of plots of land at Rothamsted Experimental Station will be apt to conclude that the wQrk is no, concern of his. If, having a few minutes to spare, he samples -a few pages and discovers that, although a most sparing use is made of mathematical symbols or formulae, the author's argument requires a strenuous effort of attention to follow, his conclusion may be strengthened. It is, however, quite wrong. The adjecti-ves \" great,\" \" masterly,\" and \" epoch-making \" may properly be abandoned to the Sunday reviewers of novels and poems, but it is safe to say that this book is one of the most important contributions to scientific methodology of our generation. The lessons the author has to teach are well epitomized in the following quotations. \" It is an essential condition of experimentation that the experimental material is known to be variable, but it is not known, in respect of any individual, in what direction his response to a given treatment will vary from the average. No direct allowance for this variability can, therefore, be made. The knowledge. which guides us in increasing the precision of an experiment is not a knowledge of the individual peculiarities of particular experimental units, such as plots ofland, experimental animals,coco-nut palms, or hospital patients, but a knowledge that there is less variation within certain aggregates of these than there is among different individuals belonging to different aggregates.\" (Page 78.) \" We are usually ignorant which, out of innumerable possible factors, may prove ultimately to be the most important, though we may have strong presuppositions that some few of them are particularly worthy of study. We have usually no knowledge that any one factor will exert its effect independently of all others that can be varied, or that its effects are particularly simply related to variations in these other factors. On the contrary, if single factors are chosen for investigation, it is not because we anticipate that the laws of nature can be expressed with any particular simplicity in terms of these variables, but because they are variables which can be controlled or measured with comparative ease. If the investigator, in these circumstances, confines his attention to any single factor, we may infer either that he is the unfortunate victim of a doctrinaire theory as to how experimentation should proceed, or that the time, material, or equipment at his disposal are too limited to allow him to give attention to more than one narrow aspect of his problem.\" (Page 97.) \"We may, by deliberately varying in each case some of the conditions of the experiment, achieve a wider inductive basis for our conclusions, witlout in any degree impairing their precision.\" (Page 107.)

Planning of experiments

This paper proposes tools for robust inference in diﬀerence-in-diﬀerences and event-study designs where the parallel trends assumption may be violated. Instead of re-quiring that parallel trends holds exactly, we impose restrictions on how diﬀerent the post-treatment violations of parallel trends can be from the pre-treatment diﬀerences in trends (“pre-trends”). The causal parameter of interest is partially identiﬁed under these restrictions. We introduce two approaches that guarantee uniformly valid inference under the imposed restrictions, and we derive novel results showing that they have desirable power properties in our context. We illustrate how economic knowledge can inform the restrictions on the possible violations of parallel trends in two economic applications. We also highlight how our approach can be used to conduct sensitivity analyses showing what causal conclusions can be drawn under various restrictions on the possible violations of the parallel trends assumption. This online appendix contains proofs and additional results for the paper “A More Credible Approach to Parallel Trends” by Ashesh Rambachan and Jonathan Roth. Section A contains proofs and auxilliary lemmas for results stated in the main text. Section B contains additional details and results from our simulations. the causal eﬀect in the ﬁrst post-period ( θ “ τ 1 ). For ∆ SD p M q and ∆ SDPB p M q , we report the performance of the FLCI, conditional conﬁdence set, and conditional-least favorable conﬁdence set. For ∆ SDRM p ¯ M q and ∆ RM p ¯ M q , we report the performance of the conditional conﬁdence set and the conditional-least favorable conﬁdence set. All results are averaged over 1000 bootstrap samples.

A More Credible Approach to Parallel Trends ∗

There are widespread concerns that the growing use of machine learning algorithms in important decisions may reproduce and reinforce existing discrimination against legally protected groups. Most of the attention to date on issues of "algorithmic bias" or "algorithmic fairness" has come from computer scientists and machine learning researchers. We argue that concerns about algorithmic fairness are at least as much about questions of how discrimination manifests itself in data, decision-making under uncertainty, and optimal regulation. To fully answer these questions, an economic framework is necessary—and as a result, economists have much to contribute.

An Economic Perspective on Algorithmic Fairness

In panel experiments, we randomly assign units to different interventions, measuring their outcomes, and repeating the procedure in several periods. Using the potential outcomes framework, we define finite population dynamic causal effects that capture the relative effectiveness of alternative treatment paths. For a rich class of dynamic causal effects, we provide a nonparametric estimator that is unbiased over the randomization distribution and derive its finite population limiting distribution as either the sample size or the duration of the experiment increases. We develop two methods for inference: a conservative test for weak null hypotheses and an exact randomization test for sharp null hypotheses. We further analyze the finite population probability limit of linear fixed effects estimators. These commonly-used estimators do not recover a causally interpretable estimand if there are dynamic causal effects and serial correlation in the assignments, highlighting the value of our proposed estimator.

Panel Experiments and Dynamic Causal Effects: A Finite Population Perspective

There is growing concern about "algorithmic bias" - that predictive algorithms used in decision-making might bake in or exacerbate discrimination in society. We argue that such concerns are naturally addressed using the tools of welfare economics. This approach overturns prevailing wisdom about the remedies for algorithmic bias. First, when a social planner builds the algorithm herself, her equity preference has no effect on the training procedure. So long as the data, however biased, contain signal, they will be used and the learning algorithm will be the same. Equity preferences alone provide no reason to alter how information is extracted from data - only how that information enters decision-making. Second, when private (possibly discriminatory) actors are the ones building algorithms, optimal regulation involves algorithmic disclosure but otherwise no restriction on training procedures. Under such disclosure, the use of algorithms strictly reduces the extent of discrimination relative to a world in which humans make all the decisions.

Institutional subscribers to the NBER working paper series, and residents of developing countries may download this paper without additional charge at www.nber.org.

/pdf/an-economic-approach-to-regulating-algorithms-3njhw1teon.pdf

An Economic Approach to Regulating Algorithms

We evaluate the folk wisdom that algorithmic decision rules trained on data produced by biased human decision-makers necessarily reflect this bias. We consider a setting where training labels are only generated if a biased decision-maker takes a particular action, and so "biased" training data arise due to discriminatory selection into the training data. In our baseline model, the more biased the decision-maker is against a group, the more the algorithmic decision rule favors that group. We refer to this phenomenon as "bias reversal." We then clarify the conditions that give rise to bias reversal. Whether a prediction algorithm reverses or inherits bias depends critically on how the decision-maker affects the training data as well as the label used in training. We illustrate our main theoretical results in a simulation study applied to the New York City Stop, Question and Frisk dataset.

Ashesh Rambachan

Papers

A More Credible Approach to Parallel Trends ∗

An Economic Perspective on Algorithmic Fairness

Panel Experiments and Dynamic Causal Effects: A Finite Population Perspective

An Economic Approach to Regulating Algorithms

Bias In, Bias Out? Evaluating the Folk Wisdom