Showing papers by "Joshua D. Angrist published in 2013"

The Perils of Peer Effects

Abstract: Individual outcomes are highly correlated with group average outcomes, a fact often interpreted as a causal peer effect. Without covariates, however, outcome-on-outcome peer effects are vacuous, either unity or, if the average is defined as leave-out, determined by a generic intraclass correlation coefficient. When pre-determined peer characteristics are introduced as covariates in a model linking individual outcomes with group averages, the question of whether peer effects or social spillovers exist is econometrically identical to that of whether a 2SLS estimator using group dummies to instrument individual characteristics differs from OLS estimates of the effect of these characteristics. The interpretation of results from models that rely solely on chance variation in peer groups is therefore complicated by bias from weak instruments. With systematic variation in group composition, the weak IV issue falls away, but the resulting 2SLS estimates can be expected to exceed the corresponding OLS estimates as a result of measurement error and other reasons unrelated to social effects. Randomized and quasi-experimental research designs that manipulate peer characteristics in a manner unrelated to individual characteristics provide the strongest evidence on the nature of social spillovers. As an empirical matter, designs of this sort have uncovered little in the way of socially significant causal effects.

Explaining Charter School Effectiveness

Abstract: Lottery estimates suggest Massachusetts’ urban charter schools boost achievement well beyond that of traditional urban public schools students, while nonurban charters reduce achievement from a higher baseline. The fact that urban charters are most effective for poor nonwhites and low-baseline achievers contributes to, but does not fully explain, these differences. We therefore link school-level charter impacts to school inputs and practices. The relative efficacy of urban lottery sample charters is accounted for by these schools’ embrace of the No Excuses approach to urban education. In our Massachusetts sample, Non-No-Excuses urban charters are no more effective than nonurban charters. (JEL H75, I21, I28) A growing body of evidence suggests that urban charter schools have the potential to generate impressive achievement gains, especially for minority students living in high-poverty areas. In a series of studies using admissions lotteries to identify causal effects, we looked at the impact of charter attendance in Boston and at a Knowledge is Power Program (KIPP) school in Lynn, Massachusetts (Abdulkadiro ˘ g

Semiparametric Estimates of Monetary Policy Effects: String Theory Revisited

Abstract: We develop flexible semiparametric time series methods for the estimation of the causal effect of monetary policy on macroeconomic aggregates. Our estimator captures the average causal response to discrete policy interventions in a macrodynamic setting, without the need for assumptions about the process generating macroeconomic outcomes. The proposed estimation strategy, based on propensity score weighting, easily accommodates asymmetric and nonlinear responses. Using this estimator, we show that monetary tightening has clear effects on the yield curve and on economic activity. Monetary accommodation, however, appears to generate less pronounced responses from both. Estimates for recent financial crisis years display a similarly dampened response to monetary accommodation.

Long-term consequences of secondary school vouchers : evidence from administrative records in Colombia

Abstract: This brief summarizes the results of a gender impact evaluation study, entitled Long-term consequences of secondary school vouchers : evidence from administrative records in Colombia, conducted during the time period between 1994-2001 in Colombia. The study observed that Colombia's PACES program provided over 125,000 poor children with vouchers that covered the cost of private secondary school. The vouchers were renewable annually conditional on adequate academic progress. Since many vouchers were assigned by lottery, program effects can reliably be assessed by comparing lottery winners and losers. The program results in a substantial gain in both high school graduation rates and achievement as a result of the voucher program: an increase in high school graduation rates of 5-7 percentage points, relative to a base rate of 25-30 percent. The program effects for boys, who generally score lower in this population, were larger than those estimated for girls (especially in math): the voucher program acted to reduce male-female score differentials. Funding for the study derived from National Institute of Health and the World Bank.

The Perils of Peer Effects

Abstract: Individual outcomes are highly correlated with group average outcomes, a fact often interpreted as a causal peer effect. Without covariates, however, outcome-on-outcome peer effects are vacuous, either unity or, if the average is defined as leave-out, determined by a generic intraclass correlation coefficient. When pre-determined peer characteristics are introduced as covariates in a model linking individual outcomes with group averages, the question of whether peer effects or social spillovers exist is econometrically identical to that of whether a 2SLS estimator using group dummies to instrument individual characteristics differs from OLS estimates of the effect of these characteristics. The interpretation of results from models that rely solely on chance variation in peer groups is therefore complicated by bias from weak instruments. With systematic variation in group composition, the weak IV issue falls away, but the resulting 2SLS estimates can be expected to exceed the corresponding OLS estimates as a result of measurement error and other reasons unrelated to social effects. Randomized and quasi-experimental research designs that manipulate peer characteristics in a manner unrelated to individual characteristics provide the strongest evidence on the nature of social spillovers. As an empirical matter, designs of this sort have uncovered little in the way of socially significant causal effects.Institutional subscribers to the NBER working paper series, and residents of developing countries may download this paper without additional charge at www.nber.org.

Semiparametric Estimates of Monetary Policy Effects: String Theory Revisited

Abstract: We develop flexible semiparametric time series methods that are then used to assess the causal effect of monetary policy interventions on macroeconomic aggregates. Our estimator captures the average causal response to discrete policy interventions in a macro-dynamic setting, without the need for assumptions about the process generating macroeconomic outcomes. The proposed procedure, based on propensity score weighting, easily accommodates asymmetric and nonlinear responses. Application of this estimator to the effects of monetary restraint shows the Fed to be an effective inflation fighter. Our estimates of the effects of monetary accommodation, however, suggest the Federal Reserve’s ability to stimulate real economic activity is more modest. Estimates for recent financial crisis years are similar to those for the earlier, pre-crisis period.

Stand and Deliver: Effects of Boston's Charter High School on College Preparation, Entry, and Choice

Abstract: We use admissions lotteries to estimate the effects of attendance at Boston’s charter high schools on college preparation, college attendance, and college choice. Charter attendance increases pass rates on the high-stakes exam required for high school graduation in Massachusetts, with especially large effects on the likelihood of qualifying for a state-sponsored college scholarship. Charter attendance has little effect on the likelihood of taking the SAT, but shifts the distribution of scores rightward, moving students into higher quartiles of the state SAT score distribution. Boston’s charter high schools also increase the likelihood of taking an Advanced Placement (AP) exam, the number of AP exams taken, and scores on AP Calculus tests. Finally, charter attendance induces a substantial shift from two-to four-year institutions, though the effect on overall college enrollment is modest. The increase in four-year enrollment is concentrated among four-year public institutions in Massachusetts. The large gains generated by Boston’s charter high schools are unlikely to be generated by changes in peer composition or other peer effects.

Charter School Demand and Effectiveness: A Boston Update

Abstract: We studied the demand for and effectiveness of charter schools in Boston. We find: • From school year 2009–10 to school year 2012–13, applicants per seat available increased from about two applicants per seat to three applicants per seat in middle school, and from about three per seat to four in high school. • A majority of applicants are offered a seat, mostly off waitlists as opposed to on lottery day. About two-thirds of charter middle school applicants and 40 percent of high school applicants who are offered a seat accept it. • Confirming prior work, we find large positive gains on MCAS for students who attend a Boston charter school, on the order of a 6 to 12 percentage point gain in proficiency per year. Gains are largest for minority students and English language learners, as well as in middle schools, students with lower prior MCAS scores.

Wanna Get Away? RD Identification Away from the Cutoff

Abstract: In the canonical regression discontinuity (RD) design for applicants who face an award or admissions cutoff, causal effects are nonparametrically identified for those near the cutoff. The effect of treatment on inframarginal applicants is also of interest, but identification of such effects requires stronger assumptions than those required for identification at the cutoff. This paper discusses RD identification away from the cutoff. Our identification strategy exploits the availability of dependent variable predictors other than the running variable. Conditional on these predictors, the running variable is assumed to be ignorable. This identification strategy is illustrated with data on applicants to Boston exam schools. Functional-form-based extrapolation generates unsatisfying results in this context, either noisy or not very robust. By contrast, identification based on RD-specific conditional independence assumptions produces reasonably precise and surprisingly robust estimates of the effects of exam school attendance on inframarginal applicants. These estimates suggest that the causal effects of exam school attendance for 9th grade applicants with running variable values well away from admissions cutoffs differ little from those for applicants with values that put them on the margin of acceptance. An extension to fuzzy designs is shown to identify causal effects for compliers away from the cutoff.

Semiparametric Estimates of Monetary Policy Effects: String Theory Revisited

Abstract: We develop a flexible semiparametric time series estimator that is then used to assess the causal effect of monetary policy interventions on macroeconomic aggregates. Our estimator captures the average causal response to discrete policy interventions in a macro-dynamic setting, without the need for assumptions about the process generating macroeconomic outcomes. The proposed procedure, based on propensity score weighting, easily accommodates asymmetric and nonlinear responses. Application of this estimator to the effects of monetary restraint suggest contractionary policy slows real economic activity. By contrast, the Federal Reserve's ability to stimulate real economic activity through monetary expansion appears to be much more limited. Estimates for recent financial crisis years are similar to those for the earlier, pre-crisis period.

Stand and Deliver: Effects of Boston's Charter High Schools on College Preparation, Entry, and Choice

Abstract: We use admissions lotteries to estimate the effects of attendance at Boston's charter high schools on college preparation, college attendance, and college choice. Charter attendance increases pass rates on the high-stakes exam required for high school graduation in Massachusetts, with especially large effects on the likelihood of qualifying for a state-sponsored college scholarship. Charter attendance has little effect on the likelihood of taking the SAT, but shifts the distribution of scores rightward, moving students into higher quartiles of the state SAT score distribution. Boston's charter high schools also increase the likelihood of taking an Advanced Placement (AP) exam, the number of AP exams taken, and scores on AP Calculus tests. Finally, charter attendance induces a substantial shift from two- to four-year institutions, though the effect on overall college enrollment is modest. The increase in four-year enrollment is concentrated among four-year public institutions in Massachusetts. The large gains generated by Boston's charter high schools are unlikely to be generated by changes in peer composition or other peer effects.

In a Small Moment: Cheating and Class Size in Italian Primary Schools ⇤

Abstract: Using a Maimonides-Rule identification strategy based on class-size cutoffs around 25, we document a payoff to smaller classes in Italian primary schools. These gains are driven mainly by schools in Southern Italy, suggesting a substantial return to class size for relatively poor residents of the Mezzogiorno. In addition to low SES, however, the Mezzogiorno is distinguished by pervasive teacher cheating on standardized tests, a fact established by an experiment randomly assigning school monitors. We use Italy’s Maimonides Rule to show that small classes facilitate teacher cheating, providing an alternative explanation for the causal effects of class size on test scores in Southern Italy. This motivates a causal model for achievement with two endogenous variables, class size and proportion cheating. The model is identified by a combination of class size cutoffs and randomly assigned classroom monitors. The resulting estimates suggest that the effects of class size on measured achievement in Italian primary schools are driven entirely by the relationship between class size and teacher cheating. Models that estimate class size and cheating effects jointly generate precise zeros for the former.