Showing papers by "Joshua D. Angrist published in 2017"

Economic Research Evolves: Fields and Styles

Abstract: We examine the evolution of economics research using a machine-learning-based classification of publications into fields and styles The changing field distribution of publications would not seem to favor empirical papers But economics' empirical shift is a within-field phenomenon; even fields that traditionally emphasize theory have gotten more empirical Empirical work has also come to be more cited than theoretical work The citation shift is sharpened when citations are weighted by journal importance Regression analyses of citations per paper show empirical publications reaching citation parity with theoretical publications around 2000 Within fields and journals, however, empirical work is now cited more

Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation

Abstract: A growing number of school districts use centralized assignment mechanisms to allocate school seats in a manner that reflects student preferences and school priorities. Many of these assignment schemes use lotteries to ration seats when schools are oversubscribed. The resulting random assignment opens the door to credible quasi-experimental research designs for the evaluation of school effectiveness. Yet the question of how best to separate the lottery-generated randomization integral to such designs from non-random preferences and priorities remains open. This paper develops easily-implemented empirical strategies that fully exploit the random assignment embedded in a wide class of mechanisms, while also revealing why seats are randomized at one school but not another. We use these methods to evaluate charter schools in Denver, one of a growing number of districts that combine charter and traditional public schools in a unified assignment system. The resulting estimates show large achievement gains from charter school attendance. Our approach generates efficiency gains over ad hoc methods, such as those that focus on schools ranked first, while also identifying a more representative average causal effect. We also show how to use centralized assignment mechanisms to identify causal effects in models with multiple school sectors.

Uber vs. Taxi: A Driver’s Eye View

Abstract: Ride-hailing drivers pay a proportion of their fares to the ride-hailing platform operator, a commission-based compensation model used by many internet-mediated service providers. To Uber drivers, this commission is known as the Uber fee. By contrast, traditional taxi drivers in most US cities make a fixed payment independent of their earnings, usually a weekly or daily medallion lease, but keep every fare dollar net of expenses. We assess these compensation models from a driver’s point of view using an experiment that offered random samples of Boston Uber drivers opportunities to lease a virtual taxi medallion that eliminates the Uber fee. Some drivers were offered a negative fee. Drivers’ labor supply response to our offers reveals a large intertemporal substitution elasticity, on the order of 1.2. At the same time, our virtual lease program was under-subscribed: many drivers who would have benefitted from buying an inexpensive lease chose to opt out. We use these results to compute the average compensation required to make drivers indifferent between ride-hailing and a traditional taxi compensation contract. The results suggest that ride-hailing drivers gain considerably from the opportunity to drive without leasing.

Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation

Abstract: A growing number of school districts use centralized assignment mechanisms to allocate school seats in a manner that reflects student preferences and school priorities. Many of these assignment schemes use lotteries to ration seats when schools are oversubscribed. The resulting random assignment opens the door to credible quasi-experimental research designs for the evaluation of school effectiveness. Yet the question of how best to separate the lottery-generated randomization integral to such designs from non-random preferences and priorities remains open. This paper develops easily-implemented empirical strategies that fully exploit the random assignment embedded in a wide class of mechanisms, while also revealing why seats are randomized at one school but not another. We use these methods to evaluate charter schools in Denver, one of a growing number of districts that combine charter and traditional public schools in a unified assignment system. The resulting estimates show large achievement gains from charter school attendance. Our approach generates efficiency gains over ad hoc methods, such as those that focus on schools ranked first, while also identifying a more representative average causal effect. We also show how to use centralized assignment mechanisms to identify causal effects in models with multiple school sectors.

Leveraging Lotteries for School Value-Added: Testing and Estimation

Abstract: Conventional value-added models (VAMs) compare average test scores across schools after regression-adjusting for students’ demographic characteristics and previous scores. This article tests for VAM bias using a procedure that asks whether VAM estimates accurately predict the achievement consequences of random assignment to specific schools. Test results from admissions lotteries in Boston suggest conventional VAM estimates are biased, a finding that motivates the development of a hierarchical model describing the joint distribution of school value-added, bias, and lottery compliance. We use this model to assess the substantive importance of bias in conventional VAM estimates and to construct hybrid value-added estimates that optimally combine ordinary least squares and lottery-based estimates of VAM parameters. The hybrid estimation strategy provides a general recipe for combining nonexperimental and quasi-experimental estimates. While still biased, hybrid school value-added estimates have lower mean squared error than conventional VAM estimates. Simulations calibrated to the Boston data show that, bias notwithstanding, policy decisions based on conventional VAMs that control for lagged achievement are likely to generate substantial achievement gains. Hybrid estimates that incorporate lotteries yield further gains.

Undergraduate Econometrics Instruction: Through Our Classes, Darkly

Abstract: The past half-century has seen economic research become increasingly empirical, while the nature of empirical economic research has also changed. In the 1960s and 1970s, an empirical economist’s typical mission was to “explain” economic variables like wages or GDP growth. Applied econometrics has since evolved to prioritize the estimation of specific causal effects and empirical policy analysis over general models of outcome determination. Yet econometric instruction remains mostly abstract, focusing on the search for “true models” and technical concerns associated with classical regression assumptions. Questions of research design and causality still take a back seat in the classroom, in spite of having risen to the top of the modern empirical agenda. This essay traces the divergent development of econometric teaching and empirical practice, arguing for a pedagogical paradigm shift.

In a Small Moment: Class Size and Moral Hazard in the Italian Mezzogiorno

Abstract: Instrumental variables (IV ) estimates show strong class-size effects in Southern Italy. But Italy's Mezzogiorno is distinguished by manipulation of standardized test scores as well as by economic disadvantage. IV estimates suggest small classes increase manipulation. We argue that score manipulation is a consequence of teacher shirking. IV estimates of a causal model for achievement as a function of class size and score manipulation show that class-size effects on measured achievement are driven entirely by the relationship between class size and manipulation. These results illustrate how consequential score manipulation can arise even in assessment systems with few accountability concerns. (JEL D82, H75, I21, I26, I28, J24, R23)

Undergraduate Econometrics Instruction: Through Our Classes, Darkly

Abstract: The past half-century has seen economic research become increasingly empirical, while the nature of empirical economic research has also changed. In the 1960s and 1970s, an empirical economist's typical mission was to "explain" economic variables like wages or GDP growth. Applied econometrics has since evolved to prioritize the estimation of specific causal effects and empirical policy analysis over general models of outcome determination. Yet econometric instruction remains mostly abstract, focusing on the search for "true models" and technical concerns associated with classical regression assumptions. Questions of research design and causality still take a back seat in the classroom, in spite of having risen to the top of the modern empirical agenda. This essay traces the divergent development of econometric teaching and empirical practice, arguing for a pedagogical paradigm shift.

Regression Discontinuity in Serial Dictatorship: Achievement Effects at Chicago's Exam Schools

Abstract: Many school and college admission systems use centralized mechanisms to allocate seats based on applicant preferences and school priorities. When tie-breaking uses non-randomly assigned cr...

Inside Job or Deep Impact? Using Extramural Citations to Assess Economic Scholarship

Abstract: Does academic economic research produce material of scientific value, or are academic economists writing only for clients and peers? Is economics scholarship uniquely insular? We address these questions by quantifying interactions between economics and other disciplines. Changes in the impact of economic scholarship are measured here by the way other disciplines cite us. We document a clear rise in the extramural influence of economic research, while also showing that economics is increasingly likely to reference other social sciences. A breakdown of extramural citations by economics fields shows broad field impact. Differentiating between theoretical and empirical papers classified using machine learning, we see that much of the rise in economics’ extramural influence reflects growth in citations to empirical work. This parallels a growing share of empirical cites within economics. At the same time, the disciplines of computer science and operations research are mostly influenced by economic theory.

Impact Evaluation in Matching Markets with General Tie-Breaking

Abstract: Many centralized matching schemes incorporate a mix of random lottery and non-lottery tie-breaking. A leading example is the New York City public school district, which uses criteria like test scores and interviews to generate applicant rankings for some schools, combined with lottery tie-breaking at other schools. We develop methods that identify causal effects of assignment in such settings. Our approach generalizes the standard regression discontinuity design to allow for many running variables and treatments, some of which are randomly assigned. We show that lottery variation generates assignment risk at non-lottery programs for applicants away from non-lottery cutoffs, while non-lottery variation randomizes applicants near cutoffs regardless of lottery risk. These methods are applied to evaluate New York City’s school progress assessments, which give schools letter grades as a summary measure of quality. Our estimates reveal that although Grade A schools boost achievement, these gains emerge only for students who attend lottery schools. Attendance at a coveted Grade A screened school, including some of the highest performing in the district, generates no measurable effects. Evaluation methods that fail to take advantage of both lottery and non-lottery variation miss this difference in impact.

Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation

Abstract: A growing number of school districts use centralized assignment mechanisms to allocate school seats in a manner that reflects student preferences and school priorities. Many of these assignment schemes use lotteries to ration seats when schools are oversubscribed. The resulting random assignment opens the door to credible quasi-experimental research designs for the evaluation of school effectiveness. Yet the question of how best to separate the lottery-generated variation integral to such designs from non-random preferences and priorities remains open. This paper develops easily-implemented empirical strategies that fully exploit the random assignment embedded in the widely-used deferred acceptance mechanism and its variants. We use these methods to evaluate charter schools in Denver, one of a growing number of districts that integrate charter and traditional public schools in a unified assignment system. The resulting estimates show large achievement gains from charter school attendance. Our approach expands the scope for impact evaluation by maximizing the number of students and schools that can be studied using random assignment. We also show how to use DA to identify causal effects in models with multiple school sectors.Institutional subscribers to the NBER working paper series, and residents of developing countries may download this paper without additional charge at www.nber.org.

Maimonides Rule Redux

Abstract: We use the discontinuous function of enrollment known as Maimonides Rule as an instrument for class size in large Israeli samples from 2002-2011. As in the 1991 data analyzed by Angrist and Lavy (1999), Maimonides Rule still has a strong first stage. In contrast with the earlier Israeli estimates, however, Maimonides-based instrumental variables estimates using more recent data show no effect of class size on achievement. The new data also reveal substantial enrollment sorting near Maimonides cutoffs, with too many schools having enrollment values that just barely produce an extra class. A modified rule that uses data on students’ birthdays to compute statutory enrollment in the absence of enrollment manipulation also generates a precisely estimated zero. In older data, the original Maimonides Rule is unrelated to socioeconomic characteristics, while in more recent data, the original rule is unrelated to socioeconomic characteristics conditional on a few controls. Enrollment manipulation therefore appears to be innocuous: neither the original negative effects nor the recent data zeros seem likely to be manipulation artifacts.

Uber vs. Taxi: A Driver’s Eye View

Abstract: Ride-hailing drivers pay a proportion of their fares to the ride-hailing platform operator, a commission-based compensation model used by many internet-mediated service providers. To Uber drivers, this commission is known as the Uber fee. By contrast, traditional taxi drivers in most US cities make a fixed payment independent of their earnings, usually a weekly or daily medallion lease, but keep every fare dollar net of expenses. We assess these compensation models from a driver’s point of view using an experiment that offered random samples of Boston Uber drivers opportunities to lease a virtual taxi medallion that eliminates the Uber fee. Some drivers were offered a negative fee. Drivers’ labor supply response to our offers reveals a large intertemporal substitution elasticity, on the order of 1.2. At the same time, our virtual lease program was under-subscribed: many drivers who would have benefitted from buying an inexpensive lease chose to opt out. We use these results to compute the average compensation required to make drivers indifferent between ride-hailing and a traditional taxi compensation contract. The results suggest that ride-hailing drivers gain considerably from the opportunity to drive without leasing.

Maimonides Rule Redux

Abstract: We use the discontinuous function of enrollment known as Maimonides Rule as an instrument for class size in large Israeli samples from 2002-2011. As in the 1991 data analyzed by Angrist and Lavy (1999), Maimonides Rule still has a strong first stage. In contrast with the earlier Israeli estimates, however, Maimonides-based instrumental variables estimates using more recent data show no effect of class size on achievement. The new data also reveal substantial enrollment sorting near Maimonides cutoffs, with too many schools having enrollment values that just barely produce an extra class. A modified rule that uses data on students’ birthdays to compute statutory enrollment in the absence of enrollment manipulation also generates a precisely estimated zero. In older data, the original Maimonides Rule is unrelated to socioeconomic characteristics, while in more recent data, the original rule is unrelated to socioeconomic characteristics conditional on a few controls. Enrollment manipulation therefore appears to be innocuous: neither the original negative effects nor the recent data zeros seem likely to be manipulation artifacts.

Uber vs. Taxi: A Driver's Eye View

Abstract: Ride-hailing drivers pay a proportion of their fares to the ride-hailing platform operator, a commission-based compensation model used by many internet-mediated service providers. To Uber drivers, this commission is known as the Uber fee. By contrast, traditional taxi drivers in most US cities make a fixed payment independent of their earnings, usually a weekly or daily medallion lease, but keep every fare dollar net of expenses. We assess these compensation models from a driver’s point of view using an experiment that offered random samples of Boston Uber drivers opportunities to lease a virtual taxi medallion that eliminates the Uber fee. Some drivers were offered a negative fee. Drivers’ labor supply response to our offers reveals a large intertemporal substitution elasticity, on the order of 1.2. At the same time, our virtual lease program was under-subscribed: many drivers who would have benefitted from buying an inexpensive lease chose to opt out. We use these results to compute the average compensation required to make drivers indifferent between ride-hailing and a traditional taxi compensation contract. The results suggest that ride-hailing drivers gain considerably from the opportunity to drive without leasing.

Impact Evaluation in Matching Markets with General Tie-Breaking

Abstract: Many centralized matching schemes incorporate a mix of random lottery and non-lottery tie-breaking. A leading example is the New York City public school district, which uses criteria like test scores and interviews to generate applicant rankings for some schools, combined with lottery tie-breaking at other schools. We develop methods that identify causal effects of assignment in such settings. Our approach generalizes the standard regression discontinuity design to allow for many running variables and treatments, some of which are randomly assigned. We show that lottery variation generates assignment risk at non-lottery programs for applicants away from non-lottery cutoffs, while non-lottery variation randomizes applicants near cutoffs regardless of lottery risk. These methods are applied to evaluate New York City’s school progress assessments, which give schools letter grades as a summary measure of quality. Our estimates reveal that although Grade A schools boost achievement, these gains emerge only for students who attend lottery schools. Attendance at a coveted Grade A screened school, including some of the highest performing in the district, generates no measurable effects. Evaluation methods that fail to take advantage of both lottery and non-lottery variation miss this difference in impact.