We investigate conditions sufficient for identification of average treatment effects using instrumental variables. First we show that the existence of valid instruments is not sufficient to identify any meaningful average treatment effect. We then establish that the combination of an instrument and a condition on the relation between the instrument and the participation status is sufficient for identification of a local average treatment effect for those who can be induced to change their participation status by changing the value of the instrument. Finally we derive the probability limit of the standard IV estimator under these conditions. It is seen to be a weighted average of local average treatment effects.

Evolution and Rationality Some Recent Game-Theoretic Results. Identification and Estimation of Local Average Treatment Effects

https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736(07)60112-3.pdf

Strategies to avoid the loss of developmental potential in more than 200 million children in the developing world

The paper critically reviews the methods available for the ex-post counterfactual analysis of programs that are assigned exclusively to individuals, households or locations. The discussion covers both experimental and non- experimental methods (including propensity-score matching, discontinuity designs, double and triple differences and instrumental variables). Two main lessons emerge: Firstly, despite the claims of advocates, no single method dominates; rigorous, policy-relevant evaluations should be open-minded about methodology. Secondly, future efforts to draw more useful lessons from evaluations will call for more policy-relevant measures and deeper explanations of measured impacts than are possible from the classic ("black box") assessment of mean impact.

/pdf/evaluating-anti-poverty-programs-2coaozx9ic.pdf

Evaluating anti-poverty programs

This paper analyzes 12 recent within-study comparisons contrasting causal estimates from a randomized experiment with those from an observational study sharing the same treatment group. The aim is to test whether different causal estimates result when a counterfactual group is formed, either with or without random assignment, and when statistical adjustments for selection are made in the group from which random assignment is absent. We identify three studies comparing experiments and regression-discontinuity (RD) studies. They produce quite comparable causal estimates at points around the RD cutoff. We identify three other studies where the quasi-experiment involves careful intact group matching on the pretest. Despite the logical possibility of hidden bias in this instance, all three cases also reproduce their experimental estimates, especially if the match is geographically local. We then identify two studies where the treatment and nonrandomized comparison groups manifestly differ at pretest but where the selection process into treatment is completely or very plausibly known. Here too, experimental results are recreated. Two of the remaining studies result in correspondent experimental and nonexperimental results under some circumstances but not others, while two others produce different experimental and nonexperimental estimates, though in each case the observational study was poorly designed and analyzed. Such evidence is more promising than what was achieved in past within-study comparisons, most involving job training. Reasons for this difference are discussed. © 2008 by the Association for Public Policy Analysis and Management.

/pdf/three-conditions-under-which-experiments-and-observational-3fnjn00h79.pdf

Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within-study comparisons

There are large differences in health outcomes by socioeconomic status (SES) that cannot be explained fully by traditional arguments, such as access to care or poor health behaviors. We consider a different explanation - better self-management of disease by the more educated. We examine differences by education in treatment adherence among patients with two illnesses, diabetes and HIV, and then assess the subsequent impact of differential adherence on health status. One unique component of this research is that for diabetes we combine two different surveys - one cohort study and one randomized clinical trial - that are usually used exclusively by either biomedical or/and social scientists separately. For both illnesses, we find significant effects of adherence that are much stronger among patients with high SES. After controlling for other factors, more educated HIV+ patients are more likely to adhere to therapy, and this adherence made them experience improvements in their self-reported general health. Similarly, among diabetics, the less educated were much more likely to switch treatment, which led to worsening general health. In the randomized trial setting, intensive treatment regimens that compensated for poor adherence led to better improvements in glycemic control for the less educated. Among two distinct chronic illnesses, the ability to maintain a better health regimen is an important independent determinant of subsequent health outcomes. This finding is robust across clinical trial and population-based settings. Because this ability varies by schooling, self-maintenance is an important reason for the steep SES gradient in health outcomes.

Can Patient Self-Management Help Explain the SES Health Gradient?

In this working paper the authors present evidence on the reliability of propensity score matching to estimate the bias associated with the effect of treatment on the treated, exploiting the availability of experimental data from a Mexican antipoverty program (PROGRESA: Programa de Educacion, Salud y Alimentacion). The data comes from several outcomes such as food expenditure and child schooling and labor. The methodology compares the results of the experimental impact estimator with those using matched samples drawn from a (non-experimental) national survey carried out to measure household income and expenditures. The results show that simple-cross sectional matching produces significant bias for outcomes measured in different ways. Results are more positive for outcomes measured similarly across survey instruments, but even in this case there are indications of bias depending on sample and matching method.

An Assessment of Propensity Score Matching as a Nonexperimental Impact Estimator Evidence from Mexico's PROGRESA Program

Objective. To examine the determinants of the utilization of health care services among the population of older adults in Mexico. Three types of health care services are analyzed: preventive care, visits to the doctor, and hospitalizations. Material and Methods. Data was used from the 2001 Mexican Health and Aging Study (MHAS/ENASEM) and estimates were made using multivariate probit regression methods. Results. Socioeconomic factors, health conditions reported by the individuals, and the availability of health insurance are significant determinants of the differential use of services by older adults. Conclusion. Specific health conditions are important determinants of use of the various types of health care services. For all three types, however, the availability of health insurance is an enabling factor of health care use. Older age is associated with greater propensity to use health care services but its effect is small when controlling for health conditions.

/pdf/health-care-utilization-among-older-mexicans-health-and-4dvm6agcq4.pdf

Health care utilization among older mexicans: health and socioeconomic inequalities

The "Youth Labor Training Program" (Programa de Capacitacion Laboral Juvenil - PROJoven) is an ongoing job-training program created in 1996 by the Ministry of Labor (Ministerio de Trabajo y Promocion del Empleo) in response to the precarious conditions of youth in the Peruvian labor market. The goals of the program are to improve employment opportunities of youth in poverty and to promote competition and higher quality of services in the vocational training system. In this paper, we present the main results of an evaluation of PROJoven impacts on program beneficiaries in terms of employment status (employed, paid employment and formal jobs), earnings (monthly and hourly) and weekly hours of work. The authors also provide an institutional analysis of the program, documenting the origins and rationale of the intervention, we explore how and why PROJoven was designed, and the political and economic environment at the time of its inception and afterwards.

An evaluation of the peruvian "youth labor training program" - projoven

This paper estimates the returns to cognitive and socio-emotional (“non-cognitive”) skills in a developing country setting using a recent labor force survey of the working-age (14-50) urban population in Peru designed to measure cognitive skills (Peabody Picture Vocabulary, verbal fluency, working memory, and numeracy/problem-solving) and personality traits to proxy for socio-emotional skills (Big-five Factors; Grit - Duckworth et al 2009), and also collects data on some instrumental variables as skills are measured contemporaneously to schooling and earnings. We corroborate findings from developed countries that both types of skills are important correlates of earnings. After correcting for the potential endogeneity of measured skills vis-a-vis schooling, the findings confirm that both socio-emotional and cognitive skills are equally valued in the Peruvian labor market. A one standard deviation change in an overall cognitive skill measure and in the perseverance facet of Grit each generates a 9% increase on average earnings, conditional on schooling. The effect size of an increase in years of schooling (about 3 years) is a 15% increase in earnings, conditional on skills. The returns to other socio-emotional skills vary across dimensions of personality: 5% higher earnings for emotional stability while 8% lower earnings for agreeableness.

/pdf/does-perseverance-pay-as-much-as-being-smart-the-returns-to-ljdx1my4ul.pdf

Does Perseverance Pay as Much as Being Smart?: The Returns to Cognitive and Non-cognitive Skills in urban Peru 1

One of the major educational problems in Peru is the low level of efficiency attained in elementary schools. In this paper we analyze the impact of two types of preschool programs,  Centros de Educacion Inicial (CEI) and  Programas No Escolarizados de Educacion Inicial (PRONOEI). We carried out a retrospective survery in nine public schools in Lima that had a high repetition rate. Among children with preschool education at CEI the probability of getting a higher achievement increases in 25% in language and 22% in mathematics. Among children with preschool education at PRONOEI, the probability of obtaining a higher achievement increases in 15% in language and 17% in mathematics.

/pdf/impacto-de-la-educacion-inicial-en-el-rendimiento-en-primer-21gtmfoedd.pdf

Juan José Díaz

Papers

An Assessment of Propensity Score Matching as a Nonexperimental Impact Estimator Evidence from Mexico's PROGRESA Program

Health care utilization among older mexicans: health and socioeconomic inequalities

An evaluation of the peruvian "youth labor training program" - projoven

Does Perseverance Pay as Much as Being Smart?: The Returns to Cognitive and Non-cognitive Skills in urban Peru 1

Impacto de la educación inicial en el rendimiento en primer grado de primaria en escuelas públicas urbanas de Lima