To the extent that students benefit from high-achieving peers, tracking will help strong students and hurt weak ones. However, all students may benefit if tracking allows teachers to present material at a more appropriate level. Lower-achieving pupils are particularly likely to benefit from tracking if teachers would otherwise have incentives to teach to the top of the distribution. We propose a simple model nesting these effects. We compare 61 Kenyan schools in which students were randomly assigned to a first grade class with 60 in which students were assigned based on initial achievement. In non-tracking schools, students randomly assigned to academically stronger peers scored higher, consistent with a positive direct effect of academically strong peers. However, compared to their counterparts in non-tracking schools, students in tracking schools scored 0.14 standard deviations higher after 18 months, and this effect persisted one year after the program ended. Furthermore, students at all levels of the distribution benefited from tracking. Students near the median of the pre-test distribution benefited similarly whether assigned to the lower or upper section. A natural interpretation is that the direct effect of high-achieving peers is positive, but that tracking benefited lower-achieving pupils indirectly by allowing teachers to teach at a level more appropriate to them.

Replication data for: Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya

Recent evaluations have begun to use qualitative data in a manner that helps improve the quality and relevance of studies through the inferences that are drawn from them, and their applicability to...

/pdf/mixing-and-matching-using-qualitative-methods-to-improve-2o8xvesijq.pdf

Mixing and matching: using qualitative methods to improve quantitative impact evaluations (IEs) and systematic reviews (SRs) of development outcomes

Development evaluation refers to evaluating projects and programmes in development contexts. Some evaluations are too narrow. Narrow within-discipline impact evaluations are weaker than multidisciplinary, mixed-methods evaluations. A two-step process leads toward profoundly better arguments in assessing the impact of a development intervention. The first step is setting out the arena for discussion, including what the various entities are in the social, political, cultural and natural environment surrounding the chosen problem. The second step is that, once this arena has been declared, the project and triangulation of data can be brought to bear upon logical arguments with clear, transparent reasoning leading to a set of conclusions. In this second step, we do need scientific methods such as peer review, data and so on, but, crucially, the impact evaluation process must not rest upon a single data type, such as survey data. It is dangerous and undesirable to have the entire validity of the conclusions resting upon randomised control trials, or even a mixture of data types. Different contributions to knowledge exist within the evaluation process, including the interaction of people during action research, ethnography, case-study methods, process tracing and qualitative methods. The cement holding my argument together is that multiple logics are used (retroductive, deductive, and inductive, in particular). Deductive mathematics should not dominate the evaluation of an intervention, as randomised controlled trials on their own lend themselves to worrying fallacies about causality. I show this using Boolean fuzzy set logic. An indicator of high-quality development evaluation is the use of multiple logics in a transparent way.

/pdf/bridging-to-action-requires-mixed-methods-not-only-4ehny813dj.pdf

Bridging to Action Requires Mixed Methods, Not Only Randomised Control Trials

Recent advances in the use of randomised control trials to evaluate the effect of development interventions promise to enhance our knowledge of what works and why. A core argument supporting randomised studies is the claim that they have high internal validity. The authors argue that this claim is weak as long as a trial registry of development interventions is not in place. Without a trial registry, the possibilities for data mining, created by analyses of multiple outcomes and subgroups, undermine internal validity. Drawing on experience from evidence-based medicine and recent examples from microfinance, they argue that a trial registry would also enhance external validity and foster innovative research.

Walking the talk: the need for a trial registry for development interventions

Perspectives on the application of CONSORT guidelines to randomised controlled trials in nutrition

The Consolidated Standards of Reporting Trials (CONSORT) checklist was developed to assist investigators, authors, reviewers, and journal editors provide the necessary information to be included in reports of controlled medical trials. We augment the CONSORT reporting by adapting and elaborating the checklist to the context of trials of development interventions. We call this revised list the CONSORT Extensions for Development Effectiveness (CEDE). This checklist emphasises the reporting of underlying theories and descriptions of intervention and comparison conditions, research design, and detailed discussion of the protocol to mitigate the threats to the randomised evaluation design of studies. Systematising, and greater transparency, in the reporting formats for randomised controlled trials (RCTs) will enable the community of evaluators, policy-makers, and programme officers to be privy to the many steps in an RCT implementation, and to better judge the internal and external validity of specific RCTs, b...

CONSORT Extensions for Development Effectiveness: guidelines for the reporting of randomised control trials of social and economic policy interventions in developing countries

Ron Bose

Papers

CONSORT Extensions for Development Effectiveness: guidelines for the reporting of randomised control trials of social and economic policy interventions in developing countries