Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact.
01 Feb 1999-International Journal of Epidemiology (Oxford University Press)-Vol. 28, Iss: 1, pp 10-18
TL;DR: It is argued that the answer to this question is essential for choosing an appropriate evaluation design, drawing upon examples from the fields of health and nutrition, and a framework is proposed for deciding upon appropriate evaluation designs.
Abstract: The question of why to evaluate a programme is seldom discussed in the literature. The present paper argues that the answer to this question is essential for choosing an appropriate evaluation design. The discussion is centered on summative evaluations of large-scale programme effectiveness, drawing upon examples from the fields of health and nutrition but the findings may be applicable to other subject areas. The main objective of an evaluation is to influence decisions. How complex and precise the evaluation must be depends on who the decision maker is and on what types of decisions will be taken as a consequence of the findings. Different decision makers demand not only different types of information but also vary in their requirements of how informative and precise the findings must be. Both complex and simple evaluations, however, should be equally rigorous in relating the design to the decisions. Based on the types of decisions that may be taken, a framework is proposed for deciding upon appropriate evaluation designs. Its first axis concerns the indicators of interest, whether these refer to provision or utilization of services, coverage or impact measures. The second axis refers to the type of inference to be made, whether this is a statement of adequacy, plausibility or probability. In addition to the above framework, other factors affect the choice of an evaluation design, including the efficacy of the intervention, the field of knowledge, timing and costs. Regarding the latter, decision makers should be made aware that evaluation costs increase rapidly with complexity so that often a compromise must be reached. Examples are given of how to use the two classification axes, as well as these additional factors, for helping decision makers and evaluators translate the need for evaluation--the why--into the appropriate design--the how.
Citations
More filters
Journal Article•
1,631 citations
Book•
01 Jan 2006
TL;DR: Food fortification has the dual advantage of being able to deliver nutrients to large segments of the population without requiring radical changes in food consumption patterns.
Abstract: Food and Agricultural Organization of the United Nations PA N I S F I A T G u id e in e s o n fo o d fo r tific atio n w th m ic r o n u tr ie n ts Interest in micronutrient malnutrition has increased greatly over the last few years. One of the main reasons is the realization that micronutrient malnutrition contributes substantially to the global burden of disease. Furthermore, although micronutrient malnutrition is more frequent and severe in the developing world and among disadvantaged populations, it also represents a public health problem in some industrialized countries. Measures to correct micronutrient deficiencies aim at ensuring consumption of a balanced diet that is adequate in every nutrient. Unfortunately, this is far from being achieved everywhere since it requires universal access to adequate food and appropriate dietary habits. Food fortification has the dual advantage of being able to deliver nutrients to large segments of the population without requiring radical changes in food consumption patterns.
1,338 citations
Cites background from "Evaluation designs for adequacy, pl..."
...200 1 An efficacy trial is one that applies an intervention under controlled conditions to determine the magnitude of effect that can be achieved under the best possible circumstances (344)....
[...]
...(344) have devised a useful way of classifying the various approaches to evaluating public health interventions....
[...]
Journal Article•
TL;DR: This chapter discusses zinc Nutrition, which focuses on dietary requirements and recommended intakes for zinc, and causes of zinc deficiency and groups at high risk.
Abstract: Chapter 1: Overview of Zinc Nutrition ............................................................................................................... S99 1.1 Biological functions of zinc ............................................................................................................................. S99 1.2 Tissue zinc distribution and reserves .............................................................................................................. S99 1.3 Zinc metabolism ........................................................................................................................................... S100 1.4 Importance of zinc for human health........................................................................................................... S101 1.5 Human zinc requirements............................................................................................................................. S105 1.5.1 Adult men ............................................................................................................................................. S106 1.5.2 Adult women......................................................................................................................................... S109 1.5.3 Children ................................................................................................................................................ S110 1.5.4 Pregnancy.............................................................................................................................................. S111 1.5.5 Lactation ............................................................................................................................................... S112 1.6 Dietary sources of zinc and suggested revisions of Recommended Daily Intakes .................................... S112 1.6.1 Dietary sources of zinc and factors affecting the proportion of zinc available for absorption ........ S112 1.6.2 Revised estimates of dietary requirements and recommended intakes for zinc ............................... S114 1.7 Zinc toxicity.................................................................................................................................................... S118 1.8 Causes of zinc deficiency and groups at high risk ....................................................................................... S121 1.9 Summary ........................................................................................................................................................ S123
1,280 citations
TL;DR: There is an urgent need to develop evaluation standards and protocols for use in circumstances where RCTs are not appropriate, and both the internal and external validity of RCT findings can be greatly enhanced by observational studies using adequacy or plausibility designs.
Abstract: Randomized controlled trials (RCTs) are essential for evaluating the efficacy of clinical interventions, where the causal chain between the agent and the outcome is relatively short and simple and where results may be safely extrapolated to other settings. However, causal chains in public health interventions are complex, making RCT results subject to effect modification in different populations. Both the internal and external validity of RCT findings can be greatly enhanced by observational studies using adequacy or plausibility designs. For evaluating large-scale interventions, studies with plausibility designs are often the only feasible option and may provide valid evidence of impact. There is an urgent need to develop evaluation standards and protocols for use in circumstances where RCTs are not appropriate.
1,032 citations
TL;DR: A combined microfinance and training intervention can lead to reductions in levels of intimate-partner violence in programme participants, and social and economic development interventions have the potential to alter risk environments for HIV and intimacy violence in southern Africa.
878 citations
References
More filters
Book•
01 Jan 1979
11,977 citations
Book•
01 Jan 1980TL;DR: The Classical Inference Approach for the General Linear Model, Statistical Decision Theory and Biased Estimation, and the Bayesian Approach to Inference are reviewed.
Abstract: SAMPLING THEORY AND BAYESIAN APPROACHES TO INFERENCE. The Classical Inference Approach for the General Linear Model. Statistical Decision Theory and Biased Estimation. The Bayesian Approach to Inference. INFERENCE IN GENERAL STATISTICAL MODELS AND TIME SERIES. Some Asymptotic Theory and Other General Results for the Linear Statistical Model. Nonlinear Statistical Models. Time Series. DYNAMIC SPECIFICATIONS. Autocorrelation. Finite Distributed Lags. Infinite Distributed Lags. SOME ALTERNATIVE COVARIANCE STRUCTURES. Heteroskedasticity. Disturbance--Related Sets of Regression Equations. Inference in Models that Combine Time Series and Cross--Sectional Data. INFERENCE IN SIMULTANEOUS EQUATION MODELS. Specification and Identification in Simultaneous Equation Models. Estimation and Inference in a System of Simultaneous Equations. Multiple Time Series and Systems of Dynamic Simultaneous Equations. FURTHER MODEL EXTENSIONS. Unobservable Variables. Qualitative and Limited Dependent Variable Models. Varying and Random Coefficient Models. Non--Normal Disturbances. On Selecting the Set of Aggressors. Multicollinearity. Appendices.
4,469 citations
Book•
21 Jan 1982
TL;DR: Case-control studies, often called 'retrospective' studies, provide a research method for investigating factors that may prevent or cause disease.
Abstract: Case-control studies, often called 'retrospective' studies, provide a research method for investigating factors that may prevent or cause disease. Basically the method involves comparison of patients (cases) with a group of controls that consists of persons who are free of the disease under study.
2,495 citations
Journal Article•
1,631 citations
TL;DR: The view is widely held that experimental methods (randomised controlled trials) are the "gold standard" for evaluation and that observational methods have little or no value, but this ignores the limitations of randomised trials.
Abstract: The view is widely held that experimental methods (randomised controlled trials) are the “gold standard” for evaluation and that observational methods (cohort and case control studies) have little or no value. This ignores the limitations of randomised trials, which may prove unnecessary, inappropriate, impossible, or inadequate. Many of the problems of conducting randomised trials could often, in theory, be overcome, but the practical implications for researchers and funding bodies mean that this is often not possible. The false conflict between those who advocate randomised trials in all situations and those who believe observational data provide sufficient evidence needs to be replaced with mutual recognition of the complementary roles of the two approaches. Researchers should be united in their quest for scientific rigour in evaluation, regardless of the method used.
Despite the essential role of observational methods in shedding light on the effectiveness of many aspects of health care, some scientists believe such methods have little or even nothing to contribute. In his summing up at a major conference held in 1993, the eminent medical epidemiologist Richard Doll concluded that observational methods “provide no useful means of assessing the value of a therapy.”1 The widely held view that experimental methods (randomised controlled trials) are the “gold standard” for evaluation has led to the denigration of non-experimental methods, to the extent that research funding bodies and journal editors automatically reject them. I suggest that such attitudes limit our potential to evaluate health care and hence to improve the scientific basis of how to treat individuals and how to organise services.
My main contention is that those who are opposed to the use of observational methods have assumed that they represent an alternative to experimentation rather than a set of complementary approaches. This in turn stems from a misguided notion that everything …
1,562 citations