Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement

Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement [in Spanish]

Systematic reviews and meta-analyses are essential to summarize evidence relating to efficacy and safety of health care interventions accurately and reliably. The clarity and transparency of these reports, however, is not optimal. Poor reporting of systematic reviews diminishes their value to clinicians, policy makers, and other users.

Since the development of the QUOROM (QUality Of Reporting Of Meta-analysis) Statement—a reporting guideline published in 1999—there have been several conceptual, methodological, and practical advances regarding the conduct and reporting of systematic reviews and meta-analyses. Also, reviews of published systematic reviews have found that key information about these studies is often poorly reported. Realizing these issues, an international group that included experienced authors and methodologists developed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) as an evolution of the original QUOROM guideline for systematic reviews and meta-analyses of evaluations of health care interventions.

The PRISMA Statement consists of a 27-item checklist and a four-phase flow diagram. The checklist includes items deemed essential for transparent reporting of a systematic review. In this Explanation and Elaboration document, we explain the meaning and rationale for each checklist item. For each item, we include an example of good reporting and, where possible, references to relevant empirical studies and methodological literature. The PRISMA Statement, this document, and the associated Web site (http://www.prisma-statement.org/) should be helpful resources to improve reporting of systematic reviews and meta-analyses.

/pdf/the-prisma-statement-for-reporting-systematic-reviews-and-3j53e5oe33.pdf

The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration

Moher and colleagues introduce PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), an update of the QUOROM guidelines for reporting systematic reviews and meta-analyses. Us...

/pdf/preferred-reporting-items-for-systematic-reviews-and-meta-5c1h0wv2gs.pdf

Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement

Flaws in the design, conduct, analysis, and reporting of randomised trials can cause the effect of an intervention to be underestimated or overestimated. The Cochrane Collaboration’s tool for assessing risk of bias aims to make the process clearer and more accurate

https://www.bmj.com/content/bmj/343/bmj.d5928.full.pdf

The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials

/pdf/cochrane-handbook-for-systematic-reviews-of-interventions-32sv80ql7o.pdf

Cochrane Handbook for Systematic Reviews of Interventions

Background: A large number of evaluation theorists have debated the issue of objectivity and bias in evaluation over the last five decades. In particular, the degree to which distance from the evaluand enhances the validity and reliability of evaluation findings has been a prominent topic of discussion. Purpose: This article has two primary objectives. First, it intends to present some of the positivist and post-positivist theories on distance that have dominated the evaluation discourse since the late 1960’s, by also showing the limitations of their respective assumptions. Second, it describes a more recent evaluation theory on distance that is proving to help evaluators rapport with their evaluand more effectively, especially in the case of complex programs involving a large variety of stakeholders. Setting: Not applicable. Subjects: Not applicable. Research Design: Not applicable. Data Collection and Analysis: The paper is the result of both a desk review and a series of interviews with some major evaluation theorists aimed to compare and contrast some of the most relevant ideas on distance in evaluation expressed over the last five decades. Findings: The author shows that evaluators today are still facing the dilemma of whether they should seek proximity to or distance from the evaluand. However, the author identifies an increasingly popular evaluation approach (herewith referred as the pluralist approach as opposed to the niche approach) that promises to overcome the issue of distance in evaluation more successfully than any other earlier theory. The author dismisses the idea of absolute distance, predicated by both Scriven and Campbell. In doing so, he also shows that evaluators who are closer to the evaluand and the context contiguous to it tend to have a deeper understanding of the issues at stake and therefore enhance the overall quality of their evaluation. In addition, the author acknowledges that evaluators today have a new important role to play vis-a-vis their evaluand: mediating stakeholders’ competing values and agendas for the sake of equity and social justice. Conclusions: The author concludes that the proximity to the evaluand and the integration of multiple perspectives in an evaluation represent two of evaluation’s most enriching—rather than detrimental—factors. The author also asserts that a truly participatory approach can effectively coexist with advocacy, so long as evaluators are able to clarify their stances vis-a-vis the social, economic, and political issues associated with their evaluand. Keywords: distance, bias, transformative evaluation, advocacy, pluralist and niche approaches

/pdf/being-blind-in-a-world-of-multiple-perspectives-the-3ecflbuf0i.pdf

Being Blind in a World of Multiple Perspectives: The Evaluator’s Dilemma Between the Hope of Becoming a Team Player and the Fear of Becoming a Critical Friend with No Friends

Background: Evaluation is sometimes viewed as a professional practice rather than a discipline corresponding to a well defined set of theories. However, Shadish, Cook and Leviton (1991) were able to demonstrate that evaluators’ work does have theoretical foundations. In particular, the authors identified five main elements for evaluation theory and described the contribution made to each of them by seven of the most influential scholars in the field over the last five decades. Purpose: This paper intends to further the discussion on evaluation theory, by examining some of the contributions made by Donna Mertens. The main focus of the paper is on her innovative ideas on each of the five main elements of evaluation theory. Setting: Not applicable. Subjects: Not applicable. Research Design: Not applicable. Data Collection and Analysis: The paper is the result of both a desk review of her work on transformative evaluation, inclusiveness, and social justice, and a phone interview with her. For the sake of accuracy, the text of the interview and the corresponding analysis were submitted to Mertens for review prior to publication. Findings: The author demonstrates how Mertens has incorporated the five principles into her own work and practice. According to Mertens, evaluators need first to be cognizant of the plurality of values held by the communities where they work and second to let those very same values guide the design, implementation, and use of evaluation. Based on her belief that knowledge is not neutral but is influenced by human interests, Mertens calls upon evaluators to play a more active role in the construction of new and “socially compensatory” knowledge. In doing so, Mertens particularly stresses how evaluation practice could address and challenge the social, cultural and economic inequities perpetuated by the status quo. Conclusions: Pursuing inclusiveness and social justice in evaluation is possible. Transformative evaluation is capable of altering the status quo. Transformative evaluation is instrumental in bringing about not only social change but also social transformation. This article also shows that the debate on evaluation’s main theoretical foundations is still relevant. Such theoretical categories are especially beneficial in that provide common ground for understanding. Keywords: transformative evaluation, social justice, cultural competence, inclusiveness, valuing, social programming, knowledge, practice, use

/pdf/inclusiveness-and-social-justice-in-evaluation-can-the-4t8607b37c.pdf

Inclusiveness and Social Justice in Evaluation: Can the Transformative Agenda Really Alter the Status Quo? A Conversation with Donna M. Mertens

Background: Evaluation is sometimes viewed as a professional practice rather than a discipline corresponding to a well defined set of theories. However, Shadish, Cook and Leviton (1991) were able to demonstrate that evaluators’ work does have theoretical foundations. In particular, the authors identified five main elements for evaluation theory and described the contribution made to each of them by seven of the most influential scholars in the field over the last five decades. Purpose: This paper intends to further the discussion on evaluation theory, by examining some of the contributions made Jennifer Greene, one of the most influential figures in contemporary evaluation. The paper mainly focuses on Greene’s innovative ideas on each of the five main elements of evaluation theory.  Setting: Not applicable. Subjects: Not applicable. Research Design: Not applicable. Data Collection and Analysis: The paper is the result of both a desk review of Jennifer Greene’s most relevant work on bias, objectivity, and advocacy in evaluation, and a phone interview with her. For the sake of accuracy, the text of the interview and the corresponding analysis were submitted to Greene for review prior to publication. Findings: The author shows how Greene has incorporated the five principles into her own work and how this eventually influenced her practice. Greene sorts evaluation approaches based on the interests they serve and the values they promote. However, she seems to have developed her theory on valuing further over the years and today she claims that evaluators should never privilege anyone’s specific side in the course of their assignments. Second, as knowledge is mediated by evaluators’ perceptual frames, Greene believes that an unfiltered (objective) view of the world is not feasible. Third, she views evaluation as a force for democratizing public conversations about important issues. Fourth, although the evaluator’s relationship with program staff could be collegial, Greene believes that evaluators have no authority or responsibility for the program design and implementation. Fifth, Greene declares that advocacy in evaluation is inevitable and, as a result, evaluators should play a socially enfranchising role today.  Conclusions: The debate on evaluation’s main theoretical foundations is still relevant. For this purpose, the author recommends that the five main theories applied by this article to examine Jennifer Greene’s work should be used more systematically in the future to describe and analyze evaluators’ practice. Such theoretical categories would be especially beneficial in that they will provide some common ground of understanding among both practitioners and scholars on evaluation concepts and practices which experience has shown to be in constant evolution. Keywords: evaluation theory, valuing, social programming, knowledge, practice, use

/pdf/theorists-theories-of-evaluation-a-conversation-with-2ipmbmezoi.pdf

Michele Tarsilla

Papers

Cochrane Handbook for Systematic Reviews of Interventions

Being Blind in a World of Multiple Perspectives: The Evaluator’s Dilemma Between the Hope of Becoming a Team Player and the Fear of Becoming a Critical Friend with No Friends

Inclusiveness and Social Justice in Evaluation: Can the Transformative Agenda Really Alter the Status Quo? A Conversation with Donna M. Mertens

Theorists’ Theories of Evaluation: A Conversation with Jennifer Greene