David Moher and colleagues introduce PRISMA, an update of the QUOROM guidelines for reporting systematic reviews and meta-analyses

/pdf/preferred-reporting-items-for-systematic-reviews-and-meta-3k0wa2mp7o.pdf

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement

Systematic reviews and meta-analyses have become increasingly important in health care. Clinicians read them to keep up to date with their field,1,2 and they are often used as a starting point for developing clinical practice guidelines. Granting agencies may require a systematic review to ensure there is justification for further research,3 and some health care journals are moving in this direction.4 As with all research, the value of a systematic review depends on what was done, what was found, and the clarity of reporting. As with other publications, the reporting quality of systematic reviews varies, limiting readers' ability to assess the strengths and weaknesses of those reviews.

Several early studies evaluated the quality of review reports. In 1987, Mulrow examined 50 review articles published in 4 leading medical journals in 1985 and 1986 and found that none met all 8 explicit scientific criteria, such as a quality assessment of included studies.5 In 1987, Sacks and colleagues6 evaluated the adequacy of reporting of 83 meta-analyses on 23 characteristics in 6 domains. Reporting was generally poor; between 1 and 14 characteristics were adequately reported (mean = 7.7; standard deviation = 2.7). A 1996 update of this study found little improvement.7

In 1996, to address the suboptimal reporting of meta-analyses, an international group developed a guidance called the QUOROM Statement (QUality Of Reporting Of Meta-analyses), which focused on the reporting of meta-analyses of randomized controlled trials.8 In this article, we summarize a revision of these guidelines, renamed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses), which have been updated to address several conceptual and practical advances in the science of systematic reviews (Box 1).



Box 1

Conceptual issues in the evolution from QUOROM to PRISMA

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.

Objective: Funnel plots (plots of effect estimates against sample size) may be useful to detect bias in meta-analyses that were later contradicted by large trials. We examined whether a simple test of asymmetry of funnel plots predicts discordance of results when meta-analyses are compared to large trials, and we assessed the prevalence of bias in published meta-analyses. Design: Medline search to identify pairs consisting of a meta-analysis and a single large trial (concordance of results was assumed if effects were in the same direction and the meta-analytic estimate was within 30% of the trial); analysis of funnel plots from 37 meta-analyses identified from a hand search of four leading general medicine journals 1993-6 and 38 meta-analyses from the second 1996 issue of the Cochrane Database of Systematic Reviews . Main outcome measure: Degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates against precision. Results: In the eight pairs of meta-analysis and large trial that were identified (five from cardiovascular medicine, one from diabetic medicine, one from geriatric medicine, one from perinatal medicine) there were four concordant and four discordant pairs. In all cases discordance was due to meta-analyses showing larger effects. Funnel plot asymmetry was present in three out of four discordant pairs but in none of concordant pairs. In 14 (38%) journal meta-analyses and 5 (13%) Cochrane reviews, funnel plot asymmetry indicated that there was bias. Conclusions: A simple analysis of funnel plots provides a useful test for the likely presence of bias in meta-analyses, but as the capacity to detect bias will be limited when meta-analyses are based on a limited number of small trials the results from such analyses should be treated with considerable caution. Key messages Systematic reviews of randomised trials are the best strategy for appraising evidence; however, the findings of some meta-analyses were later contradicted by large trials Funnel plots, plots of the trials9 effect estimates against sample size, are skewed and asymmetrical in the presence of publication bias and other biases Funnel plot asymmetry, measured by regression analysis, predicts discordance of results when meta-analyses are compared with single large trials Funnel plot asymmetry was found in 38% of meta-analyses published in leading general medicine journals and in 13% of reviews from the Cochrane Database of Systematic Reviews Critical examination of systematic reviews for publication and related biases should be considered a routine procedure

/pdf/bias-in-meta-analysis-detected-by-a-simple-graphical-test-5erh428s5c.pdf

Bias in meta-analysis detected by a simple, graphical test

Systematic review and meta-analysis

Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement

Erratum to: “CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials” [J Clin Epidemiol 2010;63(8):e1–37]

Background Randomized trials without reported adequate allocation concealment have been shown to overestimate the benefit of experimental interventions. We investigated the robustness of conclusions drawn from meta-analyses to exclusion of such trials. Material Random sample of 38 reviews from The Cochrane Library 2003, issue 2 and 32 other reviews from PubMed accessed in 2002. Eligible reviews presented a binary effect estimate from a meta-analysis of randomized controlled trials as the first statistically significant result that supported a conclusion in favour of one of the interventions. Methods We assessed the methods sections of the trials in each included meta-analysis for adequacy of allocation concealment. We replicated each meta-analysis using the authors' methods but included only trials that had adequate allocation concealment. Conclusions were defined as not supported if our result was not statistically significant. Results Thirty-four of the 70 meta-analyses contained a mixture of trials with unclear or inadequate concealment as well as trials with adequate allocation concealment. Four meta-analyses only contained trials with adequate concealment, and 32, only trials with unclear or inadequate concealment. When only trials with adequate concealment were included, 48 of 70 conclusions (69%; 95% confidence interval: 56-79%) lost support. The loss of support mainly reflected loss of power (the total number of patients was reduced by 49%) but also a shift in the point estimate towards a less beneficial effect. Conclusion Two-thirds of conclusions in favour of one of the interventions were no longer supported if only trials with adequate allocation concealment were included.

Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials

It can be challenging for patients and clinicians to properly interpret a change in the clinical condition after a treatment has been given. It is not known to which extent spontaneous improvement, effect of placebo and effect of active interventions contribute to the observed change from baseline, and we aimed at quantifying these contributions. Systematic review and meta-analysis, based on a Cochrane review of the effect of placebo interventions for all clinical conditions. We selected all trials that had randomised the patients to three arms: no treatment, placebo and active intervention, and that had used an outcome that was measured on a continuous scale or on a ranking scale. Clinical conditions that had been studied in less than three trials were excluded. We analysed 37 trials (2900 patients) that covered 8 clinical conditions. The active interventions were psychological in 17 trials, physical in 15 trials, and pharmacological in 5 trials. Overall, across all conditions and interventions, there was a statistically significant change from baseline in all three arms. The standardized mean difference (SMD) for change from baseline was -0.24 (95% confidence interval -0.36 to -0.12) for no treatment, -0.44 (-0.61 to -0.28) for placebo, and -1.01 (-1.16 to -0.86) for active treatment. Thus, on average, the relative contributions of spontaneous improvement and of placebo to that of the active interventions were 24% and 20%, respectively, but with some uncertainty, as indicated by the confidence intervals for the three SMDs. The conditions that had the most pronounced spontaneous improvement were nausea (45%), smoking (40%), depression (35%), phobia (34%) and acute pain (25%). Spontaneous improvement and effect of placebo contributed importantly to the observed treatment effect in actively treated patients, but the relative importance of these factors differed according to clinical condition and intervention.

https://link.springer.com/content/pdf/10.1186%2F1471-2288-9-1.pdf

Spontaneous improvement in randomised clinical trials: meta-analysis of three-armed trials comparing no treatment, placebo and active intervention

Objective To compare the methodological quality and conclusions in Cochrane reviews with those in industry supported meta-analyses and other meta-analyses of the same drugs.

Design Systematic review comparing pairs of meta-analyses that studied the same two drugs in the same disease and were published within two years of each other.

Data sources Cochrane Database of Systematic Reviews (2003, issue 1), PubMed, and Embase.

Data extraction Two observers independently extracted data and used a validated scale to judge the methodological quality of the reviews.

Results 175 of 1596 Cochrane reviews had a meta-analysis that compared two drugs. Twenty four meta-analyses that matched the Cochrane reviews were found: eight were industry supported, nine had undeclared support, and seven had no support or were supported by non-industry sources. On a 0-7 scale, the median quality score was 7 for Cochrane reviews and 3 for other reviews (P < 0.01). Compared with industry supported reviews and reviews with undeclared support, Cochrane reviews had more often considered the potential for bias in the review—for example, by describing the method of concealment of allocation and describing excluded patients or studies. The seven industry supported reviews that had conclusions recommended the experimental drug without reservations, compared with none of the Cochrane reviews (P = 0.02), although the estimated treatment effect was similar on average (z = 0.46, P = 0.64). Reviews with undeclared support and reviews with not for profit support or no support had conclusions that were similar in cautiousness to the Cochrane reviews.

Conclusions Industry supported reviews of drugs should be read with caution as they were less transparent, had few reservations about methodological limitations of the included trials, and had more favourable conclusions than the corresponding Cochrane reviews.

https://www.bmj.com/content/bmj/333/7572/782.full.pdf

Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review

Definition Non-steroidal anti-inflammatory drugs (NSAIDs) have anti-inflammatory, analgesic, and antipyretic effects and inhibit thrombocyte aggregation. The drugs have no documented effect on the disease process itself.

Incidence/prevalence NSAIDs are widely used. Almost 10% of people in the Netherlands had used a non-aspirin NSAID in 1987; the overall use was 11 defined daily doses (see box) per 1000 persons a day.1 In Australia in 1994, overall use was 35 defined daily doses per 1000 persons a day, with 36% of the people receiving NSAIDs for osteoarthritis, 42% for sprain and strain or low back pain, and 4% for rheumatoid arthritis; 35% were aged over 60 years.2

Aims To reduce symptoms in rheumatic disorders; to avoid severe gastrointestinal adverse effects.

Outcomes Primary outcomes: pain intensity, person's preference for one drug over another, global efficacy, and clinically significant gastrointestinal complications. Secondary outcomes: number of tender joints, perforation, gastrointestinal haemorrhage, dyspepsia, and ulcer detected by routine endoscopy.

Defined daily dose : The assumed average daily dose for the main indication of a specified drug. The defined daily dose per 1000 population per day is an estimate of the proportion of that population receiving treatment with that drug.

We searched Medline and the Cochrane Library in July 1999 for systematic reviews and randomised controlled trials (RCTs) that included at least 100 people. More than 100 meta-analyses and thousands of RCTs have compared various NSAIDs. Many trials are unpublished or published in sources that are not indexed in publicly available databases. The quality of the trials is variable and bias is common, both in the design and analysis of the trials, to such an extent that a systematic review identified false significant findings favouring new drugs over control drugs in 6% of trials.3

#### Interventions

##### Beneficial:

NSAIDs in rheumatoid arthritis

Misoprostol in high risk patients who …

Peter C Gøtzsche

Papers

Erratum to: “CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials” [J Clin Epidemiol 2010;63(8):e1–37]

Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials

Spontaneous improvement in randomised clinical trials: meta-analysis of three-armed trials comparing no treatment, placebo and active intervention

Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review

Non-steroidal anti-inflammatory drugs.