David Moher and colleagues introduce PRISMA, an update of the QUOROM guidelines for reporting systematic reviews and meta-analyses

/pdf/preferred-reporting-items-for-systematic-reviews-and-meta-3k0wa2mp7o.pdf

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement

Systematic reviews and meta-analyses have become increasingly important in health care. Clinicians read them to keep up to date with their field,1,2 and they are often used as a starting point for developing clinical practice guidelines. Granting agencies may require a systematic review to ensure there is justification for further research,3 and some health care journals are moving in this direction.4 As with all research, the value of a systematic review depends on what was done, what was found, and the clarity of reporting. As with other publications, the reporting quality of systematic reviews varies, limiting readers' ability to assess the strengths and weaknesses of those reviews.

Several early studies evaluated the quality of review reports. In 1987, Mulrow examined 50 review articles published in 4 leading medical journals in 1985 and 1986 and found that none met all 8 explicit scientific criteria, such as a quality assessment of included studies.5 In 1987, Sacks and colleagues6 evaluated the adequacy of reporting of 83 meta-analyses on 23 characteristics in 6 domains. Reporting was generally poor; between 1 and 14 characteristics were adequately reported (mean = 7.7; standard deviation = 2.7). A 1996 update of this study found little improvement.7

In 1996, to address the suboptimal reporting of meta-analyses, an international group developed a guidance called the QUOROM Statement (QUality Of Reporting Of Meta-analyses), which focused on the reporting of meta-analyses of randomized controlled trials.8 In this article, we summarize a revision of these guidelines, renamed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses), which have been updated to address several conceptual and practical advances in the science of systematic reviews (Box 1).



Box 1

Conceptual issues in the evolution from QUOROM to PRISMA

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.

Objective: Funnel plots (plots of effect estimates against sample size) may be useful to detect bias in meta-analyses that were later contradicted by large trials. We examined whether a simple test of asymmetry of funnel plots predicts discordance of results when meta-analyses are compared to large trials, and we assessed the prevalence of bias in published meta-analyses. Design: Medline search to identify pairs consisting of a meta-analysis and a single large trial (concordance of results was assumed if effects were in the same direction and the meta-analytic estimate was within 30% of the trial); analysis of funnel plots from 37 meta-analyses identified from a hand search of four leading general medicine journals 1993-6 and 38 meta-analyses from the second 1996 issue of the Cochrane Database of Systematic Reviews . Main outcome measure: Degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates against precision. Results: In the eight pairs of meta-analysis and large trial that were identified (five from cardiovascular medicine, one from diabetic medicine, one from geriatric medicine, one from perinatal medicine) there were four concordant and four discordant pairs. In all cases discordance was due to meta-analyses showing larger effects. Funnel plot asymmetry was present in three out of four discordant pairs but in none of concordant pairs. In 14 (38%) journal meta-analyses and 5 (13%) Cochrane reviews, funnel plot asymmetry indicated that there was bias. Conclusions: A simple analysis of funnel plots provides a useful test for the likely presence of bias in meta-analyses, but as the capacity to detect bias will be limited when meta-analyses are based on a limited number of small trials the results from such analyses should be treated with considerable caution. Key messages Systematic reviews of randomised trials are the best strategy for appraising evidence; however, the findings of some meta-analyses were later contradicted by large trials Funnel plots, plots of the trials9 effect estimates against sample size, are skewed and asymmetrical in the presence of publication bias and other biases Funnel plot asymmetry, measured by regression analysis, predicts discordance of results when meta-analyses are compared with single large trials Funnel plot asymmetry was found in 38% of meta-analyses published in leading general medicine journals and in 13% of reviews from the Cochrane Database of Systematic Reviews Critical examination of systematic reviews for publication and related biases should be considered a routine procedure

/pdf/bias-in-meta-analysis-detected-by-a-simple-graphical-test-5erh428s5c.pdf

Bias in meta-analysis detected by a simple, graphical test

Systematic review and meta-analysis

Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement

The US Food and Drug Administration's authorisation of Purdue's controlled-release methylphenidate for adult ADHD: comments on the regulatory practice

Acupuncture Treatment for Pain: Systematic Review of Randomised Clinical Trials with Acupuncture, Placebo Acupuncture, and no Acupuncture Groups

No study has looked at differences of pooled estimates—such as meta-analyses—of corresponding study documents of the same intervention. In this study, we compared meta-analyses of human papillomavirus (HPV) vaccine trial data from clinical study reports with trial data from corresponding trial register entries and journal publications. We obtained clinical study reports from the European Medicines Agency and GlaxoSmithKline, corresponding trial register entries from ClinicalTrials.gov and corresponding journal publications via the Cochrane Collaboration’s Central Register of Controlled Trials, Google Scholar and PubMed. Two researchers extracted data. We compared reporting of trial design aspects and 20 prespecified benefit and harm outcomes extracted from each study document type. Risk ratios were calculated with the random effects inverse variance method. We included study documents from 22 randomized clinical trials and 2 follow-up studies with 95,670 healthy participants and non-HPV vaccine comparators (placebo, HPV vaccine adjuvants and hepatitis vaccines). We obtained 24 clinical study reports, 24 corresponding trial register entries and 23 corresponding journal publications; the median number of pages was 1351 (range 357 to 11,456), 32 (range 11 to 167) and 11 (range 7 to 83), respectively. All 24 (100%) clinical study reports, no (0%) trial register entries and 9 (39%) journal publications reported on all six major design-related biases defined by the Cochrane Handbook version 2011. The clinical study reports reported more inclusion criteria (mean 7.0 vs. 5.8 [trial register entries] and 4.0 [journal publications]) and exclusion criteria (mean 17.8 vs. 11.7 and 5.0) but fewer primary outcomes (mean 1.6 vs. 3.5 and 1.2) and secondary outcomes (mean 8.8 vs. 13.0 and 3.2) than the trial register entries. Results were posted for 19 trial register entries (79%). Compared to the clinical study reports, the trial register entries and journal publications contained 3% and 44% of the seven assessed benefit data points (6879 vs. 230 and 3015) and 38% and 31% of the 13 assessed harm data points (167,550 vs. 64,143 and 51,899). No meta-analysis estimate differed significantly when we compared pooled risk ratio estimates of corresponding study document data as ratios of relative risk. There were no significant differences in the meta-analysis estimates of the assessed outcomes from corresponding study documents. The clinical study reports were the superior study documents in terms of the quantity and the quality of the data they contained and should be used as primary data sources in systematic reviews. The protocol for our comparison is registered on PROSPERO as an addendum to our systematic review of the benefits and harms of the HPV vaccines: https://www.crd.york.ac.uk/PROSPEROFILES/56093_PROTOCOL_20180320.pdf: CRD42017056093. Our systematic review protocol was registered on PROSPERO on January 2017: https://www.crd.york.ac.uk/PROSPEROFILES/56093_PROTOCOL_20170030.pdf. Two protocol amendments were registered on PROSPERO on November 2017: https://www.crd.york.ac.uk/PROSPEROFILES/56093_PROTOCOL_20171116.pdf. Our index of the HPV vaccine studies was published in Systematic Reviews on January 2018: https://doi.org/10.1186/s13643-018-0675-z. A description of the challenges obtaining the data was published on September 2018: https://doi.org/10.1136/bmj.k3694.

/pdf/benefits-and-harms-of-the-human-papillomavirus-hpv-vaccines-2jnmjdwuer.pdf

Benefits and harms of the human papillomavirus (HPV) vaccines: comparison of trial data from clinical study reports with corresponding trial register entries and journal publications

At an open seminar arranged by Health Action International (Europe) on 12 October 2012 about access to medical research data, I asked why European governments had not sued Roche to get the money back that they had spent on needlessly stockpiling oseltamivir (Tamiflu). Roche has withheld data that purport to show that oseltamivir has dramatic effects. We all wonder why it …

https://www.bmj.com/content/bmj/345/bmj.e7689.full.pdf

Peter C Gøtzsche

Papers

The US Food and Drug Administration's authorisation of Purdue's controlled-release methylphenidate for adult ADHD: comments on the regulatory practice

Acupuncture Treatment for Pain: Systematic Review of Randomised Clinical Trials with Acupuncture, Placebo Acupuncture, and no Acupuncture Groups

Benefits and harms of the human papillomavirus (HPV) vaccines: comparison of trial data from clinical study reports with corresponding trial register entries and journal publications

European governments should sue Roche and prescribers should boycott its drugs.

Trends in breast-conserving surgery in the Southeast Netherlands