David Moher and colleagues introduce PRISMA, an update of the QUOROM guidelines for reporting systematic reviews and meta-analyses

/pdf/preferred-reporting-items-for-systematic-reviews-and-meta-3k0wa2mp7o.pdf

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement

Systematic reviews and meta-analyses have become increasingly important in health care. Clinicians read them to keep up to date with their field,1,2 and they are often used as a starting point for developing clinical practice guidelines. Granting agencies may require a systematic review to ensure there is justification for further research,3 and some health care journals are moving in this direction.4 As with all research, the value of a systematic review depends on what was done, what was found, and the clarity of reporting. As with other publications, the reporting quality of systematic reviews varies, limiting readers' ability to assess the strengths and weaknesses of those reviews.

Several early studies evaluated the quality of review reports. In 1987, Mulrow examined 50 review articles published in 4 leading medical journals in 1985 and 1986 and found that none met all 8 explicit scientific criteria, such as a quality assessment of included studies.5 In 1987, Sacks and colleagues6 evaluated the adequacy of reporting of 83 meta-analyses on 23 characteristics in 6 domains. Reporting was generally poor; between 1 and 14 characteristics were adequately reported (mean = 7.7; standard deviation = 2.7). A 1996 update of this study found little improvement.7

In 1996, to address the suboptimal reporting of meta-analyses, an international group developed a guidance called the QUOROM Statement (QUality Of Reporting Of Meta-analyses), which focused on the reporting of meta-analyses of randomized controlled trials.8 In this article, we summarize a revision of these guidelines, renamed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses), which have been updated to address several conceptual and practical advances in the science of systematic reviews (Box 1).



Box 1

Conceptual issues in the evolution from QUOROM to PRISMA

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.

Objective: Funnel plots (plots of effect estimates against sample size) may be useful to detect bias in meta-analyses that were later contradicted by large trials. We examined whether a simple test of asymmetry of funnel plots predicts discordance of results when meta-analyses are compared to large trials, and we assessed the prevalence of bias in published meta-analyses. Design: Medline search to identify pairs consisting of a meta-analysis and a single large trial (concordance of results was assumed if effects were in the same direction and the meta-analytic estimate was within 30% of the trial); analysis of funnel plots from 37 meta-analyses identified from a hand search of four leading general medicine journals 1993-6 and 38 meta-analyses from the second 1996 issue of the Cochrane Database of Systematic Reviews . Main outcome measure: Degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates against precision. Results: In the eight pairs of meta-analysis and large trial that were identified (five from cardiovascular medicine, one from diabetic medicine, one from geriatric medicine, one from perinatal medicine) there were four concordant and four discordant pairs. In all cases discordance was due to meta-analyses showing larger effects. Funnel plot asymmetry was present in three out of four discordant pairs but in none of concordant pairs. In 14 (38%) journal meta-analyses and 5 (13%) Cochrane reviews, funnel plot asymmetry indicated that there was bias. Conclusions: A simple analysis of funnel plots provides a useful test for the likely presence of bias in meta-analyses, but as the capacity to detect bias will be limited when meta-analyses are based on a limited number of small trials the results from such analyses should be treated with considerable caution. Key messages Systematic reviews of randomised trials are the best strategy for appraising evidence; however, the findings of some meta-analyses were later contradicted by large trials Funnel plots, plots of the trials9 effect estimates against sample size, are skewed and asymmetrical in the presence of publication bias and other biases Funnel plot asymmetry, measured by regression analysis, predicts discordance of results when meta-analyses are compared with single large trials Funnel plot asymmetry was found in 38% of meta-analyses published in leading general medicine journals and in 13% of reviews from the Cochrane Database of Systematic Reviews Critical examination of systematic reviews for publication and related biases should be considered a routine procedure

/pdf/bias-in-meta-analysis-detected-by-a-simple-graphical-test-5erh428s5c.pdf

Bias in meta-analysis detected by a simple, graphical test

Systematic review and meta-analysis

Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement

AIM Our objective was to explore communications from drug agencies about benzodiazepine dependence and selective serotonin reuptake inhibitors (SSRIs) withdrawal reactions over time. METHODS Documentary study. We searched the web-sites of the European Medicines Agency and the drug agencies in USA, UK, and Denmark for documents mentioning benzodiazepines or SSRIs. We supplemented with other relevant literature that could contribute to our study. The searches were performed in 2009 in PubMed, Google, BMJ and JAMA. RESULTS It took many years before the drug regulators acknowledged benzodiazepine dependence and SSRI withdrawal reactions and before the prescribers and the public were informed. Drug regulators relied mainly on the definitions of dependence and withdrawal reactions from the diagnostic psychiatric manuals, which contributed to the idea that SSRIs do not cause dependence, although it is difficult for many patients to stop treatment. In the perspective of a precautionary principle, drug agencies have failed to acknowledge that SSRIs can cause dependence and have minimised the problem with regard to its frequency and severity. In the perspective of a risk management principle, the drug agencies have reacted in concordance with the slowly growing knowledge of adverse drug reactions and have sharpened the information to the prescribers and the public over time. However, solely relying on spontaneous reporting of adverse effects leads to underestimation and delayed information about the problems. CONCLUSION Given the experience with the benzodiazepines, we believe the regulatory bodies should have required studies from the manufacturers that could have elucidated the dependence potential of the SSRIs before marketing authorization was granted.

Dependence and withdrawal reactions to benzodiazepines and selective serotonin reuptake inhibitors. How did the health authorities react

Olsen et al. used largely the same data material and the same methods as those used by Kalager et al. to estimate the impact of breast screening in Norway on breast cancer mortality. Unsurprisingly, the results were very similar. Olsen et al. found a nonsignificant 11% reduction (95% confidence interval: a 23% reduction to a 12% increase) and Kalager et al. found a 10% reduction (95% confidence interval: a 24% reduction to a 4% increase). Both author teams compared screened and unscreened women using a contemporary and two historical control groups. Despite the nonsignificant difference, both teams nevertheless concluded that there was an association between mammography screening and reductions in breast cancer mortality. However, the further interpretation of these same findings differed radically. Kalager et al. argued that ‘‘. . .screening itself accounted for only about a third of the total reduction’’ because specialized treatment was introduced simultaneously with organized screening, and similar reductions were found in nonscreened age groups in the screened areas. Olsen et al. argued that prior regular mammography use in 40% of Norwegian women diluted the ‘‘true’’ effect of screening of 25% seen in the screening trials and that their result ‘‘corresponded very well’’ with expectations. A ‘‘true’’ effect of 25% is exactly the same as Olsen’s previous estimate of a 25% effect of breast screening in Copenhagen based on an observational study. We have published data with longer follow-up and shown that there is no measurable effect of breast screening in Copenhagen and Funen. Their previous publication from 2011, which measured mammography use in Norway, also exactly predicted the results in this article. They wrote: ‘‘Assuming that the Norwegian program would in absence of prior screening have decreased breast cancer mortality by 25%, and that the activity inand outside the organized program were equally effective, the measured effect of the organized program would under actual circumstances be a reduction of 11%.’’ Such precision and predictive powers are astonishing, but their basic arguments do not hold. The assessment of prior mammography use is flawed and it is speculative whether opportunistic screening will ‘‘dilute’’ a screening effect. First of all, Olsen et al. do not differentiate between screening and clinical mammograms, neither in this article nor in their previous study to which they refer. In this article, they write that ‘‘40% of women used regular mammography prior to the program,’’ which suggests that 40% of Norwegian women had prior opportunistic screening mammograms. However, what they previously reported was that ‘‘Prior to start of the organized program, 40% of women in target age groups reported to have had mammography examination.’’ This could be both screening and clinical mammograms, and there was no quantification of either type of mammogram in their report. This difference is crucial, because the idea with organized breast screening is to reduce breast cancer mortality above that which can be obtained by using mammograms on a clinical suspicion of breast cancer. Thirty-six percent of women in the screened age range in nonscreened areas of Denmark reported to have had a clinical mammogram previously, whereas only 5% had a screening mammogram, and 3% had both. There is no reason to think opportunistic screening is more common in Norway than in Denmark and some of the authors of the present article have previously shown that opportunistic screening in Denmark is rare. In that study, the authors did make the crucial distinction. Back then, in 2005, they argued that the only way to obtain widespread screening use would be through implementation of an organized programme. A national breast screening program was implemented in Denmark from 2007. Second, opportunistic breast screening largely takes place in private clinics or at diagnostic radiological departments, not in specialized screening units such as those used in the randomized trials or in organized screening programmes. The assumption by Olsen et al. that the effect of opportunistic screening on breast cancer mortality is ‘‘equal’’ to that of organized screening is speculative and unsupported by evidence and contrasts with the notion that screening should only be undertaken within the setting of an organized programme with rigid quality assurance as specified in guidelines. Olsen et al. note that screening has not led to a reduction in advanced breast cancers in Norway, but they do not draw the obvious conclusion that screening therefore cannot have reduced breast cancer mortality. The only conclusion that can be supported by the available data is that; ‘‘An association between breast screening in Norway and a reduction in breast cancer mortality could not be demonstrated.’’ Thirteen years of breast screening in Norway have had no measurable effect on breast cancer mortality; it has not reduced the occurrence of advanced breast cancer either but has led to substantial overdiagnosis and increased Le tt er to th e E di to r

Thirteen years of breast screening had no measurable effect on breast cancer mortality in Norway

The figure in the news item by Mayor shows that UK breast cancer mortality has fallen steadily since 1989.1 According to Cancer Research UK, the NHS breast screening programme has contributed, but it was introduced in 1988, only a year before the decline began.1 Screening works through advancing the time of detection, and …

Peter C Gøtzsche

Papers

Dependence and withdrawal reactions to benzodiazepines and selective serotonin reuptake inhibitors. How did the health authorities react

Thirteen years of breast screening had no measurable effect on breast cancer mortality in Norway

Fall in breast cancer deaths: A cause for celebration, and caution

Dødelig medicin og organiseret kriminalitet: Hvordan medicinalindustrien har korrumperet sundhedsvæsenet

Core belief in powerful effects of placebo interventions is in conflict with no evidence of important effects in a large systematic review