David Moher and colleagues introduce PRISMA, an update of the QUOROM guidelines for reporting systematic reviews and meta-analyses

/pdf/preferred-reporting-items-for-systematic-reviews-and-meta-3k0wa2mp7o.pdf

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement

Systematic reviews and meta-analyses have become increasingly important in health care. Clinicians read them to keep up to date with their field,1,2 and they are often used as a starting point for developing clinical practice guidelines. Granting agencies may require a systematic review to ensure there is justification for further research,3 and some health care journals are moving in this direction.4 As with all research, the value of a systematic review depends on what was done, what was found, and the clarity of reporting. As with other publications, the reporting quality of systematic reviews varies, limiting readers' ability to assess the strengths and weaknesses of those reviews.

Several early studies evaluated the quality of review reports. In 1987, Mulrow examined 50 review articles published in 4 leading medical journals in 1985 and 1986 and found that none met all 8 explicit scientific criteria, such as a quality assessment of included studies.5 In 1987, Sacks and colleagues6 evaluated the adequacy of reporting of 83 meta-analyses on 23 characteristics in 6 domains. Reporting was generally poor; between 1 and 14 characteristics were adequately reported (mean = 7.7; standard deviation = 2.7). A 1996 update of this study found little improvement.7

In 1996, to address the suboptimal reporting of meta-analyses, an international group developed a guidance called the QUOROM Statement (QUality Of Reporting Of Meta-analyses), which focused on the reporting of meta-analyses of randomized controlled trials.8 In this article, we summarize a revision of these guidelines, renamed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses), which have been updated to address several conceptual and practical advances in the science of systematic reviews (Box 1).



Box 1

Conceptual issues in the evolution from QUOROM to PRISMA

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.

Objective: Funnel plots (plots of effect estimates against sample size) may be useful to detect bias in meta-analyses that were later contradicted by large trials. We examined whether a simple test of asymmetry of funnel plots predicts discordance of results when meta-analyses are compared to large trials, and we assessed the prevalence of bias in published meta-analyses. Design: Medline search to identify pairs consisting of a meta-analysis and a single large trial (concordance of results was assumed if effects were in the same direction and the meta-analytic estimate was within 30% of the trial); analysis of funnel plots from 37 meta-analyses identified from a hand search of four leading general medicine journals 1993-6 and 38 meta-analyses from the second 1996 issue of the Cochrane Database of Systematic Reviews . Main outcome measure: Degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates against precision. Results: In the eight pairs of meta-analysis and large trial that were identified (five from cardiovascular medicine, one from diabetic medicine, one from geriatric medicine, one from perinatal medicine) there were four concordant and four discordant pairs. In all cases discordance was due to meta-analyses showing larger effects. Funnel plot asymmetry was present in three out of four discordant pairs but in none of concordant pairs. In 14 (38%) journal meta-analyses and 5 (13%) Cochrane reviews, funnel plot asymmetry indicated that there was bias. Conclusions: A simple analysis of funnel plots provides a useful test for the likely presence of bias in meta-analyses, but as the capacity to detect bias will be limited when meta-analyses are based on a limited number of small trials the results from such analyses should be treated with considerable caution. Key messages Systematic reviews of randomised trials are the best strategy for appraising evidence; however, the findings of some meta-analyses were later contradicted by large trials Funnel plots, plots of the trials9 effect estimates against sample size, are skewed and asymmetrical in the presence of publication bias and other biases Funnel plot asymmetry, measured by regression analysis, predicts discordance of results when meta-analyses are compared with single large trials Funnel plot asymmetry was found in 38% of meta-analyses published in leading general medicine journals and in 13% of reviews from the Cochrane Database of Systematic Reviews Critical examination of systematic reviews for publication and related biases should be considered a routine procedure

/pdf/bias-in-meta-analysis-detected-by-a-simple-graphical-test-5erh428s5c.pdf

Bias in meta-analysis detected by a simple, graphical test

Systematic review and meta-analysis

Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement

We studied the effects of increasingly intensive treatment regimens on anti-pseudomonal antibody response and survival in five successive cohorts of a total of 157 Danish cystic fibrosis patients after they had acquired chronic P. aeruginosa lung infection. The time periods were 1971–1975 (N = 21), 1976–1980 (N = 64), 1981–1986 (N = 27), 1987–1993 (N = 26), and 1994–2000 (N = 19). During this 30-year period, we introduced elective 2-week courses of chemotherapy every third month in all chronically infected patients, early aggressive treatment with inhalation of colistin and oral ciprofloxacin for 3 months whenever P. aeruginosa was cultured in sputum from noncolonized patients, and inhalation of recombinant human dornase alfa. There was a significant correlation between the calendar year when chronic P. aeruginosa infection was acquired and the subsequent increase in the level of precipitins (P < 0.00001). The median number of precipitins increased by 5 per year in the oldest calendar year cohort, and 1 per year in the youngest. The median age of onset of chronic P. aeruginosa increased from 9.3 years from 1981–1986 to 13.8 years from 1987–2000. Survival after acquisition of chronic P. aeruginosa lung infection improved with time (P = 0.008). Our study shows that CF patients who are treated intensively have lower antibody responses and longer survival after acquisition of chronic P. aeruginosa lung infection. Pediatr Pulmonol. 2004; 37:427–432. © 2004 Wiely-Liss, Inc.

Antibody response to Pseudomonas aeruginosa in cystic fibrosis patients: a marker of therapeutic success?--A 30-year cohort study of survival in Danish CF patients after onset of chronic P. aeruginosa lung infection.

Objective To compare the distribution of P values in abstracts of randomised controlled trials with that in observational studies, and to check P values between 0.04 and 0.06.

Design Cross sectional study of all 260 abstracts in PubMed of articles published in 2003 that contained “relative risk” or “odds ratio” and reported results from a randomised trial, and random samples of 130 abstracts from cohort studies and 130 from case-control studies. P values were noted or calculated if unreported.

Main outcome measures Prevalence of significant P values in abstracts and distribution of P values between 0.04 and 0.06.

Results The first result in the abstract was statistically significant in 70% of the trials, 84% of cohort studies, and 84% of case-control studies. Although many of these results were derived from subgroup or secondary analyses, or biased selection of results, they were presented without reservations in 98% of the trials. P values were more extreme in observational studies (P < 0.001) and in cohort studies than in case-control studies (P = 0.04). The distribution of P values around P = 0.05 was extremely skewed. Only five trials had 0.05 ≤ P < 0.06, whereas 29 trials had 0.04 ≤ P < 0.05. I could check the calculations for 27 of these trials. One of four non-significant results was significant. Four of the 23 significant results were wrong, five were doubtful, and four could be discussed. Nine cohort studies and eight case-control studies reported P values between 0.04 and 0.06, but in all 17 cases P < 0.05. Because the analyses had been adjusted for confounders, these results could not be checked.

Conclusions Significant results in abstracts are common but should generally be disbelieved.

https://www.bmj.com/content/bmj/333/7561/231.full.pdf

Believability of relative risks and odds ratios in abstracts: cross sectional study

Objective: To study whether somatostatin or its derivative octreotide is more effective than placebo for treating bleeding oesophageal varices. Methods: Randomised, double blind trial and meta-analysis with blinded analysis of data and writing of manuscripts. Setting: Departments of medical and surgical gastroenterology in Copenhagen. Subjects: Patients suspected of bleeding from oesophageal varices and of having cirrhosis of the liver. Main outcome measures: Survival, number of blood transfusions, and use of Sengstaken-Blakemore tube. Results: 86 patients were randomised; in each group 16 died within six weeks (95% confidence interval for difference in mortality —19% to 22%). There were no differences between those treated with somatostatin or placebo in median number of blood transfusions (8 v 5, P=0.07, 0 to 4 transfusions) or in numbers of patients who needed balloon tamponade (16 v 13, P=0.54, -11% to 28%). In a meta-analysis of three trials involving 290 patients somatostatin had no effect on survival compared with placebo (P=0.59, odds ratio 1.16; 0.67 to 2.01). For blood transfusions and use of balloon tamponade there was heterogeneity between the trials with no convincing evidence in favour of somatostatin. No placebo controlled trials have been performed with octreotide. Conclusion: Within the limited power of this study and meta-analysis we were unable to show a clinical benefit of somatostatin in the emergency treatment of bleeding oesophageal varices. Key messages Key messages This study failed to show a benefit of somatostatin Meta-analysis of the placebo controlled studies also failed to show a benefit Treatment with somatostatin or octreotide cannot be recommended at present Larger studies are needed to give a definitive answer

/pdf/somatostatin-v-placebo-in-bleeding-oesophageal-varices-2bztkn0fw1.pdf

Somatostatin v placebo in bleeding oesophageal varices: randomised trial and meta-analysis

Background
Studies have shown that industry-sponsored meta-analyses of drugs lack scientific rigour and have biased conclusions. However, these studies have been restricted to certain medical specialities. We compared all industry-supported meta-analyses of drug-drug comparisons with those without industry support.

/pdf/industry-supported-meta-analyses-compared-with-meta-analyses-1slibg6e5p.pdf

Industry-supported meta-analyses compared with meta-analyses with non-profit or no support: Differences in methodological quality and conclusions

BACKGROUND\nThe major allergen in house dust comes from mites. Chemical, physical and combined methods of reducing mite allergen levels are intended to reduce asthma symptoms in people who are sensitive to house dust mites.\n\n\nOBJECTIVES\nThe objective of this review was to assess the effects of reducing exposure to house dust mite antigens in the homes of mite-sensitive asthmatics.\n\n\nSEARCH STRATEGY\nWe searched the Cochrane Airways Group trials register, checked reference lists of articles and hand-searched Respiration (1980 to 1996) and Clinical and Experimental Allergy (1980 to 1996). The Cochrane Library is searched every three months.\n\n\nSELECTION CRITERIA\nRandomised trials of mite control measures vs placebo or no treatment in asthmatic people known to be sensitive to house dust mites.\n\n\nDATA COLLECTION AND ANALYSIS\nTwo reviewers applied the trial inclusion criteria, assessed their quality and extracted the data independently. Study authors were contacted to clarify information.\n\n\nMAIN RESULTS\nTwenty-nine trials (939 patients in the analyses) were included, with two trials awaiting assessment. There was little difference in improvement of asthma between patients in experimental groups compared to control groups (relative risk 1.04, 95% confidence interval (95%CI) 0.83 to 1.31). Asthma symptom scores were also similar for the experimental and control groups (standardised mean difference (SMD) -0.07, 95% CI -0.35 to 0.22), however there was significant heterogeneity between studies p=0.015. This appears to have been due, in part, to the parallel group studies of physical treatments. These three studies (107 patients) showed a significant reduction in symptoms; SMD -0.44 (95% CI -0.83, -0.06) with no heterogeneity between the trials. No significant difference was noted for medication usage (SMD -0.14, 95%CI -0.43 to 0.15). Peak flow in the morning showed no significant difference between the experimental and the control groups (SMD 0.04, 95%CI -0.13 to 0.21).\n\n\nREVIEWER'S CONCLUSIONS\nCurrent chemical methods aimed at reducing exposure to house dust mite allergens seem to be ineffective and cannot be recommended as prophylaxis for mite sensitive asthmatics. Physical reduction methods may reduce asthma symptoms, but results of larger and more rigorous studies are required before any recommendations can be made concerning this approach.

Peter C Gøtzsche

Papers

Antibody response to Pseudomonas aeruginosa in cystic fibrosis patients: a marker of therapeutic success?--A 30-year cohort study of survival in Danish CF patients after onset of chronic P. aeruginosa lung infection.

Believability of relative risks and odds ratios in abstracts: cross sectional study

Somatostatin v placebo in bleeding oesophageal varices: randomised trial and meta-analysis

Industry-supported meta-analyses compared with meta-analyses with non-profit or no support: Differences in methodological quality and conclusions

House dust mite control measures for asthma.