David Moher and colleagues introduce PRISMA, an update of the QUOROM guidelines for reporting systematic reviews and meta-analyses

/pdf/preferred-reporting-items-for-systematic-reviews-and-meta-3k0wa2mp7o.pdf

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement

Systematic reviews and meta-analyses have become increasingly important in health care. Clinicians read them to keep up to date with their field,1,2 and they are often used as a starting point for developing clinical practice guidelines. Granting agencies may require a systematic review to ensure there is justification for further research,3 and some health care journals are moving in this direction.4 As with all research, the value of a systematic review depends on what was done, what was found, and the clarity of reporting. As with other publications, the reporting quality of systematic reviews varies, limiting readers' ability to assess the strengths and weaknesses of those reviews.

Several early studies evaluated the quality of review reports. In 1987, Mulrow examined 50 review articles published in 4 leading medical journals in 1985 and 1986 and found that none met all 8 explicit scientific criteria, such as a quality assessment of included studies.5 In 1987, Sacks and colleagues6 evaluated the adequacy of reporting of 83 meta-analyses on 23 characteristics in 6 domains. Reporting was generally poor; between 1 and 14 characteristics were adequately reported (mean = 7.7; standard deviation = 2.7). A 1996 update of this study found little improvement.7

In 1996, to address the suboptimal reporting of meta-analyses, an international group developed a guidance called the QUOROM Statement (QUality Of Reporting Of Meta-analyses), which focused on the reporting of meta-analyses of randomized controlled trials.8 In this article, we summarize a revision of these guidelines, renamed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses), which have been updated to address several conceptual and practical advances in the science of systematic reviews (Box 1).



Box 1

Conceptual issues in the evolution from QUOROM to PRISMA

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.

Objective: Funnel plots (plots of effect estimates against sample size) may be useful to detect bias in meta-analyses that were later contradicted by large trials. We examined whether a simple test of asymmetry of funnel plots predicts discordance of results when meta-analyses are compared to large trials, and we assessed the prevalence of bias in published meta-analyses. Design: Medline search to identify pairs consisting of a meta-analysis and a single large trial (concordance of results was assumed if effects were in the same direction and the meta-analytic estimate was within 30% of the trial); analysis of funnel plots from 37 meta-analyses identified from a hand search of four leading general medicine journals 1993-6 and 38 meta-analyses from the second 1996 issue of the Cochrane Database of Systematic Reviews . Main outcome measure: Degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates against precision. Results: In the eight pairs of meta-analysis and large trial that were identified (five from cardiovascular medicine, one from diabetic medicine, one from geriatric medicine, one from perinatal medicine) there were four concordant and four discordant pairs. In all cases discordance was due to meta-analyses showing larger effects. Funnel plot asymmetry was present in three out of four discordant pairs but in none of concordant pairs. In 14 (38%) journal meta-analyses and 5 (13%) Cochrane reviews, funnel plot asymmetry indicated that there was bias. Conclusions: A simple analysis of funnel plots provides a useful test for the likely presence of bias in meta-analyses, but as the capacity to detect bias will be limited when meta-analyses are based on a limited number of small trials the results from such analyses should be treated with considerable caution. Key messages Systematic reviews of randomised trials are the best strategy for appraising evidence; however, the findings of some meta-analyses were later contradicted by large trials Funnel plots, plots of the trials9 effect estimates against sample size, are skewed and asymmetrical in the presence of publication bias and other biases Funnel plot asymmetry, measured by regression analysis, predicts discordance of results when meta-analyses are compared with single large trials Funnel plot asymmetry was found in 38% of meta-analyses published in leading general medicine journals and in 13% of reviews from the Cochrane Database of Systematic Reviews Critical examination of systematic reviews for publication and related biases should be considered a routine procedure

/pdf/bias-in-meta-analysis-detected-by-a-simple-graphical-test-5erh428s5c.pdf

Bias in meta-analysis detected by a simple, graphical test

Systematic review and meta-analysis

Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement

200 sera were tested for HIV antibodies with different tests in Tanzania. The results were interpreted by 5 different observers with different laboratory experience. There was considerable variation between observers and between testing methods. HIV-Chek was easiest to perform with little interobserver variation, but a few probably false negative readings were seen. Serodia gave many false positives. Western Blots gave more diverging results due to indeterminate sera and lack of training. HIV-Chek seems best when time, facilities, and training are limited and if combined with Serodia false negative results can be excluded.

Evaluation of simple tests for detection of HIV antibodies: analysis of interobserver variation in Tanzania.

Are encasings effective in asthma caused by house dust mite allergens

Introduction: Placebo treatments have been reported to help patients with many diseases, but the quality of the evidence supporting this belief has not been rigorously evaluated. Material and methods: We conducted a systematic review of clinical trials in which patients were randomised either to placebo treatment or to no treatment. A placebo intervention could be pharmacological (for instance, a tablet), physical (for instance manipulation), or psychological (for instance conversation). Results: We identified 130 trials that met our inclusion criteria. After excluding 16, which gave no relevant outcome data, 32 with binary outcomes and 82 with continuous outcomes were left. As compared to no treatment, placebo had no significant effect on binary outcomes, the pooled relative risk was 0.95 (95% confidence interval 0.88 to 1.02), irrespective of whether these outcomes were subjective or objective. Placebo seemed to have a beneficial effect in the trials with continuous outcomes, the overall pooled standardised mean difference of the placebo and no-treatment groups was -0.28 (-0.38 to -0.19), but the effect decreased with increasing sample size, which indicates bias. The pooled standardised mean difference was significant for the trials with subjective outcomes, -0.36 (-0.47 to -0.25), but not for those with objective outcomes, -0.12 (-0.27 to 0.03). In trials involving the treatment of pain, placebo was reported to have a moderate beneficial effect, with a pooled standardised mean difference of -0.27 (-0.40 to -0.15), but bias, for instance reporting bias, was possible. Conclusion: We found no reliable evidence that placebo treatments in general had effects of clinical importance.

[What is the effect of placebo interventions? A systematic review of randomized clinical trials with placebo treated and untreated patients].

Jorgensen and I recently gained access to clinical study reports and trial protocols of placebo controlled trials at the European Medicines Agency (EMA).1 Some drugs, however, have not been approved centrally, and in such cases the EMA advises contacting the relevant national drug agencies.

For the antidepressant drug fluoxetine, the United Kingdom acts as Reference Member State according to the Mutual Recognition …

https://www.bmj.com/content/bmj/343/bmj.d4203.full.pdf

UK drug regulator destroys all evidence after 15 years

Concern has been raised about whether HPV vaccines might cause serious neurological disorders including postural orthostatic tachycardia syndrome (POTS) and chronic regional pain syndrome (CRPS). The European Medicines Agency (EMA) investigated the issue and declared in 2015 that there is no link between HPV vaccines and serious neurological adverse events. However, the certainty conveyed in EMA's official report is undermined by a leaked, confidential document that reveals important disagreements among the experts. Furthermore, in its assessments, EMA relied on the data the drug companies had provided to them even though it had been demonstrated that the companies had underreported possible neurological harms. Even though active comparators were used (aluminium adjuvants and other vaccines), our research group found significantly more serious neurological harms in the HPV vaccine groups than in the comparator groups in a systematic review based on clinical study reports in EMA's possession. We outline areas where we believe the basis for EMA's decision was flawed; highlight that the relationship between HPV vaccines and POTS remains uncertain; and suggest ways forward to resolve the uncertainty and debate.

https://ebm.bmj.com/content/ebmed/early/2021/01/28/bmjebm-2020-111470.full.pdf

Peter C Gøtzsche

Papers

Evaluation of simple tests for detection of HIV antibodies: analysis of interobserver variation in Tanzania.

Are encasings effective in asthma caused by house dust mite allergens

[What is the effect of placebo interventions? A systematic review of randomized clinical trials with placebo treated and untreated patients].

UK drug regulator destroys all evidence after 15 years

EMA’s mishandling of an investigation into suspected serious neurological harms of HPV vaccines