David Moher and colleagues introduce PRISMA, an update of the QUOROM guidelines for reporting systematic reviews and meta-analyses

/pdf/preferred-reporting-items-for-systematic-reviews-and-meta-3k0wa2mp7o.pdf

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement

Systematic reviews and meta-analyses have become increasingly important in health care. Clinicians read them to keep up to date with their field,1,2 and they are often used as a starting point for developing clinical practice guidelines. Granting agencies may require a systematic review to ensure there is justification for further research,3 and some health care journals are moving in this direction.4 As with all research, the value of a systematic review depends on what was done, what was found, and the clarity of reporting. As with other publications, the reporting quality of systematic reviews varies, limiting readers' ability to assess the strengths and weaknesses of those reviews.

Several early studies evaluated the quality of review reports. In 1987, Mulrow examined 50 review articles published in 4 leading medical journals in 1985 and 1986 and found that none met all 8 explicit scientific criteria, such as a quality assessment of included studies.5 In 1987, Sacks and colleagues6 evaluated the adequacy of reporting of 83 meta-analyses on 23 characteristics in 6 domains. Reporting was generally poor; between 1 and 14 characteristics were adequately reported (mean = 7.7; standard deviation = 2.7). A 1996 update of this study found little improvement.7

In 1996, to address the suboptimal reporting of meta-analyses, an international group developed a guidance called the QUOROM Statement (QUality Of Reporting Of Meta-analyses), which focused on the reporting of meta-analyses of randomized controlled trials.8 In this article, we summarize a revision of these guidelines, renamed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses), which have been updated to address several conceptual and practical advances in the science of systematic reviews (Box 1).



Box 1

Conceptual issues in the evolution from QUOROM to PRISMA

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.

Objective: Funnel plots (plots of effect estimates against sample size) may be useful to detect bias in meta-analyses that were later contradicted by large trials. We examined whether a simple test of asymmetry of funnel plots predicts discordance of results when meta-analyses are compared to large trials, and we assessed the prevalence of bias in published meta-analyses. Design: Medline search to identify pairs consisting of a meta-analysis and a single large trial (concordance of results was assumed if effects were in the same direction and the meta-analytic estimate was within 30% of the trial); analysis of funnel plots from 37 meta-analyses identified from a hand search of four leading general medicine journals 1993-6 and 38 meta-analyses from the second 1996 issue of the Cochrane Database of Systematic Reviews . Main outcome measure: Degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates against precision. Results: In the eight pairs of meta-analysis and large trial that were identified (five from cardiovascular medicine, one from diabetic medicine, one from geriatric medicine, one from perinatal medicine) there were four concordant and four discordant pairs. In all cases discordance was due to meta-analyses showing larger effects. Funnel plot asymmetry was present in three out of four discordant pairs but in none of concordant pairs. In 14 (38%) journal meta-analyses and 5 (13%) Cochrane reviews, funnel plot asymmetry indicated that there was bias. Conclusions: A simple analysis of funnel plots provides a useful test for the likely presence of bias in meta-analyses, but as the capacity to detect bias will be limited when meta-analyses are based on a limited number of small trials the results from such analyses should be treated with considerable caution. Key messages Systematic reviews of randomised trials are the best strategy for appraising evidence; however, the findings of some meta-analyses were later contradicted by large trials Funnel plots, plots of the trials9 effect estimates against sample size, are skewed and asymmetrical in the presence of publication bias and other biases Funnel plot asymmetry, measured by regression analysis, predicts discordance of results when meta-analyses are compared with single large trials Funnel plot asymmetry was found in 38% of meta-analyses published in leading general medicine journals and in 13% of reviews from the Cochrane Database of Systematic Reviews Critical examination of systematic reviews for publication and related biases should be considered a routine procedure

/pdf/bias-in-meta-analysis-detected-by-a-simple-graphical-test-5erh428s5c.pdf

Bias in meta-analysis detected by a simple, graphical test

Systematic review and meta-analysis

Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement

Aims The European Medicines Agency (EMA) and the US Food and Drug Administration (FDA) produce guidelines for the design of pivotal psychiatric drug trials used in new drug applications. It is unknown who are involved in the guideline development and what specific trial design recommendations they give. Methods
 Cross-sectional study of EMA Clinical Efficacy and Safety Guidelines and FDA Guidance Documents. Study outcomes: (1) guideline committee members and declared conflicts of interest; (2) guideline development and organisation of commenting phases; (3) categorisation of stakeholders who comment on draft and final guidelines according to conflicts of interest (‘industry’, ‘not-industry but with industry-related conflicts’, ‘independent’, ‘unclear’); and (4) trial design recommendations (trial duration, psychiatric comorbidity, ‘enriched design’, efficacy outcomes, comparator choice). Protocol registration https://doi.org/10.1101/2020.01.22.20018499 (27 January 2020). Results
 We included 13 EMA and five FDA guidelines covering 15 psychiatric indications. Eleven months after submission, the EMA had not processed our request regarding committee member disclosures. FDA offices draft the Guidance Documents, but the Agency is not in possession of employee conflicts of interest declarations because FDA employees generally may not hold financial interests (although some employees may hold interests up to $15,000). The EMA and FDA guideline development phases are similar; drafts and final versions are publicly announced and everybody can submit comments. Seventy stakeholders commented on ten guidelines: 38 (54%) ‘industry’, 18 (26%) ‘not-industry but with industry-related conflicts’, six (9%) ‘independent’ and eight (11%) ‘unclear’. They submitted 1014 comments: 640 (68%) ‘industry’, 243 (26%) ‘not-industry but with industry-related conflicts’, 44 (5%) ‘independent’ and 20 (2%) ‘unclear’ (67 could not be assigned to a specific stakeholder). The recommended designs were generally for trials of short duration; with restricted trial populations; allowing previous exposure to the drug; and often recommending rating scale efficacy outcomes. EMA mainly recommended three arm designs (both placebo and active comparators), whereas FDA mainly recommended placebo-controlled designs. There were also other important differences and FDA's recommendations regarding the exclusion of psychiatric comorbidity seemed less restrictive. Conclusions
 The EMA and FDA clinical research guidelines for psychiatric pivotal trials recommend designs that tend to have limited generalisability. Independent and non-conflicted stakeholders are underrepresented in the guideline development. It seems warranted with more active involvement of scientists and independent organisations without conflicts of interest in the guideline development process.

/pdf/ema-and-fda-psychiatric-drug-trial-guidelines-assessment-of-2fgz0wqaxo.pdf

EMA and FDA psychiatric drug trial guidelines: assessment of guideline development and trial design recommendations.

Lack of immunomodulating effect of disulfiram on HIV positive patients.

To examine the therapeutic consequences of diagnostic tests for AIDS-related infections, case records from 33 deceased AIDS patients were reviewed; 23 were autopsied. Determination of serum antibody titres was not important. In particular, there was no relation between titres and isolation of cytomegalovirus (CMV); isolation attempts or possibly antigen determination would be better. Samples for CMV isolation were obtained from 31 patients; only 7 were negative. Blood, faeces, and particularly sputum cultures gave a low yield; the number of such examinations could be reduced considerably. However, 4/7 disseminated infections with atypical mycobacteria were only revealed at autopsy, despite numerous cultures in vivo. Liver biopsies were not helpful. Diagnostic procedures for Pneumocystis carinii pneumonia by lung biopsy caused pneumothorax in 3/15 patients; bronchoalveolar lavage or treatment/prophylaxis without diagnosis could be considered. In 8 autopsies, microscopy was suggestive of cerebral toxoplasmosis, but only 1 patient had presented important clinical symptoms. We suggest a schedule with regular microbiologic and parasitic examinations and few antibody tests, but with more antigen tests.

Peter C Gøtzsche

Papers

EMA and FDA psychiatric drug trial guidelines: assessment of guideline development and trial design recommendations.

Lack of immunomodulating effect of disulfiram on HIV positive patients.

Yield of diagnostic tests for opportunistic infections in AIDS: a survey of 33 patients.

Mammographic screening: no reliable supporting evidence?

The Two-County breast screening trial cannot provide a reliable estimate of the effect of breast cancer screening.