David Moher and colleagues introduce PRISMA, an update of the QUOROM guidelines for reporting systematic reviews and meta-analyses

/pdf/preferred-reporting-items-for-systematic-reviews-and-meta-3k0wa2mp7o.pdf

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement

Systematic reviews and meta-analyses have become increasingly important in health care. Clinicians read them to keep up to date with their field,1,2 and they are often used as a starting point for developing clinical practice guidelines. Granting agencies may require a systematic review to ensure there is justification for further research,3 and some health care journals are moving in this direction.4 As with all research, the value of a systematic review depends on what was done, what was found, and the clarity of reporting. As with other publications, the reporting quality of systematic reviews varies, limiting readers' ability to assess the strengths and weaknesses of those reviews.

Several early studies evaluated the quality of review reports. In 1987, Mulrow examined 50 review articles published in 4 leading medical journals in 1985 and 1986 and found that none met all 8 explicit scientific criteria, such as a quality assessment of included studies.5 In 1987, Sacks and colleagues6 evaluated the adequacy of reporting of 83 meta-analyses on 23 characteristics in 6 domains. Reporting was generally poor; between 1 and 14 characteristics were adequately reported (mean = 7.7; standard deviation = 2.7). A 1996 update of this study found little improvement.7

In 1996, to address the suboptimal reporting of meta-analyses, an international group developed a guidance called the QUOROM Statement (QUality Of Reporting Of Meta-analyses), which focused on the reporting of meta-analyses of randomized controlled trials.8 In this article, we summarize a revision of these guidelines, renamed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses), which have been updated to address several conceptual and practical advances in the science of systematic reviews (Box 1).



Box 1

Conceptual issues in the evolution from QUOROM to PRISMA

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.

Cochrane Reviews have recently started including the quantity I 2 to help readers assess the consistency of the results of studies in meta-analyses. What does this new quantity mean, and why is assessment of heterogeneity so important to clinical practice? 

Systematic reviews and meta-analyses can provide convincing and reliable evidence relevant to many aspects of medicine and health care.1 Their value is especially clear when the results of the studies they include show clinically important effects of similar magnitude. However, the conclusions are less clear when the included studies have differing results. In an attempt to establish whether studies are consistent, reports of meta-analyses commonly present a statistical test of heterogeneity. The test seeks to determine whether there are genuine differences underlying the results of the studies (heterogeneity), or whether the variation in findings is compatible with chance alone (homogeneity). However, the test is susceptible to the number of trials included in the meta-analysis. We have developed a new quantity, I 2, which we believe gives a better measure of the consistency between trials in a meta-analysis.

Assessment of the consistency of effects across studies is an essential part of meta-analysis. Unless we know how consistent the results of studies are, we cannot determine the generalisability of the findings of the meta-analysis. Indeed, several hierarchical systems for grading evidence state that the results of studies must be consistent or homogeneous to obtain the highest grading.2–4

Tests for heterogeneity are commonly used to decide on methods for combining studies and for concluding consistency or inconsistency of findings.5 6 But what does the test achieve in practice, and how should the resulting P values be interpreted?

A test for heterogeneity examines the null hypothesis that all studies are evaluating the same effect. The usual test statistic …

/pdf/measuring-inconsistency-in-meta-analyses-4iysmttp08.pdf

Measuring inconsistency in meta-analyses

Systematic review and meta-analysis

Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement

In mid-1997 I went for blood and other tests after an episode of back pain. A monoclonal electrophoretic peak and a spinal lesion of uncertain origin were found. After a few months of further tests I was given a diagnosis of monoclonal gammopathy of uncertain significance (MGUS). I was no longer a subjectively healthy man but a potentially ill person, with considerable anxiety. MGUS is one of those strange noso-logical entities of modern medicine—which is so good at creating “new diseases” without necessarily knowing how to cure them. The medical literature didn't help much. Several small studies reported a cumulative risk of malignant transformation of MGUSs of between 7% and 19%, with the likeliest estimate of annual risk of transformation around 1%, but without clear predictors.

> Could a second transplant improve results? 

I was prescribed blood tests every six months to monitor any such transformation. The years went by, during which my levels of anxiety varied, increasing at times of testing and whenever any personal or work …

/pdf/an-unfinished-trip-through-uncertainties-347oir3vbw.pdf

An unfinished trip through uncertainties

This paper presents the background, objectives and preliminary results of a coordinated European Project involving researchers in seven countries aimed at harmonizing methods to assess inappropriate use of hospital care and qualifying its determinants.

After briefly reviewing the philosophy of the European Union “Concerted Action” program in the context of current attempts at reforming the way health care is financed and delivered in several member states, the paper presents results obtained so far. Besides a method workshop where the approaches and methodologies used were compared, during the first year the following tasks have been undertaken: (a) identification and appraisal of a few dozen empirical studies conducted in various countries; (b) qualitative comparison of the instruments used to assess appropriateness; (c) initial development of a common list of reasons for inappropriate hospital use of potential relevance in European health care systems. While the project has already undoubtedly indicated the great potential of European collaboration in health services research, the large number of unpublished studies traced indicates—among other things—the need to create channels through which results of this type of research can be disseminated.

A European Project assessing the appropriateness of hospital utilization: Background, objectives and preliminary results

Objective: To help to co-ordinate and harmonize research on utilization review in Europe, the US Appropriateness Evaluation Protocol (AEP) was adapted for use in the European setting. The aim of this paper is to assess the reliability of the European version of the AEP (EU-AEP).

Design: Nineteen English-language medical records were reviewed by a physician reviewer from each of six participating countries: Austria, France, Italy, Spain, Switzerland and the UK. Each of the six reviewers was asked to assess the appropriateness of the 19 admissions and 31 hospitalization days (19 admission days and 12 randomly selected days of hospital stay, excluding days of discharge) using the revised review instrument. To evaluate inter-rater reliability, the κ statistic was used to measure overall and pair-wise agreement for the assessment of appropriateness of admission and of day of care, respectively.

Results: For admission, the overall κ statistic among the six reviewers was 0.64, with κ values for each pair of reviewers in the range 0.46-0.86. For day of care, the κ was 0.59, with pair-wise κcoefficients in the range 0.25-0.95.

Conclusion: The observed agreement could be considered substantial, especially if the fact that medical records were handwritten in a language native to only one of the reviewers is considered. Besides all the study limitations, this finding provides at least preliminary support for the application of the EU-AEP as a reliable instrument in the European setting, including application in comparative studies involving two or more countries.

https://serval.unil.ch/resource/serval:BIB_11397.P001/REF.pdf

Reliability study of the European appropriateness evaluation protocol.

Physicians' view of practice guidelines. A survey of Italian physicians

Double standards in information for medical journals and practitioners should go 

Medicines can offer enormous health benefits if choices for treatment are made appropriately, and availability of valid information is a necessary condition. The asymmetry in the information available to health professionals and consumers is a fundamental barrier to rational and informed choice. Good quality information, however, is a rationed commodity for health professionals also, and the use of different standards in its dissemination represents a major determinant of the failure of the therapeutic chain.1 Healthcare systems make limited investments to provide independent information, and pharmaceutical companies—who fund most clinical research—therefore become major players in the dissemination of information to health professionals and the public. Do pharmaceutical companies and the researchers acting as opinion leaders for them behave fairly and consistently or do they adopt double standards when they write in peer reviewed journals and talk to practitioners? We know that this form of information asymmetry exists.2 Two recent examples—a document from the European Federation of Pharmaceutical Companies3 and the debate generated by the ALLHAT study, a landmark trial in the treatment of hypertension—illustrate this danger.4–8

The document from the European Federation of Pharmaceutical Companies identifies 20 diseases and conditions, such as dementia, asthma, hepatitis C, rheumatoid arthritis, some …

https://www.bmj.com/content/bmj/326/7400/1156.full.pdf

Alessandro Liberati

Papers

An unfinished trip through uncertainties

A European Project assessing the appropriateness of hospital utilization: Background, objectives and preliminary results

Reliability study of the European appropriateness evaluation protocol.

Physicians' view of practice guidelines. A survey of Italian physicians

Information from drug companies and opinion leaders