David Moher and colleagues introduce PRISMA, an update of the QUOROM guidelines for reporting systematic reviews and meta-analyses

/pdf/preferred-reporting-items-for-systematic-reviews-and-meta-3k0wa2mp7o.pdf

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement

Systematic reviews and meta-analyses have become increasingly important in health care. Clinicians read them to keep up to date with their field,1,2 and they are often used as a starting point for developing clinical practice guidelines. Granting agencies may require a systematic review to ensure there is justification for further research,3 and some health care journals are moving in this direction.4 As with all research, the value of a systematic review depends on what was done, what was found, and the clarity of reporting. As with other publications, the reporting quality of systematic reviews varies, limiting readers' ability to assess the strengths and weaknesses of those reviews.

Several early studies evaluated the quality of review reports. In 1987, Mulrow examined 50 review articles published in 4 leading medical journals in 1985 and 1986 and found that none met all 8 explicit scientific criteria, such as a quality assessment of included studies.5 In 1987, Sacks and colleagues6 evaluated the adequacy of reporting of 83 meta-analyses on 23 characteristics in 6 domains. Reporting was generally poor; between 1 and 14 characteristics were adequately reported (mean = 7.7; standard deviation = 2.7). A 1996 update of this study found little improvement.7

In 1996, to address the suboptimal reporting of meta-analyses, an international group developed a guidance called the QUOROM Statement (QUality Of Reporting Of Meta-analyses), which focused on the reporting of meta-analyses of randomized controlled trials.8 In this article, we summarize a revision of these guidelines, renamed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses), which have been updated to address several conceptual and practical advances in the science of systematic reviews (Box 1).



Box 1

Conceptual issues in the evolution from QUOROM to PRISMA

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.

Cochrane Reviews have recently started including the quantity I 2 to help readers assess the consistency of the results of studies in meta-analyses. What does this new quantity mean, and why is assessment of heterogeneity so important to clinical practice? 

Systematic reviews and meta-analyses can provide convincing and reliable evidence relevant to many aspects of medicine and health care.1 Their value is especially clear when the results of the studies they include show clinically important effects of similar magnitude. However, the conclusions are less clear when the included studies have differing results. In an attempt to establish whether studies are consistent, reports of meta-analyses commonly present a statistical test of heterogeneity. The test seeks to determine whether there are genuine differences underlying the results of the studies (heterogeneity), or whether the variation in findings is compatible with chance alone (homogeneity). However, the test is susceptible to the number of trials included in the meta-analysis. We have developed a new quantity, I 2, which we believe gives a better measure of the consistency between trials in a meta-analysis.

Assessment of the consistency of effects across studies is an essential part of meta-analysis. Unless we know how consistent the results of studies are, we cannot determine the generalisability of the findings of the meta-analysis. Indeed, several hierarchical systems for grading evidence state that the results of studies must be consistent or homogeneous to obtain the highest grading.2–4

Tests for heterogeneity are commonly used to decide on methods for combining studies and for concluding consistency or inconsistency of findings.5 6 But what does the test achieve in practice, and how should the resulting P values be interpreted?

A test for heterogeneity examines the null hypothesis that all studies are evaluating the same effect. The usual test statistic …

/pdf/measuring-inconsistency-in-meta-analyses-4iysmttp08.pdf

Measuring inconsistency in meta-analyses

Objective: Funnel plots (plots of effect estimates against sample size) may be useful to detect bias in meta-analyses that were later contradicted by large trials. We examined whether a simple test of asymmetry of funnel plots predicts discordance of results when meta-analyses are compared to large trials, and we assessed the prevalence of bias in published meta-analyses. Design: Medline search to identify pairs consisting of a meta-analysis and a single large trial (concordance of results was assumed if effects were in the same direction and the meta-analytic estimate was within 30% of the trial); analysis of funnel plots from 37 meta-analyses identified from a hand search of four leading general medicine journals 1993-6 and 38 meta-analyses from the second 1996 issue of the Cochrane Database of Systematic Reviews . Main outcome measure: Degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates against precision. Results: In the eight pairs of meta-analysis and large trial that were identified (five from cardiovascular medicine, one from diabetic medicine, one from geriatric medicine, one from perinatal medicine) there were four concordant and four discordant pairs. In all cases discordance was due to meta-analyses showing larger effects. Funnel plot asymmetry was present in three out of four discordant pairs but in none of concordant pairs. In 14 (38%) journal meta-analyses and 5 (13%) Cochrane reviews, funnel plot asymmetry indicated that there was bias. Conclusions: A simple analysis of funnel plots provides a useful test for the likely presence of bias in meta-analyses, but as the capacity to detect bias will be limited when meta-analyses are based on a limited number of small trials the results from such analyses should be treated with considerable caution. Key messages Systematic reviews of randomised trials are the best strategy for appraising evidence; however, the findings of some meta-analyses were later contradicted by large trials Funnel plots, plots of the trials9 effect estimates against sample size, are skewed and asymmetrical in the presence of publication bias and other biases Funnel plot asymmetry, measured by regression analysis, predicts discordance of results when meta-analyses are compared with single large trials Funnel plot asymmetry was found in 38% of meta-analyses published in leading general medicine journals and in 13% of reviews from the Cochrane Database of Systematic Reviews Critical examination of systematic reviews for publication and related biases should be considered a routine procedure

/pdf/bias-in-meta-analysis-detected-by-a-simple-graphical-test-5erh428s5c.pdf

Bias in meta-analysis detected by a simple, graphical test

Meta-Analysis in Clinical Trials*

The probability of failure of a structure or structural element subjected to wind forces depends, in large part, on the distribution of extreme wind speeds acting on the structure. In the past, distributions of extreme wind speeds were based on extreme wind data without regard to wind direction, and probabilities of failure were computed accordingly. A method is presented herein for calculating failure probabilities by using directional wind speed data as obtained from weather station records. The method takes advantage of the weak correlations found among wind speeds from different directions and the property of extreme value random variables that zero correlation implies statistical independence. The method is applicable to any type of structure, including structures exhibiting aerodynamic amplification or aeroelastic effects. In addition, it is shown that, in practice, the necessary distributions can be estimated almost as accurately from data obtained from readily available published documents as from...

Multivariate Distributions of Directional Wind Speeds

A matrix variance inequality

Abstract : Suppose that the p-dimensional random row vector x has a multivariate normal distribution with mean mu and covariance matrix Sigma. Wilks (Ann. Math. Statist. 17 (1946)) has derived likelihood-ratio tests of the hypothesis H sub mvc and H sub vc against general alternatives H. Here H sub mvc is the hypothesis that the components of the mean vector mu are equal (mu = eta(1, 1, ..., 1) is identical with eta e, where eta is an unknown constant) and that Sigma has the intraclass correlational form Sigma = sigma squared ((1-rho)I + rho e e prime). H sub vc is the hypothesis that Sigma has the intraclass correlational form, mu unrestricted; H is the hypothesis that mu is unrestricted and Sigma is unrestricted, positive definite. In the present paper, we consider likelihood ratio tests of the hypotheses Hr sub mvc and Hr sub vc against general alternatives H, where Hr sub mvc and Hr sub vc differ from H sub mvc and H sub vc, respectively, by constricting the common correlation to lie in an interval rho sub 0 = or < rho < 1. Such tests have practical importance in psychological testing theory, in the analysis of growth curves, and in other contexts.

Testing for equality of means, equality of variances, and equality of covariances under restrictions upon the parameter space

/pdf/multivariate-non-normal-distributions-and-models-of-23s3d87asw.pdf

Multivariate non-normal distributions and models of dependency

Let Z = (zij) be arandomp xp symmetricmatrixwith EZ = A, i.e. Ezij = aj (i,j= 1,... ,p). An application to the theory of response surface estimation led van der Vaart (1961) to consider the expectation-bias and median-bias of the characteristic roots of Z as estimators of the characteristic roots of A. Denote the (real) characteristic roots of a matrix X by A(X) with the ordering A1(X) > ... > Ap(X). Van der Vaart proved that if Z is a symmetric matrix, then EA1(Z) > A1(A) and EAp(Z) O} = P{trC(Z -A) A1(A)} > 1 and P{Ap(Z) 2. If absolute continuity is assumed, these bias inequalities become strict. The purpose of this paper is to show how to generate a wide class of inequalities between EA(Z) and A(EZ). In particular, some of the results of van der Vaart (1961) can be strengthened by a weakening of the assumption that Z be a symmetric matrix. When Z is symmetric, the expectation-bias and median-bias may also be obtained for partial sums of the roots, and when Z is positive definite, the expectation-bias is obtained for more general functions of the roots. Expectation-bias for the roots of the determinantal equation IZl Z21 = 0 is presented; of particular interest is the determinantal equation for the canonical correlations. Inequalities for the median-bias of certain linear combinations of the roots are also developed. These inequalities are obtained as direct consequences of known results concerning the convexity of scalar functions of a matrix. An extension of Jensen's inequality to convex matrix functions using the Loewner ordering for matrices is also given; this extension then serves as a source of other inequalities.

Ingram Olkin

Papers

Multivariate Distributions of Directional Wind Speeds

A matrix variance inequality

Testing for equality of means, equality of variances, and equality of covariances under restrictions upon the parameter space

Multivariate non-normal distributions and models of dependency

On the bias of functions of characteristic roots of a random matrix