Objective: Funnel plots (plots of effect estimates against sample size) may be useful to detect bias in meta-analyses that were later contradicted by large trials. We examined whether a simple test of asymmetry of funnel plots predicts discordance of results when meta-analyses are compared to large trials, and we assessed the prevalence of bias in published meta-analyses. Design: Medline search to identify pairs consisting of a meta-analysis and a single large trial (concordance of results was assumed if effects were in the same direction and the meta-analytic estimate was within 30% of the trial); analysis of funnel plots from 37 meta-analyses identified from a hand search of four leading general medicine journals 1993-6 and 38 meta-analyses from the second 1996 issue of the Cochrane Database of Systematic Reviews . Main outcome measure: Degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates against precision. Results: In the eight pairs of meta-analysis and large trial that were identified (five from cardiovascular medicine, one from diabetic medicine, one from geriatric medicine, one from perinatal medicine) there were four concordant and four discordant pairs. In all cases discordance was due to meta-analyses showing larger effects. Funnel plot asymmetry was present in three out of four discordant pairs but in none of concordant pairs. In 14 (38%) journal meta-analyses and 5 (13%) Cochrane reviews, funnel plot asymmetry indicated that there was bias. Conclusions: A simple analysis of funnel plots provides a useful test for the likely presence of bias in meta-analyses, but as the capacity to detect bias will be limited when meta-analyses are based on a limited number of small trials the results from such analyses should be treated with considerable caution. Key messages Systematic reviews of randomised trials are the best strategy for appraising evidence; however, the findings of some meta-analyses were later contradicted by large trials Funnel plots, plots of the trials9 effect estimates against sample size, are skewed and asymmetrical in the presence of publication bias and other biases Funnel plot asymmetry, measured by regression analysis, predicts discordance of results when meta-analyses are compared with single large trials Funnel plot asymmetry was found in 38% of meta-analyses published in leading general medicine journals and in 13% of reviews from the Cochrane Database of Systematic Reviews Critical examination of systematic reviews for publication and related biases should be considered a routine procedure

/pdf/bias-in-meta-analysis-detected-by-a-simple-graphical-test-5erh428s5c.pdf

Bias in meta-analysis detected by a simple, graphical test

Meta-Analysis in Clinical Trials*

Systematic review and meta-analysis

The Cochrane Handbook for Systematic Reviews of Interventions is the official document that describes in detail the process of preparing and maintaining Cochrane systematic reviews on the effects of healthcare interventions.

/pdf/cochrane-handbook-for-systematic-reviews-of-interventions-1xbdy6uvru.pdf

Cochrane Handbook for Systematic Reviews of Interventions

During the past 30 years, meta-analysis has been an indispensable tool for revealing the hidden meaning of our research literatures. The four articles in this special section on meta-analysis illus...

Meta-Analysis: A Constantly Evolving Research Integration Tool

/pdf/introduction-to-meta-analysis-23lgepcoc0.pdf

Introduction to Meta-Analysis

Preface. Introduction. Data Sets. Tests of Statistical Significance of Combined Results. Vote-Counting Methods. Estimation of a Single Effect Size: Parametric and Nonparametric Methods. Parametric Estimation of Effect Size from a Series of Experiments. Fitting Parametric Fixed Effect Models to Effect Sizes: Categorical Methods. Fitting Parametric Fixed Effect Models to Effect Sizes: General Linear Models. Random Effects Models for Effect Sizes. Multivariate Models for Effect Sizes. Combining Estimates of Correlation Coefficients. Diagnostic Procedures for Research Synthesis Models. Clustering Estimates of Effect Magnitude. Estimation of Effect Size When Not All Study Outcomes Are Observed. Meta-Analysis in the Physical and Biological Sciences. Appendix. References. Index.

Statistical Methods for Meta-Analysis

Statistical Methods for Meta-Analysis.

There are two popular statistical models for meta-analysis, the fixed-effect model and the random-effects model. The fact that these two models employ similar sets of formulas to compute statistics, and sometimes yield similar estimates for the various parameters, may lead people to believe that the models are interchangeable. In fact, though, the models represent fundamentally different assumptions about the data. The selection of the appropriate model is important to ensure that the various statistics are estimated correctly. Additionally, and more fundamentally, the model serves to place the analysis in context. It provides a framework for the goals of the analysis as well as for the interpretation of the statistics. In this paper we explain the key assumptions of each model, and then outline the differences between the models. We conclude with a discussion of factors to consider when choosing between the two models. Copyright © 2010 John Wiley & Sons, Ltd.

/pdf/a-basic-introduction-to-fixed-effect-and-random-effects-3ipc6fyxyn.pdf

A basic introduction to fixed‐effect and random‐effects models for meta‐analysis

Glass's estimator of effect size, the sample mean difference divided by the sample standard deviation, is studied in the context of an explicit statistical model. The exact distribution of Glass's estimator is obtained and the estimator is shown to have a small sample bias. The minimum variance unbiased estimator is obtained and shown to have uniformly smaller variance than Glass's (biased) estimator. Measurement error is shown to attenuate estimates of effect size and a correction is given. The effects of measurement invalidity are discussed. Expressions for weights that yield the most precise weighted estimate of effect size are also derived.

Larry V. Hedges

Papers

Introduction to Meta-Analysis

Statistical Methods for Meta-Analysis

Statistical Methods for Meta-Analysis.

A basic introduction to fixed‐effect and random‐effects models for meta‐analysis

Distribution Theory for Glass's Estimator of Effect size and Related Estimators: