Showing papers on "Funnel plot published in 2006"

Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments

Abstract: Preface Acknowledgements Notes on Contributors Chapter 1: Publication Bias in Meta-Analysis (Hannah R Rothstein, Alexander J Sutton and Michael Borenstein) Part A: Publication bias in context Chapter 2: Publication Bias: Recognizing the Problem, Understanding Its Origins and Scope, and Preventing Harm (Kay Dickersin) Chapter 3: Preventing Publication Bias: Registries and Prospective Meta-Analysis (Jesse A Berlin and Davina Ghersi) Chapter 4: Grey Literature and Systematic Reviews (Sally Hopewell, Mike Clarke and Sue Mallett) Part B: Statistical methods for assessing publication bias Chapter 5: The Funnel Plot (Jonathan AC Sterne, Betsy Jane Becker and Matthias Egger) Chapter 6: Regression Methods to Detect Publication and Other Bias in Meta-Analysis (Jonathan AC Sterne and Matthias Egger) Chapter 7: Failsafe N or File-Drawer Number (Betsy Jane Becker) Chapter 8: The Trim and Fill Method (Sue Duval) Chapter 9: Selection Method Approaches (Larry V Hedges and Jack Vevea) Chapter 10: Evidence Concerning the Consequences of Publication and Related Biases (Alexander J Sutton) Chapter 11: Software for Publication Bias (Michael Borenstein) Part C: Advanced and emerging approaches Chapter 12: Bias in Meta-Analysis Induced by Incompletely Reported Studies (Alexander J Sutton and Therese D Pigott) Chapter 13: Assessing the Evolution of Effect Sizes over Time (Thomas A Trikalinos and John PA Ioannidis) Chapter 14: Do Systematic Reviews Based on Individual Patient Data Offer a Means of Circumventing Biases Associated with Trial Publications? (Lesley Stewart, Jayne Tierney and Sarah Burdett) Chapter 15: Differentiating Biases from Genuine Heterogeneity: Distinguishing Artifactual from Substantive Effects (John PA Ioannidis) Chapter 16: Beyond Conventional Publication Bias: Other Determinants of Data Suppression (Scott D Halpern and Jesse A Berlin) Appendices Appendix A: Data Sets Appendix B: Annotated Bibliography (Hannah R Rothstein and Ashley Busing) Glossary Index

A modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints

Abstract: Publication bias and related bias in meta-analysis is often examined by visually checking for asymmetry in funnel plots of treatment effect against its standard error Formal statistical tests of funnel plot asymmetry have been proposed, but when applied to binary outcome data these can give false-positive rates that are higher than the nominal level in some situations (large treatment effects, or few events per trial, or all trials of similar sizes) We develop a modified linear regression test for funnel plot asymmetry based on the efficient score and its variance, Fisher's information The performance of this test is compared to the other proposed tests in simulation analyses based on the characteristics of published controlled trials When there is little or no between-trial heterogeneity, this modified test has a false-positive rate close to the nominal level while maintaining similar power to the original linear regression test ('Egger' test) When the degree of between-trial heterogeneity is large, none of the tests that have been proposed has uniformly good properties

The case of the misleading funnel plot.

Abstract: Evidence based medicine insists on rigorous standards to appraise clinical interventions. Failure to apply the same rules to its own tools could be equally damaging

Software for Publication Bias

Abstract: KEY POINTS • Various procedures for addressing publication bias are discussed elsewhere in this volume. The goal of this chapter is to show how these different procedures fit into an overall strategy for addressing bias, and to discuss computer programs that can be used to implement this strategy. • To address publication bias the researcher should proceed through a logical sequence of analyses. First, forest plots and funnel plots provide a visual sense of the data. Then, rank correlation and regression procedures ask whether or not there is evidence of bias, while the failsafe N and its variants ask if the entire effect size may be attributed to bias. Finally, trim and fill and the cumulative forest plot offer a more nuanced perspective, and ask how the effect size would shift if the apparent bias were to be removed. • The computer programs discussed are Comprehensive Meta Analysis, Stata, MetaWin, and RevMan. We show how the researcher would address publication bias using each of these programs in turn. The chapter closes with a subjective assessment of each program’s strengths and weaknesses. • The appendix includes a list of additional resources, including web links to data sets and to step-by-step instructions for running the publication bias procedures with each of the programs.

Helicobacter pylori infection and colorectal cancer risk: a meta-analysis

Abstract: BACKGROUND: Several studies suggested an association between Helicobacter pylori infection and colorectal carcinoma or adenoma risk. However, different authors reported quite varying estimates. We carried out a systematic review and meta-analysis of published studies investigating this association and paid special attention to the possibility of publication bias and sources of heterogeneity between studies. Materials and METHODS: An extensive literature search and cross-referencing were performed to identify all published studies. Summary estimates were obtained using random-effects models. The presence of possible publication bias was assessed using different statistical approaches. RESULTS: In a meta-analysis of the 11 identified human studies, published between 1991 and 2002, a summary odds ratio of 1.4 (95% CI, 1.1-1.8) was estimated for the association between H. pylori infection and colorectal cancer risk. The graphical funnel plot appeared asymmetrical, but the formal statistical evaluations did not provide strong evidence of publication bias. The proportion of variation of study results because of heterogeneity was small (36.5%). CONCLUSIONS: The results of our meta-analysis are consistent with a possible small increase in risk of colorectal cancer because of H. pylori infection. However, the possibility of some publication bias cannot be ruled out, although it could not be statistically confirmed. Larger, better designed and better controlled studies are needed to clarify the situation.

Association between Agent Orange and birth defects: systematic review and meta-analysis

Abstract: The association between parental exposure to Agent Orange or dioxin and birth defects is controversial due to inconsistent findings in the literature The principal aim of this study was to conduct a meta-analysis of relevant epidemiological studies that examined this association and to assess the heterogeneity among studies Relevant studies were identified through a computerized literature search of Medline and Embase from 1966 to 2002; reviewing the reference list of retrieved articles and conference proceedings; and contacting researchers for unpublished studies A specified protocol was followed to extract data on study details and outcomes Both fixed-effects and random-effects models were used to synthesize the results of individual studies The Cochrane Q test and index of heterogeneity (I2) were used to evaluate heterogeneity and a funnel plot and Eggers test were used to evaluate publication bias In total 22 studies including 13 Vietnamese and nine non-Vietnamese studies were identified The summary relative risk (RR) of birth defects associated with exposure to Agent Orange was 195 [95% confidence interval (95% CI) 159--239] with substantial heterogeneity across studies Vietnamese studies showed a higher summary RR (RR 5 300; 95% CI 219--412) than non-Vietnamese studies (RR 5 129; 95% CI 104--159) Sub-group analyses found that the magnitude of association tended to increase with greater degrees of exposure to Agent Orange rated on intensity and duration of exposure and dioxin concentrations measured in affected populations Parental exposure to Agent Orange appears to be associated with an increased risk of birth defects (authors)

Update of a meta-analysis on lung cancer and welding.

Abstract: Objectives This study is an update, over the period 1954–2004, of a previous meta-analysis completed in 1994. It was aimed at assessing lung cancer risk among welders, while addressing heterogeneity, publication bias, and confounding issues. Methods Combined relative risks (CRR) and their variances were calculated using fixed and random effects models. Heterogeneity was tested using the Q statistic. The publication bias was estimated using funnel plots, and Egger’s regression and partially controlled by excluding studies with positive reporting bias. Results The literature provided 60 studies eligible for the meta-analysis. No heterogeneity was observed. The fixed effect CRR for all of the welders and all of the studies was 1.26 (95% CI 1.20–1.32) after partial control of publication bias. No difference was observed according to welding activities. Smoking did not appear to be a marked confounder in the relationship between lung cancer and welding, but the effect of asbestos could not be assessed. Conclusions The meta-analysis showed a 26% excess of lung cancer for welders without any difference according to welding activities.

On Knowing What We Do Not Know: An Empirical Comparison of Methods to Detect Publication Bias in Meta-Analysis.

Abstract: The performance of methods for detecting publication bias in meta-analysis was evaluated using Monte Carlo methods. Four methods of bias detection were investigated: Begg’s rank correlation, Egger’s regression, funnel plot regression, and trim and fill. Five factors were included in the simulation design: number of primary studies in each meta-analysis, sample sizes of primary studies, population variances in primary studies, magnitude of population effect size, and magnitude of selection bias. Results were evaluated in terms of Type I error control and statistical power. Results suggest poor Type I error control in many conditions for all of the methods examined. One exception was the Begg’s rank correlation method using sample size rather than the estimated variance. Statistical power was typically very low for conditions in which Type I error rates were adequately controlled, although power increased with larger sample sizes in the primary studies and larger numbers of studies in the meta-analysis.

Meta-analysis of glutathione S-transferase M1 genotype and risk toward head and neck cancer.

Abstract: Background. We sought to evaluate the estimated risk of GSTM1 null genotype and squamous cell carcinoma of the head and neck (SCCHN). Methods. The studies done so far for GSTM1 null genotype as a risk factor associated with head and neck cancer are not conclusive and have shown conflicting results. A meta-analysis has been carried out on 22 case-control studies collected from the literature. Precision of the risk estimate and presence of publication bias in those studies are also evaluated by funnel plot analysis. Results. A total of 3527 cases and 4211 controls constitute the subjects of meta-analysis. The principal outcome measured was the odds ratio (OR) for the risk of head and neck cancer. Pooling the studies together, the OR was 1.50 (95% confidence interval [CI], 1.21–1.87). When stratified according to Asians and Caucasians, the OR increased in the Asians (OR, 1.93; 95% CI, 1.29–2.90), suggesting that the risk is more pronounced in Asians. However, a substantial amount of publication bias is also associated with studies conducted on the Asian population. Conclusion. A review of the 22 case-control studies for risk of SCCHN and GSTM1 null genotype indicate that greater attention should be paid to designing future studies so that a more precise risk estimate can be achieved. © 2005 Wiley Periodicals, Inc. Head Neck27: XXX–XXX, 2005

Funnel plots, performance variation and the Myocardial Infarction National Audit Project 2003-2004.

Abstract: Clinical governance requires health care professionals to improve standards of care and has resulted in comparison of clinical performance data. The Myocardial Infarction National Audit Project (a UK cardiology dataset) tabulates its performance. However funnel plots are the display method of choice for institutional comparison. We aimed to demonstrate that funnel plots may be derived from MINAP data and allow more meaningful interpretation of data. We examined the attainment of National Service Framework standards for all hospitals (n = 230) and all patients (n = 99,133) in the MINAP database between 1st April 2003 and 31st March 2004. We generated funnel plots (with control limits at 3 sigma) of Door to Needle and Call to Needle thrombolysis times, and the use of aspirin, beta-blockers and statins post myocardial infarction. Only 87,427 patients fulfilled criteria for analysis of the use of secondary prevention drugs and 15,111 patients for analysis by Door to Needle and Call to Needle times (163 hospitals achieved the standards for Door to Needle times and 215 were within or above their control limits). One hundred and sixteen hospitals fell outside the 'within 25%' and 'more than 25%' standards for Call to Needle times, but 28 were below the lower control limits. Sixteen hospitals failed to reach the standards for aspirin usage post AMI and 24 remained below the lower control limits. Thirty hospitals were below the lower CL for beta-blocker usage and 49 outside the standard. Statin use was comparable. Funnel plots may be applied to a complex dataset and allow visual comparison of data derived from multiple health-care units. Variation is readily identified permitting units to appraise their practices so that effective quality improvement may take place.

PUB_BIAS: A SAS ® Macro for Detecting Publication Bias in Meta-Analysis

Abstract: Publication bias is one threat to validity that researchers conducting meta-analysis studies confront. Although statistical methods for detecting publication bias have surfaced in the literature (e.g., Begg Rank Correlation, Egger Regression, Funnel Plot Regression, and Trim & Fill), many researchers rely on visual inspection of funnel plots. This program was created to provide meta-analysts with the ability to implement statistical methods to detect publication bias. Statistical information from the corpus of studies (i.e., effect sizes and sample sizes) is input in this SAS macro and analyzed for potential publication bias using the above mentioned statistical methods. In addition to the p-value associated with each detection method the program output provides the estimated mean effect size and random effects variance component. The application of the SAS/IML programming on two sets of example data (with and without publication bias) is provided along with the macro programming language.

Using Journal Impact Factors to Correct for the Publication Bias of Medical Studies

Abstract: Publication bias of the results of medical studies can invalidate evidence-based medicine. The existing methodology for modeling this essentially relies upon the symmetry of the funnel plot. We present a new method of modeling publication bias that uses this information plus the impact factors of the publishing journals. A simple model of the publication process enables the estimation of bias-corrected intervention effects. The procedure is illustrated using a meta-analysis of the effectiveness of single-dose oral aspirin for acute pain, and results are also obtained for five other meta-analyses. The method enables the fitting of a wide range of models and is considered more flexible than other ways of compensating for publication bias. The model also provides the basis of a statistical test for the existence of publication bias. Use of the new methodology to supplement existing methods is recommended, in the context of a sensitivity analysis.

Detecting publication bias in random effects meta-analysis: An empirical comparison of statistical methods

Abstract: Publication bias is one threat to validity that researchers conducting meta-analysis studies confront Two primary goals of this research were to examine the degree to which publication bias impacts the results of a random effects meta-analysis and to investigate the performance of five statistical methods for detecting publication bias in random effects meta-analysis Specifically, the difference between the population effect size and the estimated meta-analysis effect size, as well as the difference between the population effect size variance and the meta-analysis effect size variance, provided an indication of the impact of publication bias In addition, the performance of five statistical methods for detecting publication bias (Begg Rank Correlation with sample size, Begg Rank Correlation with variance, Egger Regression, Funnel Plot Regression, and Trim and Fill) were estimated with Type I error rates and statistical power The overall findings indicate that publication bias notably impacts the meta-analysis effect size and variance estimates Poor Type I error control was exhibited in many conditions by most of the statistical methods Even when Type I error rates were adequate the power was small, even with larger samples and greater numbers of studies in the meta-analysis 1 CHAPTER ONE: INTRODUCTION

Evidence-based research in alternative protocols to dental implantology: a closer look at publication bias.

Abstract: Several techniques exist for the surgical placement of dental implants. The aim of this study was to assess systematically, the efficacy of these protocols by the evidence-based perspective. Five best-case studies involving 607 early/immediately loaded implants and 300 conventionally loaded implants were identified by examining the available literature and rigorous inclusion/exclusion criteria. Overall analyses demonstrated a 98.4 percent success rate for the early/immediate procedure and a 95.3 percent for the conventional protocol. Success rates in the articles reviewed were based on implant survival over a follow-up period of between one to two years. A meta-analysis was generated to evaluate the presented evidence and to aid in decision-making. Despite its common implementation, this technique presents many caveats, among which publication bias is one of the most common. To investigate the possible presence of publication bias, a funnel plot analysis complemented several statistical tests. By means of the systematic investigation of dental implants, the authors' results confirm the presence of publication bias in implant dentistry literature, which strongly suggests that clinicians ought not base their decisions solely on the results presented by a few published studies. Rather, it is recommended that clinicians cautiously draw conclusions and seek studies that present accountable and clinically relevant results. Furthermore, it is suggested that clinicians attend seminars to learn of the effective advances in evidence-based dentistry, so as to develop the ability to easily detect inadequate literature due to attempted correlation with the most current research. It is also recommended that additional research is necessary to analyze which fields of research are more prone to bias, thus forewarning clinicians before formulating clinical conclusions.

ORIGINAL ARTICLES Association of Industry Sponsorship to Published Outcomes in Gastrointestinal Clinical Research

Abstract: Background & Aims: Recent years have seen an increase in industry sponsorship of clinical trials throughout medicine. We conducted a study to evaluate the association of industry spon- sorship to published outcomes in gastrointestinal (GI) clinical research. Our aims were (1) to evaluate the trends in the source of funding for GI clinical research during the period from 1992 to 2002-2003, (2) to determine whether the source of study funding predicted the likelihood that a study would publish results that favor the drug or device being tested, and (3) to determine whether differences exist in the methodologic quality of the investigational study methods used in studies funded by private industry versus other sources. Methods: We selected all clinical studies evaluating a drug or device from 4 prominent GI journals (Gastroenterology, The American Journal of Gastroenter- ology, Hepatology, and Gastrointestinal Endoscopy). All studies were abstracted by using a standardized data abstraction form. We evaluated the trends in funding source for studies published during the years 1992, 2002, and 2003. All selected studies were scored for methodologic quality by using a previously validated scoring system. The percentage of studies that reported out- comes that favored the device or drug being tested against the standard therapy or placebo was determined for each funding source. A funnel plot was constructed to assess for the presence of negative publication bias. Results: A total of 6326 studies were reviewed. For the 1992 trend data, 1860 studies were reviewed, and 135 were selected for inclusion in the study. Ninety-five studies were studies involving the investigation of drugs, and 40 studies involved the testing of a device. For the 2002-2003 data 4466 studies were reviewed, and 315 were selected for inclusion in the study. Two hundred twenty-two studies were clinical trials involving the investigation of drugs, and 93 were clinical trials involving devices. In comparing 1992 to 2002, the percentage of studies funded by industry sources more than doubled from 10% to greater than 28% of the total studies assessed. There was an associated decline in the propor- tion of studies with funding from non-industry sources during this period (62% to less than 48%). The percentage of studies that did not disclose a funding source fell modestly from 28% to 24%. We found that 86% of studies funded by private indus- try reported a result favorable to the study drug or device, and 83% of studies funded by academic sources reported a result favorable to the study drug or device (P.572). On average, studies funded by private industry had a higher methodology score than studies funded by traditional academic sources (75 of 100 vs 65 of 100; P.005). Analysis of the funnel plot did not reveal evidence of bias against the publication of small studies with insignificant results. Conclusions: The propor- tion of research funded by industry has more than doubled during the last decade and currently comprises almost half of the funding for GI clinical research. Industry-sponsored studies are, on average, of superior methodologic quality to studies funded by other sources. Industry-sponsored studies in leading GI journals were no more likely than other studies to publish results that favored the study sponsor, although an extremely high percentage of all studies in these journals reported positive results. There has been only a modest decline in studies not acknowledging a funding source.

What to do with outliers

Abstract: The paper by Spiegelhalter is a valuable contribution to the literature on presenting and displaying performance related outcome measures.1 It provides further methodological guidance on identifying service providers whose performance falls outside control limits using funnel plot methodology. When reporting on performance it is important to have procedures in place which should be followed when outliers are identified. These issues have been considered by the Paediatric Intensive Care Audit Network (PICANet) who use the funnel plot methodology for reporting risk adjusted mortality from all paediatric intensive care units (PICU) in England and Wales. Before producing these funnel plots for the latest …