scispace - formally typeset
Search or ask a question

Showing papers on "Funnel plot published in 2005"


Journal ArticleDOI
TL;DR: The effective sample size funnel plot and associated regression test of asymmetry should be used to detect publication bias and other sample size related effects in meta-analyses of test accuracy.

2,191 citations


Journal ArticleDOI
TL;DR: It is concluded that funnel plots are flexible, attractively simple, and avoid spurious ranking of institutions into 'league tables'.
Abstract: 'Funnel plots' are recommended as a graphical aid for institutional comparisons, in which an estimate of an underlying quantity is plotted against an interpretable measure of its precision. 'Control limits' form a funnel around the target outcome, in a close analogy to standard Shewhart control charts. Examples are given for comparing proportions and changes in rates, assessing association between outcome and volume of cases, and dealing with over-dispersion due to unmeasured risk factors. We conclude that funnel plots are flexible, attractively simple, and avoid spurious ranking of institutions into 'league tables'.

752 citations


Journal ArticleDOI
T. D. Stanley1
TL;DR: In this paper, the authors consider several meta-regression and graphical methods that can differentiate genuine empirical effect from publication bias, and apply them to four areas of empirical economics research: minimum wage effects, union productivity effects, price elasticity, and natural rate hypothesis.
Abstract: . This review considers several meta-regression and graphical methods that can differentiate genuine empirical effect from publication bias. Publication selection exists when editors, reviewers, or researchers have a preference for statistically significant results. Because all areas of empirical research are susceptible to publication selection, any average or tally of significant/insignificant studies is likely to be biased and potentially misleading. Meta-regression analysis can see through the murk of random sampling error and selected misspecification bias to identify the underlying statistical structures that characterize genuine empirical effect. Meta-significance testing and precision-effect testing (PET) are offered as a means to identify empirical effect beyond publication bias and are applied to four areas of empirical economics research – minimum wage effects, union-productivity effects, price elasticities, and tests of the natural rate hypothesis.

528 citations


Journal ArticleDOI
TL;DR: Researchers who assess for publication bias using the funnel plot may be misled by its shape, and authors and readers of systematic reviews need to be aware of the limitations of the funnel Plot.

438 citations


Journal ArticleDOI
TL;DR: There is a tendency for cluster trials, with evidence methodological biases, to also show an age imbalance between treatment groups, and it is shown that all cluster trials show a large positive effect of hip protectors whilst individually randomised trials shows a range of positive and negative effects, suggesting that cluster trials may be producing a biased estimate of effect.
Abstract: Cluster randomised trials can be susceptible to a range of methodological problems. These problems are not commonly recognised by many researchers. In this paper we discuss the issues that can lead to bias in cluster trials. We used a sample of cluster randomised trials from a recent review and from a systematic review of hip protectors. We compared the mean age of participants between intervention groups in a sample of 'good' cluster trials with a sample of potentially biased trials. We also compared the effect sizes, in a funnel plot, between hip protector trials that used individual randomisation compared with those that used cluster randomisation. There is a tendency for cluster trials, with evidence methodological biases, to also show an age imbalance between treatment groups. In a funnel plot we show that all cluster trials show a large positive effect of hip protectors whilst individually randomised trials show a range of positive and negative effects, suggesting that cluster trials may be producing a biased estimate of effect. Methodological biases in the design and execution of cluster randomised trials is frequent. Some of these biases associated with the use of cluster designs can be avoided through careful attention to the design of cluster trials. Firstly, if possible, individual allocation should be used. Secondly, if cluster allocation is required, then ideally participants should be identified before random allocation of the clusters. Third, if prior identification is not possible, then an independent recruiter should be used to recruit participants.

276 citations


Journal ArticleDOI
TL;DR: Egger's linear regression method and Begg’s method had stronger statistical and discriminatory powers than Macaskill's method for detecting publication bias given the same type I error level.

257 citations


Journal ArticleDOI
TL;DR: A large number of women of childbearing age suffer from depression and relatively little is known about the safety of antidepressant use during pregnancy, so it is necessary to select patients suitable for antidepressant use.
Abstract: Background A substantial number of women of childbearing age suffer from depression. Despite this, relatively little is known about the safety of antidepressant use during pregnancy. Purpose We conducted a meta-analysis of prospective comparative cohort studies to quantify the relationship between maternal exposure to the newer antidepressants and major malformations. Methods We searched Medline, Embase and Reprotox from 1996 to the present for studies comparing outcomes in first trimester exposures to citalopram, escitalopram, fluoxetine, fluvoxamine, paroxetine, sertraline, reboxetine, venlafaxine, nefazodone, trazodone, mirtazapine and bupropion to those of non-exposed mothers. Data were combined using a random effects model; heterogeneity was tested with χ2, and publication bias with a funnel plot and the Begg–Mazumdar statistic. Results Twenty-two studies were identified, 15 were rejected (4 reviews, 4 without comparison groups, 2 third trimester exposures, 2 retrospective database studies, 2 case reports and 1 duplicate); 7 studies (n = 1774) met inclusion criteria. Effects were not heterogeneous (χ2 = 2.04, p = 0.92); funnel plot and test (τ = −0.24, p = 0.45) indicated no publication bias. The summary relative risk was 1.01 (95%CI: 0.57–1.80). Conclusions As a group, the newer antidepressants are not associated with an increased risk of major malformations above the baseline of 1–3% in the population. Copyright © 2005 John Wiley & Sons, Ltd.

226 citations


Posted Content
TL;DR: In this article, the authors explore publication bias in economic growth literature by means of traditional funnel plots, meta-significance testing, as well as by bootstrapping these meta-Significance tests.
Abstract: The impact of institutions on economic performance has attracted significant attention from researchers, as well as from policy reformers. A rapidly growing area in this literature is the impact of economic freedom on economic growth. The aim of this paper was to explore publication bias in this literature by means of traditional funnel plots, meta-significance testing, as well as by bootstrapping these meta-significance tests. When all the available estimates are combined and averaged, there seems to be evidence of a genuine and positive economic freedom - economic growth effect. However, it is also shown that the economic freedom - economic growth literature is tainted strongly with publication bias. The existence of publication bias makes it difficult to identify the magnitude of the genuine effect of economic freedom on economic growth. The paper explores the differences between aggregate and disaggregate measures of economic freedom and shows that selection effects are stronger when aggregate measures are used.

173 citations


Journal ArticleDOI
TL;DR: In this article, the authors explore publication bias in economic growth literature by means of traditional funnel plots, meta-significance testing, as well as by bootstrapping these meta-Significance tests.
Abstract: . The impact of institutions on economic performance has attracted significant attention from researchers, as well as from policy reformers. A rapidly growing area in this literature is the impact of economic freedom on economic growth. The aim of this paper was to explore publication bias in this literature by means of traditional funnel plots, meta-significance testing, as well as by bootstrapping these meta-significance tests. When all the available estimates are combined and averaged, there seems to be evidence of a genuine and positive economic freedom – economic growth effect. However, it is also shown that the economic freedom – economic growth literature is tainted strongly with publication bias. The existence of publication bias makes it difficult to identify the magnitude of the genuine effect of economic freedom on economic growth. The paper explores the differences between aggregate and disaggregate measures of economic freedom and shows that selection effects are stronger when aggregate measures are used.

153 citations


Journal ArticleDOI
TL;DR: Evidence for publication bias is present in single-city time-series studies of ambient particles and selection of positive estimates from a range of lags could increase summary estimates for PM10 and daily mortality by up to 130% above those based on nondirectional approaches.
Abstract: BACKGROUND: Time-series studies have shown short-term temporal associations between low levels of ambient particulate air pollution and adverse health effects. It is not known whether or to what extent this literature is affected by publication bias. METHODS: We obtained effect estimates from time-series studies published up to January 2002. These were summarized and examined for funnel plot asymmetry. We compared summary estimates between single-city and prospective multicity studies. Using 1 multicity study, we examined the sensitivity of summary estimates to alternative lag selection policies. RESULTS: We found evidence for publication bias among single-city studies of daily mortality, hospital admissions for chronic obstructive lung disease (COPD), and incidence of cough symptom, but not for studies of lung function. Statistical correction for this bias reduced summary relative risk estimates for a 10 microg/m increment of particulate matter less than 10 microm aerodynamic diameter (PM10) as follows: daily mortality from 1.006 to 1.005 and admissions for COPD from 1.013 to 1.011; and odds ratio for cough from 1.025 to 1.015. Analysis of results from a large multicity study suggested that selection of positive estimates from a range of lags could increase summary estimates for PM10 and daily mortality by up to 130% above those based on nondirectional approaches. CONCLUSION: We conclude that publication bias is present in single-city time-series studies of ambient particles. However, after correcting for publication bias statistically, associations between particles and adverse health effects remained positive and precisely estimated. Differential selection of positive lags may also inflate estimates.

138 citations


Journal ArticleDOI
TL;DR: Although within-study selection was evident or suspected in several trials, the impact on the conclusions of the meta-analyses was minimal and sensitivity analysis was undertaken to assess the robustness of the conclusion to this bias.
Abstract: The systematic review community has become increasingly aware of the importance of addressing the issues of heterogeneity and publication bias in meta-analyses. A potentially bigger threat to the validity of a meta-analysis appears relatively unnoticed. The within-study selective reporting of outcomes, defined as the selection of a subset of the original variables recorded for inclusion in publication of trials, can theoretically have a substantial impact on the results. A cohort of meta-analyses on the Cochrane Library was reviewed to examine how often this form of within-study publication bias was suspected and explained some of the evident funnel plot asymmetry. In cases where the level of suspicion was high, sensitivity analysis was undertaken to assess the robustness of the conclusion to this bias. Although within-study selection was evident or suspected in several trials, the impact on the conclusions of the meta-analyses was minimal. This paper deals with the identification of, sensitivity analysis for, and impact of within-study selective reporting in meta-analysis.

Journal ArticleDOI
TL;DR: There is no evidence that women with GSTM1 null genotype have increased risk of developing endometriosis as compared with women with other genotypes, but the estimate could easily lose its statistical significance if there is a realistic 69-80% publication probability.
Abstract: In view of the controversies surrounding the glutathione S-transferases (GST) M1/T1-endometriosis association, a meta-analysis of the GSTM1/GSTT1 genetic association studies of endometriosis was performed. In this meta-analysis involving 14 GSTM1 studies with 1539 cases and 1805 controls and nine GSTT1 studies with 746 cases and 834 controls, respectively, substantial heterogeneities among studies were found. In addition, asymmetry in funnel plot was evident, which is likely to stem from publication bias, given no apparent indication of true heterogeneity. The bias appears to be prominent for GSTM1 studies, but is less so for GSTT1 studies. After correction for this bias, there is no evidence that women with GSTM1 null genotype have increased risk of developing endometriosis as compared with women with other genotypes. For GSTT1, the risk associated with the null genotype is 29% higher than other genotypes. However, even this estimate should be viewed with a large grain of salt, because the estimate could easily lose its statistical significance if there is a realistic 69-80% publication probability.

Journal ArticleDOI
TL;DR: In this paper, the diagnostic accuracy of new protein markers of acute coronary syndromes (ACS) in symptomatic outpatients at low risk of ACS and related complications comparable to patients evaluated in emergency department chest pain units was evaluated.
Abstract: Background: Published literature was systematically reviewed to determine the diagnostic accuracy of new protein markers of acute coronary syndromes (ACS) in symptomatic outpatients at low risk of ACS and related complications comparable to patients evaluated in emergency department chest pain units. Methods: Studies were identified by a MEDLINE® (1966 to May week 3, 2005) search. Abstracts were reviewed for relevance, and manuscripts were included by the independent consensus of 2 observers based on explicit criteria restricting the analysis to studies relevant to screening ambulatory patients with symptoms suggesting ACS. Publication bias was identified by a modified funnel plot analysis [study size ( y ) vs the inverse of the negative likelihood ratio ( x )]. Results of individual markers were reported separately. When 3 or more eligible studies were identified, data were aggregated by use of the summary ROC (SROC) curve. Results: Twenty-two protein markers in 10 unique populations met the inclusion criteria. Data required for SROC analysis (true- and false-positive rates) were available for 17 markers, in 9 unique populations, from publications and personal communications. Of these, only C-reactive protein was published in more than 2 populations to allow aggregation (6 studies total). C-Reactive protein demonstrated poor diagnostic performance on SROC curve analysis, with an area under the curve of 0.61 and a pooled diagnostic odds ratio of 1.81 (95% confidence interval, 1.06–3.07). Conclusion: Published evidence is not sufficient to support the routine use of new protein markers in screening for ACS in the emergency department setting.

01 Jan 2005
TL;DR: There may be a significant positive relationship between depression and alcoholism according to the published evidences, and the probability of publication bias is low.
Abstract: Objectives : This study was designed to integrate the results of community based studies which assessed a relationship between depression and alcoholism by meta-analysis. Methods: We identified the previons studies and included in meta-analysis by searching MEDLINE. Overall, 21 results of the studies for relationship between depression and alcoholism were selected for quantitative meta-analysis. Before the integration of the each effect size of the relationship between depression and alcoholism, a homogeneity test was conducted. For the publication bias, we also conducted the analyses of funnel plot, normal quantile plot, rank correlation test and the fail-safe n. Results : We used the random effect model to estimate the overall effect size, because the homogeneity of studies was rejected in a fixed effect model. Our quantitative meta-analysis yielded that integrated odds ratio between depression and alcoholism was 2.42 (95% C.I. 1.98-2.97). From the results of analyses of the publication bias, the probability of publication bias is considered low. Conclusion : The published evidences suggested that there may be a significant positive relationship between depression and alcoholism.

Journal ArticleDOI
15 Sep 2005-BMJ
TL;DR: In this paper, the authors conclude that there is no evidence of publication bias in reports on publication bias and that funnel plots should be used only as a "tool" and not a "rule" in evaluating publication bias.
Abstract: Editor—Dubben and Beck-Bornholdt conclude that there is no evidence of publication bias in reports on publication bias.1 Apart from the fact that funnel plots should be used only as a “tool” and not a “rule” in the evaluation of publication bias,2 I question whether such “meta-research” really helps to improve patient care or facilitates the applicability of research results. Systematic …