Excess significance bias in the literature on brain volume abnormalities.
Reads0
Chats0
TLDR
There are too many studies with statistically significant results in the literature on brain volume abnormalities that suggest strong biases in the Literature, with selective outcome reporting and selective analyses reporting being possible explanations.Abstract:
Context Many studies report volume abnormalities in diverse brain structures in patients with various mental health conditions. Objective To evaluate whether there is evidence for an excess number of statistically significant results in studies of brain volume abnormalities that suggest the presence of bias in the literature. Data Sources PubMed (articles published from January 2006 to December 2009). Study Selection Recent meta-analyses of brain volume abnormalities in participants with various mental health conditions vs control participants with 6 or more data sets included, excluding voxel-based morphometry. Data Extraction Standardized effect sizes were extracted in each data set, and it was noted whether the results were“positive” (P Data Synthesis From 8 articles, 41 meta-analyses with 461 data sets were evaluated (median, 10 data sets per meta-analysis) pertaining to 7 conditions. Twenty-one of the 41 meta-analyses had found statistically significant associations, and 142 of 461 (31%) data sets had positive results. Even if the summary effect sizes of the meta-analyses were unbiased, the expected number of positive results would have been only 78.5 compared with the observed number of 142 (P Conclusion There are too many studies with statistically significant results in the literature on brain volume abnormalities. This pattern suggests strong biases in the literature, with selective outcome reporting and selective analyses reporting being possible explanations.read more
Citations
More filters
Journal ArticleDOI
Power failure: why small sample size undermines the reliability of neuroscience
Katherine S. Button,John P. A. Ioannidis,Claire Mokrysz,Brian A. Nosek,Jonathan Flint,Emma S J Robinson,Marcus R. Munafò +6 more
TL;DR: It is shown that the average statistical power of studies in the neurosciences is very low, and the consequences include overestimates of effect size and low reproducibility of results.
Journal ArticleDOI
Vitamin D and multiple health outcomes: umbrella review of systematic reviews and meta-analyses of observational studies and randomised trials
TL;DR: Evidence does not support the argument that vitamin D only supplementation increases bone mineral density or reduces the risk of fractures or falls in older people, and highly convincing evidence of a clear role of vitamin D does not exist for any outcome, but associations with a selection of outcomes are probable.
Journal ArticleDOI
Brain Volumes in Schizophrenia: A Meta-Analysis in Over 18 000 Subjects
Sander V. Haijma,Neeltje E.M. van Haren,Wiepke Cahn,P. Cédric M. P. Koolschijn,Hilleke E. Hulshoff Pol,René S. Kahn +5 more
TL;DR: Brain loss in schizophrenia is related to a combination of (early) neurodevelopmental processes-reflected in intracranial volume reduction-as well as illness progression.
Journal ArticleDOI
The secret lives of experiments: methods reporting in the fMRI literature.
TL;DR: Significant gaps in methods reporting among fMRI studies are documented, and improved methodological descriptions in research reports would yield significant benefits for the field.
Journal ArticleDOI
Progressive brain changes in schizophrenia related to antipsychotic treatment? A meta-analysis of longitudinal MRI studies
Paolo Fusar-Poli,Renata Smieskova,Matthew J. Kempton,Beng-Choon Ho,Nancy C. Andreasen,Stefan Borgwardt +5 more
TL;DR: Schizophrenia is characterized by progressive gray matter volume decreases and lateral ventricular volume increases, and some of these neuroanatomical alterations may be associated with antipsychotic treatment.
References
More filters
Journal ArticleDOI
Amygdala volumes in mood disorders--meta-analysis of magnetic resonance volumetry studies.
Tomas Hajek,Miloslav Kopecek,Miloslav Kopecek,Jiri Kozeny,Eva Gunde,Martin Alda,Martin Alda,Cyril Höschl +7 more
TL;DR: The absence of overall differences in amygdala volumes, in the presence of significant and sometimes mirror changes in patient subgroups, demonstrates marked heterogeneity among mood disorders.
Journal ArticleDOI
Critical interpretation of Cochran's Q test depends on power and prior assumptions about heterogeneity
Tiago V. Pereira,Tiago V. Pereira,Tiago V. Pereira,Nikolaos A. Patsopoulos,Nikolaos A. Patsopoulos,Georgia Salanti,John P. A. Ioannidis +6 more
TL;DR: An evaluation of 1011 meta-analyses of clinical trials with ⩾4 studies and binary outcomes shows that power to detect typical heterogeneity was low in most situations, and usually a non-significant Q test did not change perceptibly prior convictions on heterogeneity.
Journal ArticleDOI
Journals Should Publish All “Null” Results and Should Sparingly Publish “Positive” Results
TL;DR: The X team (real but anonymous here) meets successfully most proposed criteria and has published nine articles on mostly brand new (but also some replicated) gene-disease research.
Journal ArticleDOI
MR image-based measurement of rates of change in volumes of brain structures. Part II: application to a study of Alzheimer’s disease and normal aging
Deming Wang,Jonathan B. Chalk,Stephen E. Rose,Greig I. de Zubicaray,Gary Cowin,Graham J. Galloway,Daniel Barnes,D. Spooner,David M. Doddrell,James Semple +9 more
TL;DR: Rates of brain atrophy, especially in specific regions of the brain, based on MR images can provide sensitive measures for evaluating the progression of AD and will be useful for the evaluation of therapeutic effects of novel therapies for AD.
Related Papers (5)
Bias in meta-analysis detected by a simple, graphical test
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant
Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials
Jonathan A C Sterne,Alex J. Sutton,John P. A. Ioannidis,Norma Terrin,David R. Jones,Joseph Lau,James R. Carpenter,Gerta Rücker,Roger M. Harbord,Christopher H. Schmid,Jennifer Tetzlaff,Jonathan J Deeks,Jaime Peters,Petra Macaskill,Guido Schwarzer,Sue Duval,Douglas G. Altman,David Moher,Julian P T Higgins +18 more