Using Bayes to get the most out of non-significant results
TLDR
It is argued Bayes factors allow theory to be linked to data in a way that overcomes the weaknesses of the other approaches, and provides a coherent approach to determining whether non-significant results support a null hypothesis over a theory, or whether the data are just insensitive.Abstract:
No scientific conclusion follows automatically from a statistically non-significant result, yet people routinely use non-significant results to guide conclusions about the status of theories (or the effectiveness of practices). To know whether a non-significant result counts against a theory, or if it just indicates data insensitivity, researchers must use one of: power, intervals (such as confidence or credibility intervals), or else an indicator of the relative evidence for one theory over another, such as a Bayes factor. I argue Bayes factors allow theory to be linked to data in a way that overcomes the weaknesses of the other approaches. Specifically, Bayes factors use the data themselves to determine their sensitivity in distinguishing theories (unlike power), and they make use of those aspects of a theory’s predictions that are often easiest to specify (unlike power and intervals, which require specifying the minimal interesting value in order to address theory). Bayes factors provide a coherent approach to determining whether non-significant results support a null hypothesis over a theory, or whether the data are just insensitive. They allow accepting and rejecting the null hypothesis to be put on an equal footing. Concrete examples are provided to indicate the range of application of a simple online Bayes calculator, which reveal both the strengths and weaknesses of Bayes factors.read more
Citations
More filters
Journal ArticleDOI
Continuous theta-burst stimulation over the dorsolateral prefrontal cortex inhibits improvement on a working memory task
Teodóra Vékony,Viola Luca Németh,Adrienn Holczer,Krisztián Kocsis,Zsigmond Tamás Kincses,László Vécsei,Anita Must +6 more
TL;DR: Theta-burst stimulation over the dorsolateral prefrontal cortex (DLPFC) may be more effective for modulating cortical excitability compared to standard repetitive transcranial magnetic stimulation and the possibility of clarifying processes underlying WM performance changes by using non-invasive brain stimulation is indicated.
Journal ArticleDOI
Commentary: Oxytocin-gaze positive loop and the coevolution of human-dog bonds.
Zoltan Kekecs,Aba Szollosi,Bence Palfi,Barnabas Szaszi,Krisztina Kovacs,Zoltan Dienes,Balazs Aczel +6 more
TL;DR: An oxytocin-mediated positive loop, which developed through the coevolution of human–dog bonding, was hypothesized, which was repeated with hand-reared wolves and their owners to evaluate whether the proposed oxytocIn loop was specific to the human– dog interaction.
Journal ArticleDOI
Increased posterior default mode network activity and structural connectivity in young adult APOE-ε4 carriers: a multimodal imaging investigation.
Carl J. Hodgetts,Jonathan P. Shine,Huw Williams,Mark Postans,Rebecca Sims,Julie Williams,Andrew D. Lawrence,Kim S. Graham +7 more
TL;DR: Findings are consistent with a lifespan view of Alzheimer's disease risk, where early-life, connectivity-related changes in specific, vulnerable “hubs” (e.g., pDMN) lead to increased neural activity, and such changes may reflect reduced network efficiency/flexibility in APOE-ε4 carriers.
Journal ArticleDOI
Aripiprazole for cocaine abstinence: a randomized-controlled trial with ecological momentary assessment.
Landhing M. Moran,Karran A. Phillips,William J. Kowalczyk,Udi E. Ghitza,Daniel A. Agage,David H. Epstein,Kenzie L. Preston +6 more
TL;DR: The results suggest that in recently abstinent cocaine users, aripiprazole might delay relapse, but might also slightly increase craving, which underscores the fact that initial abstinence from cocaine is not a trivial hurdle.
Journal ArticleDOI
The Zero-Sum Fallacy in Evidence Evaluation.
TL;DR: This work demonstrates that individuals erroneously assume that evidence that is equally predicted by two competing hypotheses offers no support for either hypothesis, and argues that this reasoning error is due to a zero-sum perspective on evidence.
References
More filters
Book
Statistical Power Analysis for the Behavioral Sciences
TL;DR: The concepts of power analysis are discussed in this paper, where Chi-square Tests for Goodness of Fit and Contingency Tables, t-Test for Means, and Sign Test are used.
Book
Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach
TL;DR: The second edition of this book is unique in that it focuses on methods for making formal statistical inference from all the models in an a priori set (Multi-Model Inference).
Journal ArticleDOI
Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses.
TL;DR: In the new version, procedures to analyze the power of tests based on single-sample tetrachoric correlations, comparisons of dependent correlations, bivariate linear regression, multiple linear regression based on the random predictor model, logistic regression, and Poisson regression are added.
Journal ArticleDOI
Bayesian data analysis.
TL;DR: A fatal flaw of NHST is reviewed and some benefits of Bayesian data analysis are introduced and illustrative examples of multiple comparisons in Bayesian analysis of variance and Bayesian approaches to statistical power are presented.
Journal ArticleDOI
Power failure: why small sample size undermines the reliability of neuroscience
Katherine S. Button,John P. A. Ioannidis,Claire Mokrysz,Brian A. Nosek,Jonathan Flint,Emma S J Robinson,Marcus R. Munafò +6 more
TL;DR: It is shown that the average statistical power of studies in the neurosciences is very low, and the consequences include overestimates of effect size and low reproducibility of results.