Using Bayes to get the most out of non-significant results
Reads0
Chats0
TLDR
It is argued Bayes factors allow theory to be linked to data in a way that overcomes the weaknesses of the other approaches, and provides a coherent approach to determining whether non-significant results support a null hypothesis over a theory, or whether the data are just insensitive.Abstract:
No scientific conclusion follows automatically from a statistically non-significant result, yet people routinely use non-significant results to guide conclusions about the status of theories (or the effectiveness of practices). To know whether a non-significant result counts against a theory, or if it just indicates data insensitivity, researchers must use one of: power, intervals (such as confidence or credibility intervals), or else an indicator of the relative evidence for one theory over another, such as a Bayes factor. I argue Bayes factors allow theory to be linked to data in a way that overcomes the weaknesses of the other approaches. Specifically, Bayes factors use the data themselves to determine their sensitivity in distinguishing theories (unlike power), and they make use of those aspects of a theory’s predictions that are often easiest to specify (unlike power and intervals, which require specifying the minimal interesting value in order to address theory). Bayes factors provide a coherent approach to determining whether non-significant results support a null hypothesis over a theory, or whether the data are just insensitive. They allow accepting and rejecting the null hypothesis to be put on an equal footing. Concrete examples are provided to indicate the range of application of a simple online Bayes calculator, which reveal both the strengths and weaknesses of Bayes factors.read more
Citations
More filters
Journal ArticleDOI
Bayesian inference for psychology. Part II: Example applications with JASP
Eric-Jan Wagenmakers,Jonathon Love,Maarten Marsman,Tahira Jamil,Alexander Ly,Josine Verhagen,Ravi Selker,Quentin Frederik Gronau,Damian Dropmann,Bruno Boutin,Frans Meerhoff,Patrick Knight,Akash Raj,Erik-Jan van Kesteren,Johnny van Doorn,Martin Šmíra,Sacha Epskamp,Alexander Etz,Dora Matzke,Tim de Jong,Don van den Bergh,Alexandra Sarafoglou,Helen Steingroever,Koen Derks,Jeffrey N. Rouder,Richard D. Morey +25 more
TL;DR: This part of this series introduces JASP (http://www.jasp-stats.org), an open-source, cross-platform, user-friendly graphical software package that allows users to carry out Bayesian hypothesis tests for standard statistical problems.
Journal ArticleDOI
Equivalence tests : a practical primer for t tests, correlations, and meta-analyses
TL;DR: This practical primer with accompanying spreadsheet and R package enables psychologists to easily perform equivalence tests (and power analyses) by setting equivalence bounds based on standardized effect sizes and provides recommendations to prespecify equivalence limits.
Journal ArticleDOI
Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications.
Eric-Jan Wagenmakers,Maarten Marsman,Tahira Jamil,Alexander Ly,Josine Verhagen,Jonathon Love,Ravi Selker,Quentin Frederik Gronau,Martin Šmíra,Sacha Epskamp,Dora Matzke,Jeffrey N. Rouder,Richard D. Morey +12 more
TL;DR: Ten prominent advantages of the Bayesian approach are outlined, and several objections to Bayesian hypothesis testing are countered.
Journal ArticleDOI
The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective
TL;DR: In this paper, the authors compare Bayesian and frequentist approaches to hypothesis testing and estimation with confidence or credible intervals, and explain how Bayesian methods achieve the goals of the New Statistics better than frequentist methods.
Journal ArticleDOI
Homocysteine‐lowering interventions for preventing cardiovascular events
TL;DR: Whether homocysteine-lowering interventions, provided to patients with and without pre-existing cardiovascular disease are effective in preventing cardiovascular events, as well as reducing all-cause mortality, and to evaluate their safety is evaluated.
References
More filters
Journal ArticleDOI
Distraction and placebo: two separate routes to pain control.
TL;DR: The data suggest that placebo analgesia does not depend on active redirection of attention and that expectancy and distraction can be combined to maximize pain relief.
Journal ArticleDOI
The Theory That Would Not Die
TL;DR: McGrayne et al. as mentioned in this paper argued that the scientific world is not immune from having different schools of thought or attitudes or attitudes, and pointed out the need for diversity in the scientific community.
Journal ArticleDOI
An Ethical Approach to Peeking at Data
TL;DR: For the common occurrence of post-hoc dataset augmentation, a new statistic is developed, paugmented, that represents the magnitude of the resulting Type I error inflation that elevates such augmentation from a questionable research practice to an ethical research decision.
Journal ArticleDOI
A default Bayesian hypothesis test for mediation
TL;DR: This work specifies a default Bayesian hypothesis test based on the Jeffreys–Zellner–Siow approach and extends thisdefault Bayesian test by allowing a comparison to directional or one-sided alternatives, using Markov chain Monte Carlo techniques implemented in JAGS.
Journal ArticleDOI
Effect of mindfulness meditation on brain–computer interface performance
TL;DR: The mindfulness meditation training group obtained a significantly higher BCI accuracy compared to both the music training and no-treatment control groups after the intervention, indicating effects of meditation above and beyond expectancy effects.