Open Access
Robust misinterpretation of confidence intervals
Reads0
Chats0
TLDR
Although all six statements were false, both researchers and students endorsed, on average, more than three statements, indicating a gross misunderstanding of CIs, which suggests that many researchers do not know the correct interpretation of a CI.Abstract:
Null hypothesis significance testing (NHST) is undoubtedly the most common inferential technique used to justify claims in the social sciences. However, even staunch defenders of NHST agree that its outcomes are often misinterpreted. Confidence intervals (CIs) have frequently been proposed as a more useful alternative to NHST, and their use is strongly encouraged in the APA Manual. Nevertheless, little is known about how researchers interpret CIs. In this study, 120 researchers and 442 students—all in the field of psychology—were asked to assess the truth value of six particular statements involving different interpretations of a CI. Although all six statements were false, both researchers and students endorsed, on average, more than three statements, indicating a gross misunderstanding of CIs. Self-declared experience with statistics was not related to researchers’ performance, and, even more surprisingly, researchers hardly outperformed the students, even though the students had not received any education on statistical inference whatsoever. Our findings suggest that many researchers do not know the correct interpretation of a CI. The misunderstandings surrounding p-values and CIs are particularly unfortunate because they constitute the main tools by which psychologists draw conclusions from data.read more
Citations
More filters
Journal ArticleDOI
Bayesian Analysis in Strategic Management Research: Time to Update Your Priors
Brian T. McCann,Andreas Schwab +1 more
TL;DR: Bayesian statistical methods offer an important and increasingly endorsed alternative to traditional statistical significance testing as discussed by the authors , providing guidance to strategic management researchers who may wish to incorporate these methods into their research.
Journal ArticleDOI
Bayesian deconstruction of climate sensitivity estimates using simple models: implicit priors and the confusion of the inverse
TL;DR: It is argued that the explicitly Bayesian approach is preferred, as it both clarifies the role of the prior and allows researchers to transparently test the sensitivity of their results to it.
Journal ArticleDOI
Beyond p-values in the evaluation of brain-computer interfaces: A Bayesian estimation approach.
Filip Melinscak,Luis Montesano +1 more
TL;DR: A wider adoption of the Bayesian estimation approach in BCI studies could bring about greater transparency in data analysis, allow accumulation of knowledge across studies, and reduce questionable practices such as "p-hacking".
Posted Content
Visual Reasoning Strategies for Effect Size Judgments and Decisions
TL;DR: In this paper, the authors present findings from a mixed design experiment on Mechanical Turk which tested eight uncertainty visualization designs: 95% containment intervals, hypothetical outcome plots, densities, and quantile dotplots, each with and without means added.
Journal ArticleDOI
Evidential Calibration of Confidence Intervals
TL;DR: In this article , the authors present a method for calibrating error-rate based confidence intervals to evidence-based support intervals, which are obtained from inverting Bayes factors based on a parameter estimate and its standard error.
References
More filters
Journal ArticleDOI
The earth is round (p < .05)
TL;DR: The authors reviewed the problems with null hypothesis significance testing, including near universal misinterpretation of p as the probability that H is false, the misinterpretation that its complement is the probability of successful replication, and the mistaken assumption that if one rejects H₀ one thereby affirms the theory that led to the test.
Journal ArticleDOI
Statistical Methods in Psychology Journals: Guidelines and Explanations
TL;DR: The Task Force on Statistical Inference (TFSI) of the American Psychological Association (APA) as discussed by the authors was formed to discuss the application of significance testing in psychology journals and its alternatives, including alternative underlying models and data transformation.
Journal ArticleDOI
The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis
John M. Hoenig,Dennis M. Heisey +1 more
TL;DR: The problem of post-experiment power calculation is discussed in this paper. But, the problem is extensive and present arguments to demonstrate the flaw in the logic, which is fundamentally flawed.
Journal ArticleDOI
Publication Manual of the American Psychological AssociationPublication Manual of the American Psychological Association.
Kate E Decleene,Jennifer L. Fogo +1 more
TL;DR: The book provides stronger standards for maintaining the participant confidentiality and for reducing bias in language describing participants and suggests that researchers avoid the use of derogatory language such as using “minority” for “non-white” populations.
Book
Introduction to Probability and Statistics
TL;DR: The twelve edition of the Introduction to Probability and Statistics (INTRODUCTION TO PROBABILITY and STATISTICS) as discussed by the authors has been used by hundreds of thousands of students since its first edition.