Open Access
Robust misinterpretation of confidence intervals
Reads0
Chats0
TLDR
Although all six statements were false, both researchers and students endorsed, on average, more than three statements, indicating a gross misunderstanding of CIs, which suggests that many researchers do not know the correct interpretation of a CI.Abstract:
Null hypothesis significance testing (NHST) is undoubtedly the most common inferential technique used to justify claims in the social sciences. However, even staunch defenders of NHST agree that its outcomes are often misinterpreted. Confidence intervals (CIs) have frequently been proposed as a more useful alternative to NHST, and their use is strongly encouraged in the APA Manual. Nevertheless, little is known about how researchers interpret CIs. In this study, 120 researchers and 442 students—all in the field of psychology—were asked to assess the truth value of six particular statements involving different interpretations of a CI. Although all six statements were false, both researchers and students endorsed, on average, more than three statements, indicating a gross misunderstanding of CIs. Self-declared experience with statistics was not related to researchers’ performance, and, even more surprisingly, researchers hardly outperformed the students, even though the students had not received any education on statistical inference whatsoever. Our findings suggest that many researchers do not know the correct interpretation of a CI. The misunderstandings surrounding p-values and CIs are particularly unfortunate because they constitute the main tools by which psychologists draw conclusions from data.read more
Citations
More filters
Journal ArticleDOI
Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications.
Eric-Jan Wagenmakers,Maarten Marsman,Tahira Jamil,Alexander Ly,Josine Verhagen,Jonathon Love,Ravi Selker,Quentin Frederik Gronau,Martin Šmíra,Sacha Epskamp,Dora Matzke,Jeffrey N. Rouder,Richard D. Morey +12 more
TL;DR: Ten prominent advantages of the Bayesian approach are outlined, and several objections to Bayesian hypothesis testing are countered.
Journal ArticleDOI
The fallacy of placing confidence in confidence intervals
TL;DR: It is shown in a number of examples that CIs do not necessarily have any of the properties of confidence intervals, and can lead to unjustified or arbitrary inferences, and is suggested that other theories of interval estimation should be used instead.
Journal ArticleDOI
Ordinal Regression Models in Psychology: A Tutorial
TL;DR: In psychology, ordinal variables, although extremely common in psychology, are almost exclusively analyzed with statistical models that falsely assume them to be metric as discussed by the authors, which can lead to distorted effect.
Journal ArticleDOI
Painfree and accurate Bayesian estimation of psychometric functions for (potentially) overdispersed data
Heiko H. Schütt,Heiko H. Schütt,Stefan Harmeling,Jakob H. Macke,Jakob H. Macke,Felix A. Wichmann +5 more
TL;DR: It is shown that the use of the beta-binomial model makes it possible to determine accurate credible intervals even in data which exhibit substantial overdispersion, and Bayesian inference methods are used for estimating the posterior distribution of the parameters of the psychometric function.
Journal ArticleDOI
The philosophy of Bayes’ factors and the quantification of statistical evidence
TL;DR: In this article, the authors explore the concept of statistical evidence and how it can be quantified using the Bayes factor, and discuss the philosophical issues inherent in the use of the BFA.
References
More filters
Journal ArticleDOI
Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research.
TL;DR: Kline as discussed by the authors reviewed the controversy regarding significance testing, and offered methods for effect size and confidence interval estimation, and suggested some alternative methodologies, and concluded that there is no "magical alternative" to statistical tests and that such tests are appropriate in some circumstances when applied correctly.
Book
Statistical Significance: Rationale, Validity and Utility
TL;DR: A Litany of Criticisms of NHSTP The Null-Hypothesis Significance-Test Procedure (NHSTP) A Phenomenon and its Quartets of Hypotheses Evidential Support for a Theory and NH-STP Effect Size and Related Issues A Critical Look at Statistical Power Bayesianism A Case for NH STP
Journal ArticleDOI
What do doctors know about statistics
TL;DR: It is concluded that the statistical knowledge of most doctors is so limited that they cannot be expected to draw the right conclusions from those statistical analyses which are found in papers in medical journals.