Confidence intervals permit, but do not guarantee, better inference than statistical significance testing.
Reads0
Chats0
TLDR
Improved statistical inference can result from encouragement of meta-analytic thinking and use of CIs but, for full benefit, such highly desirable statistical reform requires also that researchers interpret CIs without recourse to NHST.Abstract:
A statistically significant result, and a non-significant result may differ little, although significance status may tempt an interpretation of difference. Two studies are reported that compared interpretation of such results presented using null hypothesis significance testing (NHST), or confidence intervals (CIs). Authors of articles published in psychology, behavioural neuroscience, and medical journals were asked, via email, to interpret two fictitious studies that found similar results, one statistically significant, and the other non-significant. Responses from 330 authors varied greatly, but interpretation was generally poor, whether results were presented as CIs or using NHST. However, when interpreting CIs respondents who mentioned NHST were 60% likely to conclude, unjustifiably, the two results conflicted, whereas those who interpreted CIs without reference to NHST were 95% likely to conclude, justifiably, the two results were consistent. Findings were generally similar for all three disciplines. An email survey of academic psychologists confirmed that CIs elicit better interpretations if NHST is not invoked. Improved statistical inference can result from encouragement of meta-analytic thinking and use of CIs but, for full benefit, such highly desirable statistical reform requires also that researchers interpret CIs without recourse to NHST.read more
Citations
More filters
Journal ArticleDOI
The New Statistics Why and How
TL;DR: An eight-step new-statistics strategy for research with integrity is described, which starts with formulation of research questions in estimation terms, has no place for NHST, and is aimed at building a cumulative quantitative discipline.
Book
Understanding the New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis
TL;DR: The ESCI for the Normal and t Distributions, and values of z and t are presented in this article, along with a discussion of the ESCI Modules and their use in practice.
Journal ArticleDOI
Bayesian versus Orthodox statistics: which side are you on?
TL;DR: This article presents some common situations in which Bayesian and orthodox approaches to significance testing come to different conclusions; the reader is shown how to apply Bayesian inference in practice, using free online software, to allow more coherent inferences from data.
Journal ArticleDOI
Computation of measures of effect size for neuroscience data sets
TL;DR: An open‐access matlab toolbox provides a wide range of MES to complement the frequently used types of hypothesis tests, such as t‐tests and analysis of variance, and should be useful to neuroscientists wishing to enhance their repertoire of statistical reporting.
Journal ArticleDOI
The earth is flat (p < 0.05): significance thresholds and the crisis of unreplicable research
TL;DR: The widespread use of ‘statistical significance’ as a license for making a claim of a scientific finding leads to considerable distortion of the scientific process, and potential arguments against removing significance thresholds are discussed.
References
More filters
Journal ArticleDOI
The earth is round (p < .05)
TL;DR: The authors reviewed the problems with null hypothesis significance testing, including near universal misinterpretation of p as the probability that H is false, the misinterpretation that its complement is the probability of successful replication, and the mistaken assumption that if one rejects H₀ one thereby affirms the theory that led to the test.
Journal ArticleDOI
Statistical Methods in Psychology Journals: Guidelines and Explanations
TL;DR: The Task Force on Statistical Inference (TFSI) of the American Psychological Association (APA) as discussed by the authors was formed to discuss the application of significance testing in psychology journals and its alternatives, including alternative underlying models and data transformation.
Journal ArticleDOI
Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology.
TL;DR: For example, the authors used consistency tests of a conjectural taxometric model with 94% success with zero false negatives to estimate numerical point values, even if approximate with rough tolerances; and lacking this, ranges, orderings, secondorder differences, curve peaks and valleys, and function forms should be used.
Journal ArticleDOI
Inference by eye: confidence intervals and how to read pictures of data.
Geoff Cumming,Sue Finch +1 more
TL;DR: 7 rules of eye are proposed to guide the inferential use of figures with error bars and include guidelines for inferential interpretation of the overlap of CIs on independent group means.
Journal ArticleDOI
Statistical Significance Testing and Cumulative Knowledge in Psychology: Implications for Training of Researchers
TL;DR: This article showed that the benefits that they believe flow from use of significance testing are illusory and should be replaced with point estimates and confidence intervals in individual studies and with meta-analyses in the integration of multiple studies.