Using Bayes to get the most out of non-significant results
Reads0
Chats0
TLDR
It is argued Bayes factors allow theory to be linked to data in a way that overcomes the weaknesses of the other approaches, and provides a coherent approach to determining whether non-significant results support a null hypothesis over a theory, or whether the data are just insensitive.Abstract:
No scientific conclusion follows automatically from a statistically non-significant result, yet people routinely use non-significant results to guide conclusions about the status of theories (or the effectiveness of practices). To know whether a non-significant result counts against a theory, or if it just indicates data insensitivity, researchers must use one of: power, intervals (such as confidence or credibility intervals), or else an indicator of the relative evidence for one theory over another, such as a Bayes factor. I argue Bayes factors allow theory to be linked to data in a way that overcomes the weaknesses of the other approaches. Specifically, Bayes factors use the data themselves to determine their sensitivity in distinguishing theories (unlike power), and they make use of those aspects of a theory’s predictions that are often easiest to specify (unlike power and intervals, which require specifying the minimal interesting value in order to address theory). Bayes factors provide a coherent approach to determining whether non-significant results support a null hypothesis over a theory, or whether the data are just insensitive. They allow accepting and rejecting the null hypothesis to be put on an equal footing. Concrete examples are provided to indicate the range of application of a simple online Bayes calculator, which reveal both the strengths and weaknesses of Bayes factors.read more
Citations
More filters
Posted ContentDOI
Retrieval of a well-established skill is resistant to distraction: evidence from an implicit probabilistic sequence learning task
Teodóra Vékony,Lilla Török,Felipe Pedraza,Kate Schipper,Claire Pleche,László Tóth,Karolina Janacsek,Karolina Janacsek,Karolina Janacsek,Dezso Nemeth,Dezso Nemeth,Dezso Nemeth +11 more
TL;DR: It is shown that although the dual-task condition significantly prolonged the overall reaction times in the primary task, the access to the previously learned probabilistic representations remained intact and an inverse relationship between the ability to successfully retrieve sequence knowledge and the accuracy of the secondary task was found.
Proceedings ArticleDOI
Null hypothesis significance testing in simulation
TL;DR: A critical reflection of the arguments contra NHST shows that although NHST is indeed ill-suited for many simulation applications and objectives it is by no means superfluous, neither in general, nor in particular for simulation.
Dissertation
Reliability, replicability and reproducibility in PET imaging
TL;DR: This thesis explores themes of reliability, replicability and reproducibility for PET research, which allows researchers to more effectively gauge the feasibility of new between-individual studies before collection of any data, and to focus their efforts on research questions which can be expected to yield more interpretable outcomes.
Journal ArticleDOI
A novel team Familiarity Score for operating teams is a predictor of length of a procedure—A retrospective Bayesian analysis
TL;DR: FS in vascular teams was shown to be strongly associated with LOP, suggesting that more familiar teams might collaborate more efficiently, and Bayesian statistics was used to analyze the data.
Journal ArticleDOI
Situational factors shape moral judgements in the trolley dilemma in Eastern, Southern and Western countries in a culturally diverse sample
Bence Bago,Marton Kovacs,John Protzko,Tamás Nagy,Zoltan Kekecs,Bence Palfi,Matus Adamkovic,Sylwia Adamus,Sumaya Albalooshi,Nihan Albayrak-Aydemir,Ilham Nur Alfian,Sinan Alper,Sara Álvarez-Solas,Sara Alves,Santiago Barriga Amaya,Pia K. Andresen,Gulnaz Anjum,Daniel Ansari,Patrícia Arriaga,John Jamir Benzon R. Aruta,Alexiοs Arvanitis,Peter Babincak,Krystian Barzykowski,Bana Bashour,Ernest Baskin,Luís Manuel Cunha Batalha,Carlota Batres,Jozef Bavolar,Fatih Bayrak,Benjamin Becker,Maja Becker,Anabel Belaus,Michał Białek,Ennio Bilancini,Daniel Boller,Leonardo Boncinelli,Jordane Boudesseul,Benjamin T. Brown,Erin Michelle Buchanan,Muhammad Mussaffa Butt,Dustin P. Calvillo,Nate C. Carnes,Jared Celniker,Christopher R. Chartier,William J. Chopik,Poom Chotikavan,Hu Chuan-Peng,Rockwell F. Clancy,Ogeday Çoker,Rita C. Correia,Vera Cubela Adoric,Carmelo P. Cubillas,Stefan Czoschke,Yalda Daryani,Job de Grefte,Wieteke C. de Vries,Elif Gizem Demirag Burak,Carina Dias,Barnaby J. W. Dixson,Xinkai Du,Francesca Dumančić,Andrei Dumbrava,Natalia Dutra,Janina Enachescu,Celia Esteban-Serna,Luis Eudave,Thomas Rhys Evans,Gilad Feldman,Fatima M. Felisberti,Susann Fiedler,Andres Findor,Alexandra Fleischmann,Francesco Foroni,Radka Francová,Darius-Aurel Frank,Cynthia H.Y. Fu,Shan Gao,Omid Gholam Ghasemi,Ali-Reza Ghazi-Noori,Maliki E. Ghossainy,Isabella Giammusso,Tripat Gill,Biljana Gjoneska,Mario Gollwitzer,Aurélien Graton,Maurice Grinberg,Agata Groyecka-Bernard,Elizabeth Harris,Andree Hartanto,Widad Hassan,Javad Hatami,Katrina R. Heimark,Jasper J. Hidding,Evgeniya Hristova,Martin Hruska,Charlotte A. Hudson,Richard Wayne Huskey,Ayumi Ikeda,Yoel Inbar,Gordon P. D. Ingram,Ozan Isler,Chris Isloi,Aishwarya Subramanian Iyer,Bastian Jaeger,Steve M. J. Janssen,William Jimenez-Leal,Biljana Jokić,Pavol Kačmár,Veselina Kadreva,Gwenaël Kaminski,Farzan Karimi-Malekabadi,Arno T. A. Kasper,Keith M. Kendrick,B Kennedy,Halil Emre Kocalar,Rabia I. Kodapanakkal,Marta Kowal,Elliot Tyler Kruse,Lenka Kucerova,Anton Kühberger,A. Kuzminska,Fanny Lalot,Claus Lamm,Joris Lammers,Elke B. Lange,Anthony Lantian,Ivy Yee-Man Lau,Ljiljana B. Lazarević,Marijke C. Leliveld,Jennifer N. Lenz,Carmel A. Levitan,Savannah C. Lewis,Manyu Li,Yansong Li,Haozheng Li,Tiago J. S. Lima,Samuel Lincoln Bezerra Lins,Marco Tullio Liuzza,Paula Lopes,Jackson G. Lu,Trent Lynds,Martin Máčel,Sean P. Mackinnon,Madhavilatha Maganti,Zoe Magraw-Mickelson,Leon F. Magson,Harry Manley,Gabriela Marcu,Darja Masli Seršić,Celine-Justine Matibag,Alan D. A. Mattiassi,Mahdi Mazidi,Joseph McFall,Neil Marvin McLatchie,Michael C. Mensink,Lena Miketta,Taciano L. Milfont,Alberto Mirisola,Michał Misiak,Panagiotis Mitkidis,Mehrad Moeini-Jazani,Arash Monajem,David Moreau,Erica D. Musser,Erita Narhetali,Danielle P. Ochoa,Jerome Olsen,Nicholas C. Owsley,Asil Ali Özdoğru,M. H. Panning,Marietta Papadatou-Pastou,Neha Parashar,Philip Pärnamets,Mariola Paruzel-Czachura,Michal Parzuchowski,Julia V. Paterlini,Jeffrey M. Pavlacic,Mehmet Peker,Kim Peters,Liudmila Piatnitckaia,Isabela Rahal De Rezende Pinto,Monica Renee Policarpio,Nada Pop-Jordanova,Annas J. Pratama,Maximilian Primbs,Ekaterina Pronizius,Danka Purić,Elisa Puvia,Vahid Qamari,Kun Qian,Alain Quiamzade,Beata Raczova,Diego A. Reinero,Ulf-Dietrich Reips,Cecilia Reyna,Kimberly Reynolds,M. F. F. Ribeiro,Jan Philipp Röer,Robert M. Ross,Petros Roussos,Fernando-Gabriel Ruiz-Dodobara,Susana Ruiz-Fernández,Bastiaan T. Rutjens,Katarzyna Rybus,Adil Samekin,A. Caetano Santos,Nicolas Say,Christopher-J. Schild,Kathleen Lynn Schmidt,Karolina Aleksandra Ścigała,Mohammad Hossein Sharifian,Jiaxin Shi,Yaoxi Shi,E. Sievers,Miroslav Sirota,Michael Slipenkyj,Çağlar Solak,Agnieszka Sorokowska,Piotr Sorokowski,Sinem Söylemez,Niklas K. Steffens,Ian D. Stephen,Anni Sternisko,Laura Stevens-Wilson,Suzanne Stewart,Stefan Stieger,Daniel Storage,Justine Strube,Kyle J. Susa,R. D. Szekely-Copîndean,Natalia M. Szostak,Bagus Takwin,Srinivasan Tatachari,Andrew Gardner Adams Thomas,Kevin E. Tiede,Lucas E. Tiong,Mirjana Tonković,Bastien Trémolière,Lauren V. Tunstead,Belgüzar Nilay Türkan,Mathias Twardawski,Miguel A. Vadillo,Zahir Vally,Leigh Ann Vaughn,Bruno Verschuere,Denis Vlašiček,Martin Voracek,Marek A. Vranka,Shuzhen Wang,S. West,Stephen Whyte,Leigh S. Wilton,Anna Marcelina Wlodarczyk,Xue Wu,Fei Xin,Su Myat Yadanar,Hiroshi Yama,Yuki Yamada,Onurcan Yilmaz,Sangsuk Yoon,Danielle M. Young,Ilya Zakharov,Rizqy Amelia Zein,Ingo Zettler,Iris Žeželj,Don C. Zhang,Jin Zhang,Xiaoxiao Zheng,Rink Hoekstra,Balazs Aczel +269 more
TL;DR: The authors empirically tested the universality of the effects of intent and personal force on moral dilemma judgements by replicating the experiments of Greene et al. in 45 countries from all inhabited continents and found that personal force and its interaction with intention exert influence on moral judgements in the US and Western cultural clusters, replicating and expanding the original findings.
References
More filters
Book
Statistical Power Analysis for the Behavioral Sciences
TL;DR: The concepts of power analysis are discussed in this paper, where Chi-square Tests for Goodness of Fit and Contingency Tables, t-Test for Means, and Sign Test are used.
Book
Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach
TL;DR: The second edition of this book is unique in that it focuses on methods for making formal statistical inference from all the models in an a priori set (Multi-Model Inference).
Journal ArticleDOI
Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses.
TL;DR: In the new version, procedures to analyze the power of tests based on single-sample tetrachoric correlations, comparisons of dependent correlations, bivariate linear regression, multiple linear regression based on the random predictor model, logistic regression, and Poisson regression are added.
Journal ArticleDOI
Bayesian data analysis.
TL;DR: A fatal flaw of NHST is reviewed and some benefits of Bayesian data analysis are introduced and illustrative examples of multiple comparisons in Bayesian analysis of variance and Bayesian approaches to statistical power are presented.
Journal ArticleDOI
Power failure: why small sample size undermines the reliability of neuroscience
Katherine S. Button,John P. A. Ioannidis,Claire Mokrysz,Brian A. Nosek,Jonathan Flint,Emma S J Robinson,Marcus R. Munafò +6 more
TL;DR: It is shown that the average statistical power of studies in the neurosciences is very low, and the consequences include overestimates of effect size and low reproducibility of results.