Using Bayes to get the most out of non-significant results
Reads0
Chats0
TLDR
It is argued Bayes factors allow theory to be linked to data in a way that overcomes the weaknesses of the other approaches, and provides a coherent approach to determining whether non-significant results support a null hypothesis over a theory, or whether the data are just insensitive.Abstract:
No scientific conclusion follows automatically from a statistically non-significant result, yet people routinely use non-significant results to guide conclusions about the status of theories (or the effectiveness of practices). To know whether a non-significant result counts against a theory, or if it just indicates data insensitivity, researchers must use one of: power, intervals (such as confidence or credibility intervals), or else an indicator of the relative evidence for one theory over another, such as a Bayes factor. I argue Bayes factors allow theory to be linked to data in a way that overcomes the weaknesses of the other approaches. Specifically, Bayes factors use the data themselves to determine their sensitivity in distinguishing theories (unlike power), and they make use of those aspects of a theory’s predictions that are often easiest to specify (unlike power and intervals, which require specifying the minimal interesting value in order to address theory). Bayes factors provide a coherent approach to determining whether non-significant results support a null hypothesis over a theory, or whether the data are just insensitive. They allow accepting and rejecting the null hypothesis to be put on an equal footing. Concrete examples are provided to indicate the range of application of a simple online Bayes calculator, which reveal both the strengths and weaknesses of Bayes factors.read more
Citations
More filters
Journal ArticleDOI
The relevance of alpha phase in human perception
Manuela Ruzzoli,Mireia Torralba,Luis Morís Fernández,Salvador Soto-Faraco,Salvador Soto-Faraco +4 more
TL;DR: The results provide evidence for the involvement of pre-stimulus oscillatory alpha amplitude but not phase in perception and provide a clear-cut EEG analysis pipeline that should support the adoption of a hypothesis-driven framework for future replications and applications.
Journal ArticleDOI
Protocol for a national monthly survey of alcohol use in England with 6-month follow-up: ‘The Alcohol Toolkit Study’
Emma Beard,Jamie Brown,Robert West,Crispin Acton,Alan Brennan,Colin Drummond,Matthew Hickman,John Holmes,Eileen Kaner,Karen Lock,Matthew Walmsley,Susan Michie +11 more
TL;DR: The Alcohol Toolkit Study will improve understanding of population level factors influencing alcohol consumption and be an important resource for policy evaluation and planning.
Journal ArticleDOI
A multi-country test of brief reappraisal interventions on emotions during the COVID-19 pandemic.
Ke Wang,Amit Goldenberg,Charles Dorison,Jeremy K. Miller,Andero Uusberg,Jennifer S. Lerner,James J. Gross,Bamikole Bamikole Agesin,Márcia Bernardo,Olatz Campos,Luis Eudave,Karolina Grzech,Karolina Grzech,Daphna Hausman Ozery,Emily A. Jackson,Elkin Oswaldo Luis Garcia,Shira Meir Drexler,Anita Penić Jurković,Kafeel Rana,John Paul Wilson,Maria Antoniadi,Kermeka Desai,Zoi Gialitaki,Elizaveta Kushnir,Khaoula Nadif,Olalla Niño Bravo,Rafia Nauman,Marlies Oosterlinck,Myrto Pantazi,Natalia Pilecka,Anna Szabelska,I. M.M. van Steenkiste,Katarzyna Filip,Andreea Ioana Bozdoc,Gabriela Marcu,Gabriela Marcu,Elena Agadullina,Matus Adamkovic,Matus Adamkovic,Marta Roczniewska,Marta Roczniewska,Cecilia Reyna,Angelos P. Kassianos,Angelos P. Kassianos,Minja Westerlund,Lina Ahlgren,Sara Pöntinen,Gabriel Agboola Adetula,Pinar Dursun,Azuka Ikechukwu Arinze,Nwadiogo C. Arinze,Chisom Esther Ogbonnaya,Izuchukwu L.G. Ndukaihe,Ilker Dalgar,Handan Akkas,Paulo Manuel Macapagal,Savannah C. Lewis,Irem Metin-Orta,Francesco Foroni,Megan L. Willis,Anabela Caetano Santos,Aviv Mokady,Niv Reggev,Merve A. Kurfali,Martin R. Vasilev,Nora L. Nock,Michal Parzuchowski,Mauricio F. Espinoza Barría,Marek A. Vranka,Markéta Braun Kohlová,Ivan Ropovik,Ivan Ropovik,Mikayel Harutyunyan,Chunhui Wang,Elvin Yao,Maja Becker,Efisio Manunta,Gwenaël Kaminski,Dafne Marko,Kortnee C. Evans,David M. G. Lewis,Andrej Findor,Anaïs Thibault Landry,John Jamir Benzon R. Aruta,Manuel S. Ortiz,Zahir Vally,Zahir Vally,Ekaterina Pronizius,Martin Voracek,Claus Lamm,Maurice Grinberg,Ranran Li,Jaroslava Varella Valentova,Giovanna Mioni,Nicola Cellini,Sau-Chin Chen,Janis Zickfeld,Karis Moon,Habiba Azab,Neil Levy,Alper Karababa,Jennifer L. Beaudry,Leanne Boucher,W. Matthew Collins,Anna Louise Todsen,Kevin van Schie,Kevin van Schie,Jáchym Vintr,Jozef Bavolar,Lada Kaliska,Valerija Križanić,Lara Samojlenko,Razieh Pourafshari,Sandra J. Geiger,Julia Beitner,Lara Warmelink,Robert M. Ross,Ian D. Stephen,Thomas J. Hostler,Soufian Azouaghe,Soufian Azouaghe,Randy J. McCarthy,Anna Szala,Caterina Grano,Claudio Singh Solorzano,Gulnaz Anjum,William Jiménez-Leal,Maria Bradford,Laura Calderón Pérez,Julio Eduardo Cruz Vasquez,Oscar J. Galindo-Caballero,Juan Camilo Vargas-Nieto,Ondřej Kácha,Alexios Arvanitis,Qinyu Xiao,Rodrigo A. Cárcamo,Saša Zorjan,Zuzanna Tajchman,Iris Vilares,Jeffrey M. Pavlacic,Jonas R. Kunst,Christian K. Tamnes,Claudia C. von Bastian,Mohammad Atari,Mohammad Hasan Sharifian,Monika Hricova,Pavol Kačmár,Jana Schrötter,Rima-Maria Rahal,Noga Cohen,Saiedeh FatahModarres,Miha Zrimsek,Ilya Zakharov,Monica A. Koehn,Celia Esteban-Serna,Robert J. Calin-Jageman,Anthony J. Krafnick,Eva Štrukelj,Peder M. Isager,Jan Urban,Jaime R. Silva,Jaime R. Silva,Marcel Martončik,Sanja Batić Očovaj,Dušana Šakan,Anna O. Kuzminska,Jasna Milosevic Djordjevic,Inês Almeida,Ana Ferreira,Ljiljana B. Lazarević,Harry Manley,Danilo Zambrano Ricaurte,Renan Pereira Monteiro,Zahra Etabari,Erica D. Musser,Daniel J. Dunleavy,Weilun Chou,Hendrik Godbersen,Susana Ruiz-Fernández,Susana Ruiz-Fernández,Crystal Reeck,Carlota Batres,Komila Kirgizova,Abdumalik Muminov,Flavio Azevedo,Daniela Serrato Alvarez,Muhammad Mussaffa Butt,Jeong Min Lee,Zhang Chen,Frederick Verbruggen,Ignazio Ziano,Murat Tümer,Abdelilah C.A. Charyate,Dmitrii Dubrov,María del Carmen M.C. Tejada Rivera,Christopher L. Aberson,Bence Palfi,Mónica Alarcón Maldonado,Barbora Hubená,Asli Sacakli,Chris D. Ceary,Karley L. Richard,Gage Singer,Jennifer T. Perillo,Tonia Ballantyne,Wilson Cyrus-Lai,Maksim Fedotov,Hongfei Du,Magdalena Wielgus,Ilse L. Pit,Matej Hruška,Daniela Sousa,Balazs Aczel,Barnabas Szaszi,Sylwia Adamus,Krystian Barzykowski,Leticia Micheli,Nadya Daniela Schmidt,Andras N. Zsido,Mariola Paruzel-Czachura,Michał Białek,Marta Kowal,Agnieszka Sorokowska,Michał Misiak,Michał Misiak,Débora Jeannete Mola,María Victoria Ortiz,Pablo Sebastián Correa,Anabel Belaus,Fany Muchembled,Rafael R. Ribeiro,Patrícia Arriaga,Raquel Oliveira,Leigh Ann Vaughn,Paulina Szwed,Małgorzata Kossowska,Gabriela Czarnek,Julita Kielińska,Benedict G. Antazo,Ruben Betlehem,Stefan Stieger,Gustav Nilsonne,Gustav Nilsonne,Nicolle Simonovic,Jennifer M. Taber,Amélie Gourdon-Kanhukamwe,Artur Domurat,Keiko Ihaya,Yuki Yamada,Anum Urooj,Tripat Gill,Martin Čadek,Lisa Bylinina,Johanna Messerschmidt,Murathan Kurfalı,Adeyemi Adetula,Ekaterina Baklanova,Nihan Albayrak-Aydemir,Heather Barry Kappes,Biljana Gjoneska,Thea House,Thea House,Marc V. Jones,Jana B. Berkessel,William J. Chopik,Sami Çoksan,Martin Seehuus,Ahmed Khaoudi,Ahmed Bokkour,Kanza Ait El Arabi,Ikhlas Djamai,Aishwarya Iyer,Neha Parashar,Arca Adiguzel,Halil Emre Kocalar,Carsten Bundt,James O. Norton,Marietta Papadatou-Pastou,Anabel De La Rosa-Gómez,Vladislav Ankushev,Natalia Bogatyreva,Dmitry Grigoryev,Aleksandr Ivanov,Irina S. Prusova,Marina Romanova,Irena Sarieva,Maria Terskova,Evgeniya Hristova,Veselina Kadreva,Allison Janak,Vidar Schei,Therese E. Sverdrup,Adrian Dahl Askelund,Lina Maria Sanabria Pineda,Dajana Krupić,Carmel A. Levitan,Niklas Johannes,Nihal Ouherrou,Nicolas Say,Sladjana Sinkolova,Kristina Janjić,Marija Stojanovska,Dragana Stojanovska,Meetu Khosla,Andrew G. Thomas,Franki Y. H. Kung,Gijsbert Bijlstra,Farnaz Mosannenzadeh,Busra Bahar Balci,Ulf-Dietrich Reips,Ernest Baskin,Byurakn Ishkhanyan,Byurakn Ishkhanyan,Johanna Czamanski-Cohen,Barnaby J. W. Dixson,David Moreau,Clare A. M. Sutherland,Clare A. M. Sutherland,Hu Chuan-Peng,Chris Noone,Heather D. Flowe,Michele Anne,Steve M. J. Janssen,Marta Topor,Nadyanna M. Majeed,Yoshihiko Kunisato,Karen Yu,Shimrit Daches,Andree Hartanto,Milica Vdovic,Lisa Anton-Boicuk,Paul A. G. Forbes,Julia Kamburidis,E. Marinova,Mina Nedelcheva-Datsova,Nikolay R. Rachev,Alina Stoyanova,Kathleen Schmidt,Jordan W. Suchow,Maria Koptjevskaja-Tamm,Teodor Jernsäther,Jonas Olofsson,Olga Bialobrzeska,Magdalena Marszalek,Srinivasan Tatachari,Reza Afhami,Wilbert Law,Jan Antfolk,Barbara Žuro,Natalia Van Doren,José A. Soto,Rachel A. Searston,Jacob Miranda,Kaja Damnjanović,Siu Kit Yeung,Dino Krupić,Karlijn Hoyer,Bastian Jaeger,Dongning Ren,Gerit Pfuhl,Kristoffer Klevjer,Nadia S. Corral-Frías,Martha Frías-Armenta,Marc Yancy Lucas,Adriana Olaya Torres,Mónica Toro,Lady Grey Javela Delgado,Diego Vega,Sara Álvarez Solas,Roosevelt Vilar,Sébastien Massoni,Thomas Frizzo,Alexandre Bran,David Vaidis,Luc Vieira,Bastien Paris,Mariagrazia Capizzi,Gabriel Lins de Holanda Coelho,Anna Greenburgh,Cassie M. Whitt,Alexa M. Tullett,Xinkai Du,Leonhard Volz,Minke Jasmijn Bosma,Cemre Karaarslan,Eylül Sarıoğuz,Tara Bulut Allred,Max Korbmacher,Melissa F. Colloff,Tiago Jessé Souza de Lima,Matheus Fernando Felix Ribeiro,Jeroen P.H. Verharen,Maria Karekla,Christiana Karashiali,Naoyuki Sunami,Lisa M. Jaremka,Daniel Storage,Sumaiya Habib,Anna Studzińska,Paul H. P. Hanel,Dawn Liu Holford,Miroslav Sirota,Kelly Wolfe,Faith Chiu,Andriana Theodoropoulou,El Rim Ahn,Yijun Lin,Erin C. Westgate,Hilmar Brohmer,Gabriela Hofer,Olivier Dujols,Kevin Vezirian,Gilad Feldman,Giovanni A. Travaglino,Afroja Ahmed,Manyu Li,Jasmijn E. Bosch,Nathan Torunsky,Hui Bai,Mathi Manavalan,Xin Song,Radosław B. Walczak,Przemysław Zdybek,Maja Friedemann,Anna Dalla Rosa,Luca Kozma,Sara Alves,Samuel Lincoln Bezerra Lins,Isabel Rocha Pinto,Rita C. Correia,Peter Babincak,Gabriel Baník,Luis Miguel Rojas-Berscia,Luis Miguel Rojas-Berscia,Marco Antonio Correa Varella,J Uttley,Julie E. Beshears,Katrine Krabbe Thommesen,Behzad Behzadnia,Shawn N. Geniole,Silan Ma,Princess Lovella G. Maturan,Johannes K. Vilsmeier,Ulrich S. Tran,Sara Morales Izquierdo,Michael C. Mensink,Piotr Sorokowski,Agata Groyecka-Bernard,Agata Groyecka-Bernard,Theda Radtke,Vera Cubela Adoric,Joëlle Carpentier,Asil Ali Özdoğru,Jennifer A. Joy-Gaba,Mattie V. Hedgebeth,Tatsunori Ishii,Aaron L. Wichman,Jan Philipp Röer,Thomas Ostermann,William E. Davis,Lilian Suter,Konstantinos Papachristopoulos,Chelsea Zabel,Charles R. Ebersole,Christopher R. Chartier,Peter R. Mallik,Heather L. Urry,Erin Michelle Buchanan,Nicholas A. Coles,Maximilian Primbs,Dana M. Basnight-Brown,Hans IJzerman,Patrick S. Forscher,Hannah Moshontz +473 more
TL;DR: In this paper, the effectiveness of reappraisal, an emotion-regulation strategy that modifies how one thinks about a situation, was tested to reduce negative emotions and increase positive emotions.
Journal ArticleDOI
Male and female guppies differ in speed but not in accuracy in visual discrimination learning.
TL;DR: It is found that males’ and females’ accuracy practically overlapped in the three experiments, suggesting that the two sexes have similar discrimination learning abilities, and the existence of consistent between-sex differences in decision speed perhaps due to impulsivity rather than speed in information processing is indicated.
Journal ArticleDOI
Visual input signaling threat gains preferential access to awareness in a breaking continuous flash suppression paradigm.
TL;DR: Results reveal that stimuli that are previously paired with an electric shock break through suppression faster than comparable stimuli that were not paired withan electric shock.
References
More filters
Book
Statistical Power Analysis for the Behavioral Sciences
TL;DR: The concepts of power analysis are discussed in this paper, where Chi-square Tests for Goodness of Fit and Contingency Tables, t-Test for Means, and Sign Test are used.
Book
Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach
TL;DR: The second edition of this book is unique in that it focuses on methods for making formal statistical inference from all the models in an a priori set (Multi-Model Inference).
Journal ArticleDOI
Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses.
TL;DR: In the new version, procedures to analyze the power of tests based on single-sample tetrachoric correlations, comparisons of dependent correlations, bivariate linear regression, multiple linear regression based on the random predictor model, logistic regression, and Poisson regression are added.
Journal ArticleDOI
Bayesian data analysis.
TL;DR: A fatal flaw of NHST is reviewed and some benefits of Bayesian data analysis are introduced and illustrative examples of multiple comparisons in Bayesian analysis of variance and Bayesian approaches to statistical power are presented.
Journal ArticleDOI
Power failure: why small sample size undermines the reliability of neuroscience
Katherine S. Button,John P. A. Ioannidis,Claire Mokrysz,Brian A. Nosek,Jonathan Flint,Emma S J Robinson,Marcus R. Munafò +6 more
TL;DR: It is shown that the average statistical power of studies in the neurosciences is very low, and the consequences include overestimates of effect size and low reproducibility of results.