Open AccessPosted Content
Redefine Statistical Significance
Daniel J. Benjamin,James O. Berger,Magnus Johannesson,Brian A. Nosek,Eric-Jan Wagenmakers,Richard A. Berk,Kenneth A. Bollen,Björn Brembs,Lawrence D. Brown,Colin F. Camerer,David Cesarini,Christopher D. Chambers,Merlise A. Clyde,Thomas D. Cook,Paul De Boeck,Zoltan Dienes,Anna Dreber,Kenny Easwaran,Charles Efferson,Ernst Fehr,Fiona Fidler,Andy P. Field,Malcom Forster,Edward I. George,Tarun Ramadorai,Richard Gonzalez,Steven N. Goodman,Edwin J. Green,Donald P. Green,Anthony G. Greenwald,Jarrod D. Hadfield,Larry V. Hedges,Leonhard Held,Teck Hau Ho,Herbert Hoijtink,James Holland Jones,Daniel J. Hruschka,Kosuke Imai,Guido W. Imbens,John P. A. Ioannidis,Minjeong Jeon,Michael Kirchler,David Laibson,John A. List,Roderick J. A. Little,Arthur Lupia,Edouard Machery,Scott E. Maxwell,Michael A. McCarthy,Don A. Moore,Stephen L. Morgan,Marcus R. Munafò,Shinichi Nakagawa,Brendan Nyhan,Timothy H. Parker,Luis R. Pericchi,Marco Perugini,Jeffrey N. Rouder,Judith Rousseau,Victoria Savalei,Felix D. Schönbrodt,Thomas Sellke,Betsy Sinclair,Dustin Tingley,Trisha Van Zandt,Simine Vazire,Duncan J. Watts,Christopher Winship,Robert L. Wolpert,Yu Xie,Cristobal Young,Jonathan Zinman,Valen E. Johnson +72 more
TLDR
This article proposed to change the default P-value threshold for statistical significance for claims of new discoveries from 0.05 to 0.005, which is the threshold used in this paper.Abstract:
We propose to change the default P-value threshold for statistical significance for claims of new discoveries from 0.05 to 0.005.read more
Citations
More filters
Journal ArticleDOI
Evaluating Effect Size in Psychological Research: Sense and Nonsense:
David C. Funder,Daniel J. Ozer +1 more
TL;DR: The most common mistakes being to describe effect sizes in ways that are uninformative (e.g., using arbitrary standards) or misleading as mentioned in this paper, i.e., squa...
Journal ArticleDOI
Moving beyond P values: data analysis with estimation graphics
Joses Ho,Tayfun Tumkaya,Tayfun Tumkaya,Sameer Aryal,Sameer Aryal,Hyungwon Choi,Hyungwon Choi,Adam Claridge-Chang,Adam Claridge-Chang +8 more
TL;DR: The estimation graphic is described, a plot that displays an experimental dataset’s complete statistical information that avoids dichotomous thinking and introduces software that makes high-quality estimation graphics available to all.
Journal ArticleDOI
The preregistration revolution.
TL;DR: Widespread adoption of preregistration will increase distinctiveness between hypothesis generation and hypothesis testing and will improve the credibility of research findings.
Journal ArticleDOI
NetworkAnalyst 3.0: a visual analytics platform for comprehensive gene expression profiling and meta-analysis.
TL;DR: The growing application of gene expression profiling demands powerful yet user-friendly bioinformatics tools to support systems-level data understanding and NetworkAnalyst was first released in 2014 to address the key need for interpreting gene expression data within the context of protein-protein interaction networks.
Journal ArticleDOI
Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015
Colin F. Camerer,Anna Dreber,Felix Holzmeister,Teck-Hua Ho,Jürgen Huber,Magnus Johannesson,Michael Kirchler,Gideon Nave,Brian A. Nosek,Brian A. Nosek,Thomas Pfeiffer,Adam Altmejd,Nick Buttrick,Nick Buttrick,Taizan Chan,Yiling Chen,Eskil Forsell,Anup Gampa,Anup Gampa,Emma Heikensten,Lily Hummer,Taisuke Imai,Siri Isaksson,Dylan Manfredi,Julia Rose,Eric-Jan Wagenmakers,Hang Wu +26 more
TL;DR: It is found that peer beliefs of replicability are strongly related to replicable, suggesting that the research community could predict which results would replicate and that failures to replicate were not the result of chance alone.
References
More filters
Journal ArticleDOI
Estimating the reproducibility of psychological science
Alexander A. Aarts,Joanna E. Anderson,Christopher J. Anderson,Peter Raymond Attridge,Peter Raymond Attridge,Angela S. Attwood,Jordan Axt,Molly Babel,Štěpán Bahník,Erica Baranski,Michael Barnett-Cowan,Elizabeth Bartmess,Jennifer S. Beer,Raoul Bell,Heather Bentley,Leah Beyan,Grace Binion,Grace Binion,Denny Borsboom,Annick Bosch,Frank A. Bosco,Sara Bowman,Mark J. Brandt,Erin L Braswell,Hilmar Brohmer,Benjamin T. Brown,Kristina G. Brown,Jovita Brüning,Jovita Brüning,Ann Calhoun-Sauls,Shannon P. Callahan,Elizabeth Chagnon,Jesse Chandler,Jesse Chandler,Christopher R. Chartier,Felix Cheung,Felix Cheung,Cody D. Christopherson,Linda Cillessen,Russ Clay,Hayley M. D. Cleary,Mark D. Cloud,Michael Conn,Johanna Cohoon,Simon Columbus,Andreas Cordes,Giulio Costantini,Leslie Cramblet Alvarez,Ed Cremata,Jan Crusius,Jamie DeCoster,Michelle A. DeGaetano,Nicolás Delia Penna,Bobby Den Bezemer,Marie K. Deserno,Olivia Devitt,Laura Dewitte,David G. Dobolyi,Geneva T. Dodson,M. Brent Donnellan,Ryan Donohue,Rebecca A. Dore,Angela Rachael Dorrough,Angela Rachael Dorrough,Anna Dreber,Michelle Dugas,Elizabeth W. Dunn,Kayleigh E Easey,Sylvia Eboigbe,Casey Eggleston,Jo Embley,Sacha Epskamp,Timothy M. Errington,Vivien Estel,Frank J. Farach,Jenelle Feather,Anna Fedor,Belén Fernández-Castilla,Susann Fiedler,James G. Field,Stanka A. Fitneva,Taru Flagan,Amanda L. Forest,Eskil Forsell,Joshua D. Foster,Michael C. Frank,Rebecca S. Frazier,Heather M. Fuchs,Philip A. Gable,Jeff Galak,Elisa Maria Galliani,Anup Gampa,Sara García,Douglas Gazarian,Elizabeth Gilbert,Roger Giner-Sorolla,Andreas Glöckner,Andreas Glöckner,Lars Goellner,Jin X. Goh,Rebecca M. Goldberg,Patrick T. Goodbourn,Shauna Gordon-McKeon,Bryan Gorges,Jessie Gorges,Justin Goss,Jesse Graham,James A. Grange,Jeremy R. Gray,Chris H.J. Hartgerink,Joshua K. Hartshorne,Fred Hasselman,Timothy Hayes,Emma Heikensten,Felix Henninger,Felix Henninger,John Hodsoll,Taylor Holubar,Gea Hoogendoorn,Denise J. Humphries,Cathy On-Ying Hung,Nathali Immelman,Vanessa C. Irsik,Georg Jahn,Frank Jäkel,Marc Jekel,Magnus Johannesson,Larissa Gabrielle Johnson,David J. Johnson,Kate M. Johnson,William J. Johnston,Kai J. Jonas,Jennifer A. Joy-Gaba,Heather Barry Kappes,Kim Kelso,Mallory C. Kidwell,Seung K. Kim,Matthew W. Kirkhart,Bennett Kleinberg,Bennett Kleinberg,Goran Knežević,Franziska Maria Kolorz,Jolanda J. Kossakowski,Robert Krause,Job Krijnen,Tim Kuhlmann,Yoram K. Kunkels,Megan M. Kyc,Calvin K. Lai,Aamir Laique,Daniel Lakens,Kristin A. Lane,Bethany Lassetter,Ljiljana B. Lazarević,Etienne P. Le Bel,Key Jung Lee,Minha Lee,Kristi M. Lemm,Carmel A. Levitan,Melissa Lewis,Lin Lin,Stephanie C. Lin,Matthias Lippold,Darren Loureiro,Ilse Luteijn,Sean P. Mackinnon,Heather N. Mainard,Denise C. Marigold,Daniel P. Martin,Tylar Martinez,E. J. Masicampo,Joshua J. Matacotta,Maya B. Mathur,Michael May,Michael May,Nicole Mechin,Pranjal H. Mehta,Johannes M. Meixner,Johannes M. Meixner,Alissa Melinger,Jeremy K. Miller,Mallorie Miller,Katherine Moore,Katherine Moore,Marcus Möschl,Matt Motyl,Stephanie M. Müller,Marcus R. Munafò,Koen Ilja Neijenhuijs,Taylor Nervi,Gandalf Nicolas,Gustav Nilsonne,Gustav Nilsonne,Brian A. Nosek,Brian A. Nosek,Michèle B. Nuijten,Catherine Olsson,Catherine Olsson,Colleen Osborne,Lutz Ostkamp,Misha Pavel,Ian S. Penton-Voak,Olivia Perna,Cyril Pernet,Marco Perugini,R. Nathan Pipitone,Michael C. Pitts,Franziska Plessow,Franziska Plessow,Jason M. Prenoveau,Rima-Maria Rahal,Rima-Maria Rahal,Kate A. Ratliff,David Reinhard,Frank Renkewitz,Ashley A. Ricker,Anastasia E. Rigney,Andrew M Rivers,Mark A. Roebke,Abraham M. Rutchick,Robert S. Ryan,Onur Sahin,Anondah R. Saide,Gillian M. Sandstrom,David Santos,David Santos,Rebecca Saxe,René Schlegelmilch,René Schlegelmilch,Kathleen Schmidt,Sabine Scholz,Larissa Seibel,Dylan Selterman,Samuel Shaki,William B. Simpson,H. Colleen Sinclair,Jeanine L. M. Skorinko,Agnieszka Slowik,Joel S. Snyder,Courtney K. Soderberg,Carina Sonnleitner,Nick Spencer,Jeffrey R. Spies,Sara Steegen,Stefan Stieger,Nina Strohminger,Gavin Brent Sullivan,Thomas Talhelm,Megan Tapia,Anniek M. te Dorsthorst,Manuela Thomae,Manuela Thomae,Sarah L. Thomas,Pia Tio,Frits Traets,Steve N.H. Tsang,Francis Tuerlinckx,Paul J. Turchan,Milan Valášek,Anna E. Van't Veer,Robbie C. M. van Aert,Marcel A.L.M. van Assen,Riet van Bork,Mathijs Van De Ven,Don van den Bergh,Marije van der Hulst,Roel van Dooren,Johnny van Doorn,Daan R. van Renswoude,Hedderik van Rijn,Wolf Vanpaemel,Alejandro Vásquez Echeverría,Melissa Vazquez,Natalia Vélez,Marieke Vermue,Mark Verschoor,Michelangelo Vianello,Martin Voracek,Gina Vuu,Eric-Jan Wagenmakers,Joanneke Weerdmeester,Ashlee Welsh,Erin C. Westgate,Joeri Wissink,Michael J. Wood,Andy T. Woods,Andy T. Woods,Emily M. Wright,Sining Wu,Marcel Zeelenberg,Kellylynn Zuni +290 more
TL;DR: A large-scale assessment suggests that experimental reproducibility in psychology leaves a lot to be desired, and correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.
Journal ArticleDOI
The ASA's Statement on p-Values: Context, Process, and Purpose
TL;DR: The American Statistical Association (ASA) released a policy statement on p-values and statistical significance in 2015 as discussed by the authors, which was based on a discussion with the ASA Board of Trustees and concerned with reproducibility and replicability of scientific conclusions.
Journal ArticleDOI
Beyond Power Calculations Assessing Type S (Sign) and Type M (Magnitude) Errors
Andrew Gelman,John B. Carlin +1 more
TL;DR: The largest challenge in a design calculation: coming up with reasonable estimates of plausible effect sizes based on external information is discussed, and design calculations in which the probability of an estimate being in the wrong direction and the magnitude of an effect might be overestimated are recommended.
Journal ArticleDOI
Evaluating replicability of laboratory experiments in economics
Colin F. Camerer,Anna Dreber,Eskil Forsell,Teck Ho,Teck Ho,Jürgen Huber,Magnus Johannesson,Michael Kirchler,Johan Almenberg,Adam Altmejd,Taizan Chan,Emma Heikensten,Felix Holzmeister,Taisuke Imai,Siri Isaksson,Gideon Nave,Thomas Pfeiffer,Michael Razen,Hang Wu +18 more
TL;DR: To contribute data about replicability in economics, 18 studies published in the American Economic Review and the Quarterly Journal of Economics between 2011 and 2014 are replicated, finding that two-thirds of the 18 studies examined yielded replicable estimates of effect size and direction.
Journal ArticleDOI
Reproducibility in Science: Improving the Standard for Basic and Preclinical Research
TL;DR: The reproducibility problems in basic and preclinical biomedical research are reviewed, some of the complexities are highlighted, and potential solutions that may help improve research quality and reproducible are discussed.
Related Papers (5)
Redefine Statistical Significance
Daniel J. Benjamin,James O. Berger,Magnus Johannesson,Brian A. Nosek,Eric-Jan Wagenmakers,Richard A. Berk,Kenneth A. Bollen,Björn Brembs,Lawrence D. Brown,Colin F. Camerer,David Cesarini,Christopher D. Chambers,Merlise A. Clyde,Thomas D. Cook,Paul De Boeck,Zoltan Dienes,Anna Dreber,Kenny Easwaran,Charles Efferson,Ernst Fehr,Fiona Fidler,Andy P. Field,Malcom Forster,Edward I. George,Tarun Ramadorai,Richard Gonzalez,Steven N. Goodman,Edwin J. Green,Donald P. Green,Anthony G. Greenwald,Jarrod D. Hadfield,Larry V. Hedges,Leonhard Held,Teck Hau Ho,Herbert Hoijtink,James Holland Jones,Daniel J. Hruschka,Kosuke Imai,Guido W. Imbens,John P. A. Ioannidis,Minjeong Jeon,Michael Kirchler,David Laibson,John A. List,Roderick J. A. Little,Arthur Lupia,Edouard Machery,Scott E. Maxwell,Michael A. McCarthy,Don A. Moore,Stephen L. Morgan,Marcus R. Munafò,Shinichi Nakagawa,Brendan Nyhan,Timothy H. Parker,Luis R. Pericchi,Marco Perugini,Jeffrey N. Rouder,Judith Rousseau,Victoria Savalei,Felix D. Schönbrodt,Thomas Sellke,Betsy Sinclair,Dustin Tingley,Trisha Van Zandt,Simine Vazire,Duncan J. Watts,Christopher Winship,Robert L. Wolpert,Yu Xie,Cristobal Young,Jonathan Zinman,Valen E. Johnson +72 more