Measuring the predictability of life outcomes with a scientific mass collaboration.
Matthew J. Salganik,Ian Lundberg,Alexander T. Kindel,Caitlin Ahearn,Khaled AlGhoneim,Abdullah Almaatouq,Drew Altschul,Jennie E. Brand,Nicole Bohme Carnegie,Ryan James Compton,Debanjan Datta,Thomas Davidson,Anna Filippova,Connor Gilroy,Brian J. Goode,Eaman Jahani,Ridhi Kashyap,Antje Kirchner,Stephen McKay,Allison C. Morgan,Alex Pentland,Kivan Polimis,Louis Raes,Daniel E Rigobon,Claudia V. Roberts,Diana Stanescu,Yoshihiko Suhara,Adaner Usmani,Erik H. Wang,Muna Adem,Abdulla Alhajri,Bedoor K. AlShebli,Redwane Amin,Ryan Amos,Lisa P. Argyle,Livia Baer-Bositis,Moritz Büchi,Bo-Ryehn Chung,William Eggert,Gregory Faletto,Zhilin Fan,Jeremy Freese,Tejomay Gadgil,Josh Gagné,Yue Gao,Andrew Halpern-Manners,Sonia P Hashim,Sonia Hausen,Guanhua He,Kimberly Higuera,Bernie Hogan,Ilana M. Horwitz,Lisa M Hummel,Naman Jain,Kun Jin,David Jurgens,Patrick Kaminski,Areg Karapetyan,Areg Karapetyan,E H Kim,Ben Leizman,Naijia Liu,Malte Möser,Andrew E Mack,Mayank Mahajan,Noah Mandell,Helge Marahrens,Diana Mercado-Garcia,Viola Mocz,Katariina Mueller-Gastell,Ahmed Musse,Qiankun Niu,William Nowak,Hamidreza Omidvar,Andrew Or,Karen Ouyang,Katy M. Pinto,Ethan Porter,Kristin E. Porter,Crystal Qian,Tamkinat Rauf,Anahit Sargsyan,Thomas Schaffner,Landon Schnabel,Bryan Schonfeld,Ben Sender,Jonathan D Tang,Emma Tsurkov,Austin van Loon,Onur Varol,Onur Varol,Xiafei Wang,Zhi Wang,Julia Wang,Flora Wang,Samantha Weissman,Kirstie Whitaker,Kirstie Whitaker,Maria Wolters,Wei Lee Woon,James M. Wu,Catherine Wu,Kengran Yang,Jingwen Yin,Bingyu Zhao,Chenyun Zhu,Jeanne Brooks-Gunn,Barbara E. Engelhardt,Moritz Hardt,Dean Knox,Karen Levy,Arvind Narayanan,Brandon M. Stewart,Duncan J. Watts,Sara McLanahan +114 more
Reads0
Chats0
TLDR
Practical limits to the predictability of life outcomes in some settings are suggested and the value of mass collaborations in the social sciences is illustrated.Abstract:
How predictable are life trajectories? We investigated this question with a scientific mass collaboration using the common task method; 160 teams built predictive models for six life outcomes using data from the Fragile Families and Child Wellbeing Study, a high-quality birth cohort study. Despite using a rich dataset and applying machine-learning methods optimized for prediction, the best predictions were not very accurate and were only slightly better than those from a simple benchmark model. Within each outcome, prediction error was strongly associated with the family being predicted and weakly associated with the technique used to generate the prediction. Overall, these results suggest practical limits to the predictability of life outcomes in some settings and illustrate the value of mass collaborations in the social sciences.read more
Citations
More filters
Journal ArticleDOI
Machine learning uncovers the most robust self-report predictors of relationship quality across 43 longitudinal couples studies
Samantha Joel,Paul W. Eastwick,Colleen J. Allison,Ximena B. Arriaga,Zachary G. Baker,Eran Bar-Kalifa,Sophie Bergeron,Gurit E. Birnbaum,Rebecca L. Brock,Claudia Chloe Brumbaugh,Cheryl L. Carmichael,Serena Chen,Jennifer Clarke,Rebecca J. Cobb,Michael K. Coolsen,Jody L. Davis,David C. de Jong,Anik Debrot,Eva C. DeHaas,Jaye L. Derrick,Jami Eller,Marie Joelle Estrada,Ruddy Faure,Eli J. Finkel,R. Chris Fraley,Shelly L. Gable,Reuma Gadassi-Polack,Yuthika U. Girme,Amie M. Gordon,Courtney L. Gosnell,Matthew D. Hammond,Peggy A. Hannon,Cheryl Harasymchuk,Wilhelm Hofmann,Andrea B. Horn,Emily A. Impett,Jeremy P. Jamieson,Dacher Keltner,James J. Kim,Jeffrey L. Kirchner,Esther S. Kluwer,Esther S. Kluwer,Madoka Kumashiro,Grace M. Larson,Gal Lazarus,Jill M. Logan,Laura B. Luchies,Geoff MacDonald,Laura V. Machia,Michael R. Maniaci,Jessica A. Maxwell,Moran Mizrahi,Amy Muise,Sylvia Niehuis,Brian G. Ogolsky,C. Rebecca Oldham,Nickola C. Overall,Meinrad Perrez,Brett J. Peters,Paula R. Pietromonaco,Sally I. Powers,Thery Prok,Rony Pshedetzky-Shochat,Eshkol Rafaeli,Eshkol Rafaeli,Erin L. Ramsdell,Maija Reblin,Michael Reicherts,Alan Reifman,Harry T. Reis,Galena K. Rhoades,William S. Rholes,Francesca Righetti,Lindsey M. Rodriguez,Ron Rogge,Natalie O. Rosen,Darby E. Saxbe,Haran Sened,Jeffry A. Simpson,Erica B. Slotter,Scott M. Stanley,Shevaun L. Stocker,Cathy Surra,Hagar Ter Kuile,Allison A. Vaughn,Amanda M. Vicary,Mariko L. Visserman,Mariko L. Visserman,Scott T. Wolf +88 more
TL;DR: The findings imply that the sum of all individual differences and partner experiences exert their influence on relationship quality via a person’s own relationship-specific experiences, and effects due to moderation byindividual differences and moderation by partner-reports may be quite small.
Journal ArticleDOI
Algorithmic injustice: a relational ethics approach.
TL;DR: In this article, a fundamental shift from rational to relational-in thinking about personhood, data, justice, and everything in between, and places ethics as something that goes above and beyond technical solutions is proposed.
Journal ArticleDOI
The American Occupational Structure
TL;DR: The American Occupational Structure (AOS) as discussed by the authors is the classic source of empirical information on the patterns of occupational achievement in American society and is renowned for its pioneering methods of statistical analysis as well as for its far-reaching conclusions about social stratification and occupational mobility in the United States.
Journal ArticleDOI
Early life stress and development: potential mechanisms for adverse outcomes.
Karen E. Smith,Seth D. Pollak +1 more
TL;DR: Assessment of factors that influence children’s interpretation of stressors, along with stressful events, has the potential to provide further insight into the mechanisms contributing to individual differences in neurodevelopmental effects of early life stress.
Journal ArticleDOI
Integrating explanation and prediction in computational social science.
Jake M. Hofman,Duncan J. Watts,Susan Athey,Filiz Garip,Thomas L. Griffiths,Jon Kleinberg,Helen Margetts,Helen Margetts,Sendhil Mullainathan,Matthew J. Salganik,Simine Vazire,Alessandro Vespignani,Tal Yarkoni +12 more
TL;DR: In this article, the authors propose a schema for thinking about research activities along two dimensions-the extent to which work is explanatory, focusing on identifying and estimating causal effects, and the degree of consideration given to testing predictions of outcomes.
References
More filters
Book
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
TL;DR: In this paper, the authors describe the important ideas in these areas in a common conceptual framework, and the emphasis is on concepts rather than mathematics, with a liberal use of color graphics.
Journal ArticleDOI
Estimating the reproducibility of psychological science
Alexander A. Aarts,Joanna E. Anderson,Christopher J. Anderson,Peter Raymond Attridge,Peter Raymond Attridge,Angela S. Attwood,Jordan Axt,Molly Babel,Štěpán Bahník,Erica Baranski,Michael Barnett-Cowan,Elizabeth Bartmess,Jennifer S. Beer,Raoul Bell,Heather Bentley,Leah Beyan,Grace Binion,Grace Binion,Denny Borsboom,Annick Bosch,Frank A. Bosco,Sara Bowman,Mark J. Brandt,Erin L Braswell,Hilmar Brohmer,Benjamin T. Brown,Kristina G. Brown,Jovita Brüning,Jovita Brüning,Ann Calhoun-Sauls,Shannon P. Callahan,Elizabeth Chagnon,Jesse Chandler,Jesse Chandler,Christopher R. Chartier,Felix Cheung,Felix Cheung,Cody D. Christopherson,Linda Cillessen,Russ Clay,Hayley M. D. Cleary,Mark D. Cloud,Michael Conn,Johanna Cohoon,Simon Columbus,Andreas Cordes,Giulio Costantini,Leslie Cramblet Alvarez,Ed Cremata,Jan Crusius,Jamie DeCoster,Michelle A. DeGaetano,Nicolás Delia Penna,Bobby Den Bezemer,Marie K. Deserno,Olivia Devitt,Laura Dewitte,David G. Dobolyi,Geneva T. Dodson,M. Brent Donnellan,Ryan Donohue,Rebecca A. Dore,Angela Rachael Dorrough,Angela Rachael Dorrough,Anna Dreber,Michelle Dugas,Elizabeth W. Dunn,Kayleigh E Easey,Sylvia Eboigbe,Casey Eggleston,Jo Embley,Sacha Epskamp,Timothy M. Errington,Vivien Estel,Frank J. Farach,Jenelle Feather,Anna Fedor,Belén Fernández-Castilla,Susann Fiedler,James G. Field,Stanka A. Fitneva,Taru Flagan,Amanda L. Forest,Eskil Forsell,Joshua D. Foster,Michael C. Frank,Rebecca S. Frazier,Heather M. Fuchs,Philip A. Gable,Jeff Galak,Elisa Maria Galliani,Anup Gampa,Sara García,Douglas Gazarian,Elizabeth Gilbert,Roger Giner-Sorolla,Andreas Glöckner,Andreas Glöckner,Lars Goellner,Jin X. Goh,Rebecca M. Goldberg,Patrick T. Goodbourn,Shauna Gordon-McKeon,Bryan Gorges,Jessie Gorges,Justin Goss,Jesse Graham,James A. Grange,Jeremy R. Gray,Chris H.J. Hartgerink,Joshua K. Hartshorne,Fred Hasselman,Timothy Hayes,Emma Heikensten,Felix Henninger,Felix Henninger,John Hodsoll,Taylor Holubar,Gea Hoogendoorn,Denise J. Humphries,Cathy On-Ying Hung,Nathali Immelman,Vanessa C. Irsik,Georg Jahn,Frank Jäkel,Marc Jekel,Magnus Johannesson,Larissa Gabrielle Johnson,David J. Johnson,Kate M. Johnson,William J. Johnston,Kai J. Jonas,Jennifer A. Joy-Gaba,Heather Barry Kappes,Kim Kelso,Mallory C. Kidwell,Seung K. Kim,Matthew W. Kirkhart,Bennett Kleinberg,Bennett Kleinberg,Goran Knežević,Franziska Maria Kolorz,Jolanda J. Kossakowski,Robert Krause,Job Krijnen,Tim Kuhlmann,Yoram K. Kunkels,Megan M. Kyc,Calvin K. Lai,Aamir Laique,Daniel Lakens,Kristin A. Lane,Bethany Lassetter,Ljiljana B. Lazarević,Etienne P. Le Bel,Key Jung Lee,Minha Lee,Kristi M. Lemm,Carmel A. Levitan,Melissa Lewis,Lin Lin,Stephanie C. Lin,Matthias Lippold,Darren Loureiro,Ilse Luteijn,Sean P. Mackinnon,Heather N. Mainard,Denise C. Marigold,Daniel P. Martin,Tylar Martinez,E. J. Masicampo,Joshua J. Matacotta,Maya B. Mathur,Michael May,Michael May,Nicole Mechin,Pranjal H. Mehta,Johannes M. Meixner,Johannes M. Meixner,Alissa Melinger,Jeremy K. Miller,Mallorie Miller,Katherine Moore,Katherine Moore,Marcus Möschl,Matt Motyl,Stephanie M. Müller,Marcus R. Munafò,Koen Ilja Neijenhuijs,Taylor Nervi,Gandalf Nicolas,Gustav Nilsonne,Gustav Nilsonne,Brian A. Nosek,Brian A. Nosek,Michèle B. Nuijten,Catherine Olsson,Catherine Olsson,Colleen Osborne,Lutz Ostkamp,Misha Pavel,Ian S. Penton-Voak,Olivia Perna,Cyril Pernet,Marco Perugini,R. Nathan Pipitone,Michael C. Pitts,Franziska Plessow,Franziska Plessow,Jason M. Prenoveau,Rima-Maria Rahal,Rima-Maria Rahal,Kate A. Ratliff,David Reinhard,Frank Renkewitz,Ashley A. Ricker,Anastasia E. Rigney,Andrew M Rivers,Mark A. Roebke,Abraham M. Rutchick,Robert S. Ryan,Onur Sahin,Anondah R. Saide,Gillian M. Sandstrom,David Santos,David Santos,Rebecca Saxe,René Schlegelmilch,René Schlegelmilch,Kathleen Schmidt,Sabine Scholz,Larissa Seibel,Dylan Selterman,Samuel Shaki,William B. Simpson,H. Colleen Sinclair,Jeanine L. M. Skorinko,Agnieszka Slowik,Joel S. Snyder,Courtney K. Soderberg,Carina Sonnleitner,Nick Spencer,Jeffrey R. Spies,Sara Steegen,Stefan Stieger,Nina Strohminger,Gavin Brent Sullivan,Thomas Talhelm,Megan Tapia,Anniek M. te Dorsthorst,Manuela Thomae,Manuela Thomae,Sarah L. Thomas,Pia Tio,Frits Traets,Steve N.H. Tsang,Francis Tuerlinckx,Paul J. Turchan,Milan Valášek,Anna E. Van't Veer,Robbie C. M. van Aert,Marcel A.L.M. van Assen,Riet van Bork,Mathijs Van De Ven,Don van den Bergh,Marije van der Hulst,Roel van Dooren,Johnny van Doorn,Daan R. van Renswoude,Hedderik van Rijn,Wolf Vanpaemel,Alejandro Vásquez Echeverría,Melissa Vazquez,Natalia Vélez,Marieke Vermue,Mark Verschoor,Michelangelo Vianello,Martin Voracek,Gina Vuu,Eric-Jan Wagenmakers,Joanneke Weerdmeester,Ashlee Welsh,Erin C. Westgate,Joeri Wissink,Michael J. Wood,Andy T. Woods,Andy T. Woods,Emily M. Wright,Sining Wu,Marcel Zeelenberg,Kellylynn Zuni +290 more
TL;DR: A large-scale assessment suggests that experimental reproducibility in psychology leaves a lot to be desired, and correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.
Book
The American occupational structure
Peter M. Blau,Otis Dudley Duncan +1 more
TL;DR: The American Occupational Structure is renowned for its pioneering methods of statistical analysis as well as for its far-reaching conclusions about social stratification and occupational mobility in the United States.
Journal ArticleDOI
Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead
TL;DR: This Perspective clarifies the chasm between explaining black boxes and using inherently interpretable models, outlines several key reasons why explainable black boxes should be avoided in high-stakes decisions, identifies challenges to interpretable machine learning, and provides several example applications whereinterpretable models could potentially replace black box models in criminal justice, healthcare and computer vision.
Journal ArticleDOI
Clinical versus actuarial judgment
TL;DR: Research comparing these two approaches to decision-making shows the actuarial method to be superior, factors underlying the greater accuracy of actuarial methods, sources of resistance to the scientific findings, and the benefits of increased reliance on actuarial approaches are discussed.