T
Thomas Henighan
Researcher at SLAC National Accelerator Laboratory
Publications - 35
Citations - 14458
Thomas Henighan is an academic researcher from SLAC National Accelerator Laboratory. The author has contributed to research in topics: Phonon & Scattering. The author has an hindex of 19, co-authored 33 publications receiving 4339 citations. Previous affiliations of Thomas Henighan include Ohio State University & Stanford University.
Papers
More filters
Proceedings Article
Language Models are Few-Shot Learners
Tom B. Brown,Benjamin Mann,Nick Ryder,Melanie Subbiah,Jared Kaplan,Prafulla Dhariwal,Arvind Neelakantan,Pranav Shyam,Girish Sastry,Amanda Askell,Sandhini Agarwal,Ariel Herbert-Voss,Gretchen Krueger,Thomas Henighan,Rewon Child,Aditya Ramesh,Daniel M. Ziegler,Jeffrey Wu,Clemens Winter,Christopher Hesse,Mark Chen,Eric Sigler,Mateusz Litwin,Scott Gray,Benjamin Chess,Jack Clark,Christopher Berner,Samuel McCandlish,Alec Radford,Ilya Sutskever,Dario Amodei +30 more
TL;DR: GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as unscrambling words, using a novel word in a sentence, or performing 3-digit arithmetic.
Posted Content
Language Models are Few-Shot Learners
Tom B. Brown,Benjamin Mann,Nick Ryder,Melanie Subbiah,Jared Kaplan,Prafulla Dhariwal,Arvind Neelakantan,Pranav Shyam,Girish Sastry,Amanda Askell,Sandhini Agarwal,Ariel Herbert-Voss,Gretchen Krueger,Thomas Henighan,Rewon Child,Aditya Ramesh,Daniel M. Ziegler,Jeffrey Wu,Clemens Winter,Christopher Hesse,Mark Chen,Eric Sigler,Mateusz Litwin,Scott Gray,Benjamin Chess,Jack Clark,Christopher Berner,Samuel McCandlish,Alec Radford,Ilya Sutskever,Dario Amodei +30 more
TL;DR: This article showed that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches.
Posted Content
Scaling Laws for Neural Language Models
Jared Kaplan,Samuel McCandlish,Thomas Henighan,Tom B. Brown,Benjamin Chess,Rewon Child,Scott Gray,Alec Radford,Jeffrey Wu,Dario Amodei +9 more
TL;DR: Larger models are significantly more sample-efficient, such that optimally compute-efficient training involves training very large models on a relatively modest amount of data and stopping significantly before convergence.
Journal ArticleDOI
Ultrafast disordering of vanadium dimers in photoexcited VO2
Simon Wall,Shan Yang,Luciana Vidas,Matthieu Chollet,James M. Glownia,Michael Kozina,Tetsuo Katayama,Thomas Henighan,Mason Jiang,Timothy A. Miller,David A. Reis,David A. Reis,Lynn A. Boatner,Olivier Delaire,Mariano Trigo +14 more
TL;DR: It is shown that atomic disordering in photoexcited vanadium dioxide (VO2) is central to the transition mechanism and that, after photoexcitation, the system explores a large volume of phase space on a time scale comparable to that of a single phonon oscillation.
Posted Content
Scaling Laws for Autoregressive Generative Modeling
Thomas Henighan,Jared Kaplan,Mor Katz,Mark Chen,Christopher Hesse,Jacob Jackson,Heewoo Jun,Tom B. Brown,Prafulla Dhariwal,Scott Gray,Chris Hallacy,Benjamin Mann,Alec Radford,Aditya Ramesh,Nick Ryder,Daniel M. Ziegler,John Schulman,Dario Amodei,Samuel McCandlish +18 more
TL;DR: The case that scaling laws have important implications for neural network performance, including on downstream tasks is strengthened, as empirical scaling laws for the cross-entropy loss are identified.