William Saunders

TL;DR: Evaluation of OpenAI's GPT models, Google-internal dense transformer architectures, and Switch-style sparse transformers on BIG-bench, across model sizes spanning millions to hundreds of billions of parameters finds that model performance and calibration both improve with scale, but are poor in absolute terms.

TL;DR: This work formalizes human intervention for RL and shows how to reduce the human labor required by training a supervised learner to imitate the human's intervention decisions, and outlines extensions of the scheme that are necessary if the authors are to train model-free agents without a single catastrophe.

TL;DR: In this article, the authors explore how human oversight can be combined with a supervised learning system to prevent catastrophic events during training and demonstrate this scheme on Atari games, with a Deep RL agent being overseen by a human for four hours.

TL;DR: Codex as discussed by the authors is a GPT language model fine-tuned on publicly available code from GitHub, and study its Python code-writing capabilities, showing that repeated sampling from the model is a surprisingly effective strategy for producing working solutions to difficult prompts.

TL;DR: This work fine-tune large language models to write natural language critiques (natural language critical comments) using behavioral cloning, and suggests that even large models may still have relevant knowledge they cannot or do not articulate as critiques with both topic-based summarization and synthetic tasks.