Chris Hallacy

TL;DR: In this article, a pre-training task of predicting which caption goes with which image is used to learn SOTA image representations from scratch on a dataset of 400 million (image, text) pairs collected from the internet.

TL;DR: The case that scaling laws have important implications for neural network performance, including on downstream tasks is strengthened, as empirical scaling laws for the cross-entropy loss are identified.

TL;DR: It is shown that contrastive pre-training on unsupervised data at scale leads to high quality vector representations of text and code.

TL;DR: In this paper, a pre-training task of predicting which caption goes with which image is used to learn SOTA image representations from scratch on a dataset of 400 million (image, text) pairs collected from the internet.

TL;DR: A simple active learning-based algorithm that outperforms all the current data subset selection algorithms on the benchmark tasks and finds that it is crucial to find a balance between easy- to-classify and hard-to- classify examples when selecting a subset.