P
Pierre H. Richemond
Researcher at Imperial College London
Publications - 20
Citations - 3438
Pierre H. Richemond is an academic researcher from Imperial College London. The author has contributed to research in topics: Computer science & Reinforcement learning. The author has an hindex of 5, co-authored 13 publications receiving 900 citations.
Papers
More filters
Posted Content
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Jean-Bastien Grill,Florian Strub,Florent Altché,Corentin Tallec,Pierre H. Richemond,Elena Buchatskaya,Carl Doersch,Bernardo Avila Pires,Zhaohan Daniel Guo,Mohammad Gheshlaghi Azar,Bilal Piot,Koray Kavukcuoglu,Rémi Munos,Michal Valko +13 more
TL;DR: This work introduces Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning that performs on par or better than the current state of the art on both transfer and semi- supervised benchmarks.
Proceedings Article
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Jean-Bastien Grill,Florian Strub,Florent Altché,Corentin Tallec,Pierre H. Richemond,Elena Buchatskaya,Carl Doersch,Bernardo Avila Pires,Zhaohan Daniel Guo,Mohammad Gheshlaghi Azar,Bilal Piot,Koray Kavukcuoglu,Rémi Munos,Michal Valko +13 more
TL;DR: In this article, the authors investigate and provide new insights on the sampling rule called Top-Two Thompson Sampling (TTTS), and justify its use for fixed-confidence best-arm identification.
Posted Content
BYOL works even without batch statistics.
Pierre H. Richemond,Jean-Bastien Grill,Florent Altché,Corentin Tallec,Florian Strub,Andrew Brock,Samuel L. Smith,Soham De,Razvan Pascanu,Bilal Piot,Michal Valko +10 more
TL;DR: In this paper, a batch-independent normalization scheme was proposed for bootstrap-your-own-latent (BYOL) to avoid negative pairs in the training objective.
Proceedings ArticleDOI
Data Distributional Properties Drive Emergent In-Context Learning in Transformers
Stephanie C.Y. Chan,Adam Santoro,Andrew K. Lampinen,Jane X. Wang,Aaditya K Singh,Pierre H. Richemond,Jay L McClelland,Felix Hill +7 more
TL;DR: It is discovered that an additional distributional property could allow the two capabilities to co-exist in the same model – a skewed, Zipf distribution over classes – which occurs in language as well.
Journal ArticleDOI
Continuous diffusion for categorical data
Sander Dieleman,Laurent Sartran,Arman Roshannai,Nikolay Savinov,Yaroslav Ganin,Pierre H. Richemond,Arnaud Doucet,Robin Strudel,Chris Dyer,Conor Durkan,Curtis Hawthorne,Rémi Leblond,Will Grathwohl,Jonas Adler +13 more
TL;DR: The authors propose CDCD, a framework for modeling categorical data with diffusion models that are continuous both in time and input space, and demonstrate its efficacy on several language modelling tasks. But it is not suitable for language modeling in general.