H
Hugo Touvron
Researcher at University of Paris
Publications - 26
Citations - 3114
Hugo Touvron is an academic researcher from University of Paris. The author has contributed to research in topics: Computer science & Contextual image classification. The author has an hindex of 13, co-authored 21 publications receiving 1267 citations. Previous affiliations of Hugo Touvron include Facebook & University of Strasbourg.
Papers
More filters
Journal ArticleDOI
LLaMA: Open and Efficient Foundation Language Models
Hugo Touvron,Thibaut Lavril,Gautier Izacard,Xavier Martinet,Marie-Anne Lachaux,Timothée Lacroix,Baptiste Roziere,Naman Goyal,Eric Hambro,Faisal Azhar,Aur'elien Rodriguez,Armand Joulin,Edouard Grave,Guillaume Lample +13 more
TL;DR: This article introduced LLaMA, a collection of foundation language models ranging from 7B to 65B parameters, and trained their models on trillions of tokens, and showed that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets.
Posted Content
Emerging Properties in Self-Supervised Vision Transformers
Mathilde Caron,Hugo Touvron,Hugo Touvron,Ishan Misra,Hervé Jégou,Julien Mairal,Piotr Bojanowski,Armand Joulin +7 more
TL;DR: In this paper, self-supervised learning provides new properties to Vision Transformer (ViT) that stand out compared to convolutional networks (convnets) beyond the fact that adapting selfsupervised methods to this architecture works particularly well, they make the following observations: first, self-vised ViT features contain explicit information about the semantic segmentation of an image, which does not emerge as clearly with supervised ViTs, nor with convnets.
Posted Content
Training data-efficient image transformers & distillation through attention
Hugo Touvron,Matthieu Cord,Matthijs Douze,Francisco Massa,Alexandre Sablayrolles,Hervé Jégou +5 more
TL;DR: In this article, a teacher-student strategy was proposed to train a convolution-free transformer on Imagenet only, achieving state-of-the-art performance on ImageNet.
Proceedings Article
ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases
TL;DR: GPSA is introduced, a form of positional self-attention which can be equipped with a "soft" convolutional inductive bias and outperforms the DeiT on ImageNet, while offering a much improved sample efficiency.
Posted Content
Fixing the train-test resolution discrepancy
TL;DR: It is experimentally validated that, for a target test resolution, using a lower train resolution offers better classification at test time, and a simple yet effective and efficient strategy to optimize the classifier performance when the train and test resolutions differ is proposed.