G
Guillaume Wenzek
Researcher at Facebook
Publications - 22
Citations - 5252
Guillaume Wenzek is an academic researcher from Facebook. The author has contributed to research in topics: Computer science & Machine translation. The author has an hindex of 10, co-authored 18 publications receiving 2192 citations.
Papers
More filters
Proceedings ArticleDOI
Unsupervised Cross-lingual Representation Learning at Scale
Alexis Conneau,Kartikay Khandelwal,Naman Goyal,Vishrav Chaudhary,Guillaume Wenzek,Francisco Guzmán,Edouard Grave,Myle Ott,Luke Zettlemoyer,Veselin Stoyanov +9 more
TL;DR: It is shown that pretraining multilingual language models at scale leads to significant performance gains for a wide range of cross-lingual transfer tasks, and the possibility of multilingual modeling without sacrificing per-language performance is shown for the first time.
Posted Content
Unsupervised Cross-lingual Representation Learning at Scale.
Alexis Conneau,Kartikay Khandelwal,Naman Goyal,Vishrav Chaudhary,Guillaume Wenzek,Francisco Guzmán,Edouard Grave,Myle Ott,Luke Zettlemoyer,Veselin Stoyanov +9 more
TL;DR: This paper showed that pretraining multilingual language models at scale leads to significant performance gains for a wide range of cross-lingual transfer tasks and proposed a Transformer-based masked language model on one hundred languages, using more than two terabytes of filtered CommonCrawl data.
Posted Content
Beyond English-Centric Multilingual Machine Translation
Angela Fan,Shruti Bhosale,Holger Schwenk,Zhiyi Ma,Ahmed El-Kishky,Siddharth Goyal,Mandeep Baines,Onur Celebi,Guillaume Wenzek,Vishrav Chaudhary,Naman Goyal,Tom Birch,Vitaliy Liptchinsky,Sergey Edunov,Edouard Grave,Michael Auli,Armand Joulin +16 more
TL;DR: This work creates a true Many-to-Many multilingual translation model that can translate directly between any pair of 100 languages and explores how to effectively increase model capacity through a combination of dense scaling and language-specific sparse parameters to create high quality models.
Proceedings Article
CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data
Guillaume Wenzek,Marie-Anne Lachaux,Alexis Conneau,Vishrav Chaudhary,Francisco Guzmán,Armand Joulin,Edouard Grave +6 more
TL;DR: An automatic pipeline to extract massive high-quality monolingual datasets from Common Crawl for a variety of languages by following the data processing introduced in fastText, that deduplicates documents and identifies their language.
Journal ArticleDOI
No Language Left Behind: Scaling Human-Centered Machine Translation
Nllb team,Marta R. Costa-jussà,James Cross,Onur cCelebi,Maha Elbayad,Kenneth Heafield,Kevin Heffernan,Elahe Kalbassi,Janice Si-Man Lam,Daniel Licht,Jean Maillard,Anna Sun,Skyler Wang,Guillaume Wenzek,Alison Youngblood,Bapi Akula,Loïc Barrault,Gabriel Mejia Gonzalez,Prangthip Hansanti,John Hoffman,Semarley Jarrett,Kaushik Ram Sadagopan,Dirk Rowe,Shannon Spruit,Chau Tran,Pierre Andrews,Necip Fazil Ayan,Shruti Bhosale,Sergey Edunov,Angela Fan,Cynthia Gao,Vedanuj Goswami,Francisco Guzm'an,Philipp Koehn,Alexandre Mourachko,Christophe Ropers,Safiyyah Saleem,Holger Schwenk,Jeff Wang +38 more
TL;DR: A conditional compute model based on Sparsely Gated Mixture of Experts that is trained on data obtained with novel and effective data mining techniques tailored for low-resource languages is developed, laying important groundwork towards realizing a universal translation system.