S
Sebastian Ruder
Researcher at Google
Publications - 105
Citations - 16026
Sebastian Ruder is an academic researcher from Google. The author has contributed to research in topics: Language model & Computer science. The author has an hindex of 37, co-authored 104 publications receiving 11465 citations. Previous affiliations of Sebastian Ruder include National University of Ireland & Allen Institute for Artificial Intelligence.
Papers
More filters
Posted Content
An overview of gradient descent optimization algorithms
TL;DR: This article looks at different variants of gradient descent, summarize challenges, introduce the most common optimization algorithms, review architectures in a parallel and distributed setting, and investigate additional strategies for optimizing gradient descent.
Posted Content
An Overview of Multi-Task Learning in Deep Neural Networks
TL;DR: This article seeks to help ML practitioners apply MTL by shedding light on how MTL works and providing guidelines for choosing appropriate auxiliary tasks, particularly in deep neural networks.
Proceedings ArticleDOI
Universal Language Model Fine-tuning for Text Classification
Jeremy Howard,Sebastian Ruder +1 more
TL;DR: Universal Language Model Fine-tuning (ULMFiT) as mentioned in this paper is an effective transfer learning method that can be applied to any task in NLP, and introduces techniques that are key for finetuning a language model.
Proceedings ArticleDOI
On the Cross-lingual Transferability of Monolingual Representations
TL;DR: This work designs an alternative approach that transfers a monolingual model to new languages at the lexical level and shows that it is competitive with multilingual BERT on standard cross-lingUAL classification benchmarks and on a new Cross-lingual Question Answering Dataset (XQuAD).
Posted Content
XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization
TL;DR: The Cross-lingual TRansfer Evaluation of Multilingual Encoders XTREME benchmark is introduced, a multi-task benchmark for evaluating the cross-lingually generalization capabilities of multilingual representations across 40 languages and 9 tasks.