Open AccessProceedings Article
Attention is All you Need
Ashish Vaswani,Noam Shazeer,Niki Parmar,Jakob Uszkoreit,Llion Jones,Aidan N. Gomez,Lukasz Kaiser,Illia Polosukhin +7 more
- Vol. 30, pp 5998-6008
Reads0
Chats0
TLDR
This paper proposed a simple network architecture based solely on an attention mechanism, dispensing with recurrence and convolutions entirely and achieved state-of-the-art performance on English-to-French translation.Abstract:
The dominant sequence transduction models are based on complex recurrent orconvolutional neural networks in an encoder and decoder configuration. The best performing such models also connect the encoder and decoder through an attentionm echanisms. We propose a novel, simple network architecture based solely onan attention mechanism, dispensing with recurrence and convolutions entirely.Experiments on two machine translation tasks show these models to be superiorin quality while being more parallelizable and requiring significantly less timeto train. Our single model with 165 million parameters, achieves 27.5 BLEU onEnglish-to-German translation, improving over the existing best ensemble result by over 1 BLEU. On English-to-French translation, we outperform the previoussingle state-of-the-art with model by 0.7 BLEU, achieving a BLEU score of 41.1.read more
Citations
More filters
Proceedings ArticleDOI
YiSi - a Unified Semantic MT Quality Evaluation and Estimation Metric for Languages with Different Levels of Available Resources
TL;DR: A significant improvement in the correlation of YiSi-1’s scores with human judgment is made by using contextual embeddings in multilingual BERT–Bidirectional Encoder Representations from Transformers to evaluate lexical semantic similarity.
Proceedings ArticleDOI
Attend and Interact: Higher-Order Object Interactions for Video Understanding
TL;DR: In this paper, higher-order interactions between arbitrary subgroups of objects are learned for fine-grained video understanding, which significantly improves accuracy for both action recognition and video captioning, while saving more than 3 times the computation over traditional pairwise relationships.
Posted Content
A Self-Attentive model for Knowledge Tracing
Shalini Pandey,George Karypis +1 more
TL;DR: This work develops an approach that identifies the KCs from the student's past activities that are relevant to the given KC and predicts his/her mastery based on the relatively few KCs that it picked, and handles the data sparsity problem better than the methods based on RNN.
Proceedings ArticleDOI
Classification and Clustering of Arguments with Contextualized Word Embeddings
Nils Reimers,Benjamin Schiller,Tilman Beck,Johannes Daxenberger,Christian Stab,Iryna Gurevych +5 more
TL;DR: For the first time, it is shown how to leverage the power of contextualized word embeddings to classify and cluster topic-dependent arguments, achieving impressive results on both tasks and across multiple datasets.
Proceedings ArticleDOI
MLCVNet: Multi-Level Context VoteNet for 3D Object Detection
TL;DR: MLCVNet as mentioned in this paper introduces three context modules into the voting and classifying stages of VoteNet to encode contextual information at different levels, namely, patch-to-patch Context (PPC), object-toobject Context (OOC), and global scene context (GSC).