Open AccessPosted Content
Neural Models for Reasoning over Multiple Mentions using Coreference
TLDR
This article proposed a coreference annotations extracted from an external system to connect entity mentions belonging to the same cluster and incorporated this layer into a state-of-the-art reading comprehension model.Abstract:
Many problems in NLP require aggregating information from multiple mentions of the same entity which may be far apart in the text. Existing Recurrent Neural Network (RNN) layers are biased towards short-term dependencies and hence not suited to such tasks. We present a recurrent layer which is instead biased towards coreferent dependencies. The layer uses coreference annotations extracted from an external system to connect entity mentions belonging to the same cluster. Incorporating this layer into a state-of-the-art reading comprehension model improves performance on three datasets -- Wikihop, LAMBADA and the bAbi AI tasks -- with large gains when training data is scarce.read more
Citations
More filters
Proceedings Article
Differentiable Reasoning over a Virtual Knowledge Base
Bhuwan Dhingra,Manzil Zaheer,Vidhisha Balachandran,Graham Neubig,Ruslan Salakhutdinov,William W. Cohen +5 more
TL;DR: A neural module, DrKIT, that traverses textual data like a virtual KB, softly following paths of relations between mentions of entities in the corpus, which improves accuracy by 9 points on 3-hop questions in the MetaQA dataset and is very efficient.
Posted Content
Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)
TL;DR: A new English-language diagnostic set called MSGS (the Mixed Signals Generalization Set), which consists of 20 ambiguous binary classification tasks that are used to test whether a pretrained model prefers linguistic or surface generalizations during fine-tuning, finds that models can learn to represent linguistic features with little pretraining data, but require far more data to learn to prefer linguistic generalizations over surface ones.
Posted Content
Differentiable Reasoning over a Virtual Knowledge Base
Bhuwan Dhingra,Manzil Zaheer,Vidhisha Balachandran,Graham Neubig,Ruslan Salakhutdinov,William W. Cohen +5 more
TL;DR: The authors proposed a neural module, DrKIT, that traverses textual data like a virtual knowledge base, softly following paths of relations between mentions of entities in the corpus, using a combination of sparse-matrix TFIDF indices and a maximum inner product search (MIPS) on a special index of contextual representations of the mentions.
Journal ArticleDOI
What Artificial Neural Networks Can Tell Us About Human Language Acquisition
TL;DR: Before language learning requires more prior domain-specific knowledge than current models possess, non-linguistic inputs in the form of multimodal stimuli and multi-agent interaction are explored as ways to make learners morecient at learning from limited linguistic input.
Posted Content
A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks
TL;DR: This paper proposed a hierarchical model trained in a multi-task learning setup on a set of carefully selected semantic tasks, which achieves state-of-the-art results on a number of tasks.