Open AccessPosted Content
Investigating Numeracy Learning Ability of a Text-to-Text Transfer Model
Kuntal Kumar Pal,Chitta Baral +1 more
Reads0
Chats0
TLDR
This article investigated the ability of text-to-text transfer learning model (T5) to learn numeracy, and found that T5 models perform reasonably well in the interpolation setting, but they struggle considerably in the extrapolation setting across all four numeracy tasks.Abstract:
The transformer-based pre-trained language models have been tremendously successful in most of the conventional NLP tasks. But they often struggle in those tasks where numerical understanding is required. Some possible reasons can be the tokenizers and pre-training objectives which are not specifically designed to learn and preserve numeracy. Here we investigate the ability of text-to-text transfer learning model (T5), which has outperformed its predecessors in the conventional NLP tasks, to learn numeracy. We consider four numeracy tasks: numeration, magnitude order prediction, finding minimum and maximum in a series, and sorting. We find that, although T5 models perform reasonably well in the interpolation setting, they struggle considerably in the extrapolation setting across all four tasks.read more
References
More filters
Proceedings ArticleDOI
SQuAD: 100,000+ Questions for Machine Comprehension of Text
TL;DR: The Stanford Question Answering Dataset (SQuAD) as mentioned in this paper is a reading comprehension dataset consisting of 100,000+ questions posed by crowdworkers on a set of Wikipedia articles, where the answer to each question is a segment of text from the corresponding reading passage.
Journal Article
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Colin Raffel,Noam Shazeer,Adam Roberts,Katherine Lee,Sharan Narang,Michael Matena,Yanqi Zhou,Wei Li,Peter J. Liu +8 more
TL;DR: This article introduced a unified framework that converts all text-based language problems into a text-to-text format and compared pre-training objectives, architectures, unlabeled data sets, transfer approaches, and other factors on dozens of language understanding tasks.
Posted Content
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
TL;DR: This article introduced a new English reading comprehension benchmark, DROP, which requires Discrete Reasoning Over the content of Paragraphs, and applied state-of-the-art methods from both the reading comprehension and semantic parsing literature on this dataset and show that the best systems only achieved 32.7% F1 on their generalized accuracy metric, while expert human performance is 96.0%.
Proceedings ArticleDOI
Injecting Numerical Reasoning Skills into Language Models
TL;DR: This work shows that numerical reasoning is amenable to automatic data generation, and thus one can inject this skill into pre-trained LMs, by generating large amounts of data, and training in a multi-task setup.
Proceedings ArticleDOI
Do NLP Models Know Numbers? Probing Numeracy in Embeddings
TL;DR: This paper investigated the numerical reasoning capabilities of a state-of-the-art question answering model on the DROP dataset and found that this model excels on questions that require numerical reasoning, i.e., it already captures numeracy.