Semi-supervised Multitask Learning for Sequence Labeling
Marek Rei
- Vol. 1, pp 2121-2130
Reads0
Chats0
TLDR
The authors proposed a language modeling objective to incentivize the system to learn general-purpose patterns of semantic and syntactic composition, which are also useful for improving accuracy on different sequence labeling tasks.Abstract:
We propose a sequence labeling framework with a secondary training objective, learning to predict surrounding words for every word in the dataset. This language modeling objective incentivises the system to learn general-purpose patterns of semantic and syntactic composition, which are also useful for improving accuracy on different sequence labeling tasks. The architecture was evaluated on a range of datasets, covering the tasks of error detection in learner texts, named entity recognition, chunking and POS-tagging. The novel language modeling objective provided consistent performance improvements on every benchmark, without requiring any additional annotated or unannotated data.read more
Citations
More filters
Dissertation
Extracting Clinical Event Timelines : Temporal Information Extraction and Coreference Resolution in Electronic Health Records
TL;DR: A neural based approach for temporal information extraction which includes categorical features and neural network components such as attention mechanisms and token character-level representations influence the performance of the coreference resolution approach in clinical narratives.
Posted Content
Neural Multi-task Learning in Automated Assessment
Ronan Cummins,Marek Rei +1 more
TL;DR: A multi-task neural network model is developed that jointly optimises for both grammatical error detection and essay scoring and shows that neural automated essay scoring can be significantly improved.
Posted Content
Multi-Task Learning for Domain-General Spoken Disfluency Detection in Dialogue Systems
TL;DR: This paper presents a multi-task LSTM-based model for incremental detection of disfluency structure, which can be hooked up to any component for incremental interpretation, or else simply used to `clean up' the current utterance as it is being produced.
Journal ArticleDOI
Dual Learning for Semi-Supervised Natural Language Understanding
Su Zhu,Ruisheng Cao,Kai Yu +2 more
TL;DR: Zhang et al. as mentioned in this paper introduced a dual task of NLU, semantic-to-sentence generation (SSG), and proposed a new framework for semi-supervised NLU with the corresponding dual model.
A Multi-Task Approach to Incremental Dialogue State Tracking
TL;DR: This paper presents the design of the incremental dialogue state tracker in detail and provides evaluation against the well known Dialogue State Tracking Challenge 2 (DSTC2) dataset and finds that the Multi-Task Learning-based model achieves state-of-the-art results for incremental processing.
References
More filters
Journal ArticleDOI
Long short-term memory
TL;DR: A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
Journal Article
Dropout: a simple way to prevent neural networks from overfitting
TL;DR: It is shown that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
Posted Content
Efficient Estimation of Word Representations in Vector Space
TL;DR: This paper proposed two novel model architectures for computing continuous vector representations of words from very large data sets, and the quality of these representations is measured in a word similarity task and the results are compared to the previously best performing techniques based on different types of neural networks.
Proceedings Article
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
TL;DR: This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.