Open AccessProceedings Article
Adversarial Training Methods for Semi-Supervised Text Classification.
TLDR
In this article, the authors extend adversarial and virtual adversarial training to text domain by applying perturbations to the word embeddings in a recurrent neural network rather than to the original input itself.Abstract:
Adversarial training provides a means of regularizing supervised learning algorithms while virtual adversarial training is able to extend supervised learning algorithms to the semi-supervised setting. However, both methods require making small perturbations to numerous entries of the input vector, which is inappropriate for sparse high-dimensional inputs such as one-hot word representations. We extend adversarial and virtual adversarial training to the text domain by applying perturbations to the word embeddings in a recurrent neural network rather than to the original input itself. The proposed method achieves state of the art results on multiple benchmark semi-supervised and purely supervised tasks. We provide visualizations and analysis showing that the learned word embeddings have improved in quality and that while training, the model is less prone to overfitting.read more
Citations
More filters
Proceedings ArticleDOI
Universal Language Model Fine-tuning for Text Classification
Jeremy Howard,Sebastian Ruder +1 more
TL;DR: Universal Language Model Fine-tuning (ULMFiT) as mentioned in this paper is an effective transfer learning method that can be applied to any task in NLP, and introduces techniques that are key for finetuning a language model.
Proceedings Article
Learned in translation: contextualized word vectors
TL;DR: Adding context vectors to a deep LSTM encoder from an attentional sequence-to-sequence model trained for machine translation to contextualize word vectors improves performance over using only unsupervised word and character vectors on a wide variety of common NLP tasks.
Proceedings ArticleDOI
Adversarial Personalized Ranking for Recommendation
TL;DR: Adversarial Personalized Ranking (APR) as mentioned in this paper enhances the pairwise ranking method BPR by performing adversarial training, where the minimization of the BPR objective function meanwhile defends an adversary, which adds adversarial perturbations on model parameters to maximize the BPE objective function.
Posted Content
Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning
TL;DR: This work shows that a naive pseudo-labeling overfits to incorrect pseudo-labels due to the so-called confirmation bias and demonstrates that mixup augmentation and setting a minimum number of labeled samples per mini-batch are effective regularization techniques for reducing it.
Proceedings ArticleDOI
Adversarial Training for Relation Extraction
TL;DR: Experimental results demonstrate that adversarial training is generally effective for both CNN and RNN models and significantly improves the precision of predicted relations.