Open AccessJournal Article
Natural Language Processing (Almost) from Scratch
Reads0
Chats0
TLDR
A unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including part-of-speech tagging, chunking, named entity recognition, and semantic role labeling is proposed.Abstract:
We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements.read more
Citations
More filters
Journal ArticleDOI
Using word embeddings in Twitter election classification
TL;DR: In this article, the effect of the configuration used to generate the word embeddings on the classification performance has not been studied in the existing literature, however, using a Twitter election classification task that aims to detect election-related tweets, the authors investigate the impact of the background dataset used to train the embedding models, as well as the parameters of the word embeddedding training process, namely the context window size, the dimensionality and the number of negative samples, on the attained classification performance.
Proceedings ArticleDOI
Disconnected Recurrent Neural Networks for Text Categorization
TL;DR: A novel model named disconnected recurrent neural network (DRNN), which incorporates position-invariance into RNN, which makes great improvements over RNN and CNN models and achieves the best performance on several benchmark datasets for text categorization.
Proceedings ArticleDOI
Part-of-Speech Tagging for Twitter with Adversarial Neural Networks
TL;DR: A novel neural network to make use of out-of-domain labeled data, unlabeled in- domain data, and labeled in-domain data is proposed to learn common features through adversarial discriminator for Tweets tagging.
Proceedings ArticleDOI
Entity Disambiguation by Knowledge and Text Jointly Embedding
TL;DR: This paper first learns low-dimensional continuous vector representations for entities and words by jointly embedding knowledge base and text in the same vector space and utilizes these embeddings to design simple but effective features and build a two-layer disambiguation model.
Journal ArticleDOI
Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function
TL;DR: This paper develops a training strategy that allows even a simple BiLSTM model, when trained with cross-entropy loss, to achieve competitive results compared with more complex approaches, and shows the generality of the mixed objective function by improving the performance on relation extraction task.
References
More filters
Journal ArticleDOI
Gradient-based learning applied to document recognition
Yann LeCun,Léon Bottou,Léon Bottou,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio,Patrick Haffner +6 more
TL;DR: In this article, a graph transformer network (GTN) is proposed for handwritten character recognition, which can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters.
Journal ArticleDOI
A tutorial on hidden Markov models and selected applications in speech recognition
TL;DR: In this paper, the authors provide an overview of the basic theory of hidden Markov models (HMMs) as originated by L.E. Baum and T. Petrie (1966) and give practical details on methods of implementation of the theory along with a description of selected applications of HMMs to distinct problems in speech recognition.
Book
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
TL;DR: Probabilistic Reasoning in Intelligent Systems as mentioned in this paper is a complete and accessible account of the theoretical foundations and computational methods that underlie plausible reasoning under uncertainty, and provides a coherent explication of probability as a language for reasoning with partial belief.
Journal ArticleDOI
A fast learning algorithm for deep belief nets
TL;DR: A fast, greedy algorithm is derived that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory.
Journal ArticleDOI
Machine learning
TL;DR: Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis.