Journal ArticleDOI
Learning representations for weakly supervised natural language processing tasks
Reads0
Chats0
TLDR
Novel techniques for extracting features from n-gram models, Hidden Markov Models, and other statistical language models are investigated, including a novel Partial Lattice Markov Random Field model.Abstract:
Finding the right representations for words is critical for building accurate NLP systems when domain-specific labeled data for the task is scarce. This article investigates novel techniques for extracting features from n-gram models, Hidden Markov Models, and other statistical language models, including a novel Partial Lattice Markov Random Field model. Experiments on part-of-speech tagging and information extraction, among other tasks, indicate that features taken from statistical language models, in combination with more traditional features, outperform traditional representations alone, and that graphical model representations outperform n-gram models, especially on sparse and polysemous words.read more
Citations
More filters
Proceedings ArticleDOI
Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks
TL;DR: This paper proposes a novel model dubbed the Piecewise Convolutional Neural Networks (PCNNs) with multi-instance learning to address the problem of wrong label problem when using distant supervision for relation extraction and adopts convolutional architecture with piecewise max pooling to automatically learn relevant features.
Proceedings ArticleDOI
Tailoring Continuous Word Representations for Dependency Parsing
TL;DR: It is found that all embeddings yield significant parsing gains, including some recent ones that can be trained in a fraction of the time of others, suggesting their complementarity.
Journal ArticleDOI
A survey on the application of recurrent neural networks to statistical language modeling
TL;DR: This paper presents a survey on the application of recurrent neural networks to the task of statistical language modeling, and gives an overview of the most important extensions.
Proceedings ArticleDOI
Deep Multilingual Correlation for Improved Word Embeddings
TL;DR: Deep non-linear transformations of word embeddings of the two languages are learned, using the recently proposed deep canonical correlation analysis, to improve their quality and consistency on multiple word and bigram similarity tasks.
Proceedings ArticleDOI
Unsupervised Morphology Induction Using Word Embeddings
Radu Soricut,Franz Josef Och +1 more
TL;DR: A language agnostic, unsupervised method for inducing morphological transformations between words that relies on certain regularities manifest in highdimensional vector spaces and is capable of discovering a wide range of morphological rules.
References
More filters
Journal ArticleDOI
Maximum likelihood from incomplete data via the EM algorithm
Journal ArticleDOI
Latent dirichlet allocation
TL;DR: This work proposes a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams, and Hofmann's aspect model.
Proceedings Article
Latent Dirichlet Allocation
TL;DR: This paper proposed a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams, and Hof-mann's aspect model, also known as probabilistic latent semantic indexing (pLSI).
Journal ArticleDOI
A tutorial on hidden Markov models and selected applications in speech recognition
TL;DR: In this paper, the authors provide an overview of the basic theory of hidden Markov models (HMMs) as originated by L.E. Baum and T. Petrie (1966) and give practical details on methods of implementation of the theory along with a description of selected applications of HMMs to distinct problems in speech recognition.
Journal ArticleDOI
Indexing by Latent Semantic Analysis
TL;DR: A new method for automatic indexing and retrieval to take advantage of implicit higher-order structure in the association of terms with documents (“semantic structure”) in order to improve the detection of relevant documents on the basis of terms found in queries.