We report on a series of experiments with convolutional neural networks (CNN) trained on top of pre-trained word vectors for sentence-level classification tasks. We show that a simple CNN with little hyperparameter tuning and static vectors achieves excellent results on multiple benchmarks. Learning task-specific vectors through fine-tuning offers further gains in performance. We additionally propose a simple modification to the architecture to allow for the use of both task-specific and static vectors. The CNN models discussed herein improve upon the state of the art on 4 out of 7 tasks, which include sentiment analysis and question classification.

/pdf/convolutional-neural-networks-for-sentence-classification-v2022hdzoh.pdf

Convolutional Neural Networks for Sentence Classification

The ability to accurately represent sentences is central to language understanding. We describe a convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) that we adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pooling, a global pooling operation over linear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily applicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment prediction, six-way question classification and Twitter sentiment prediction by distant supervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with respect to the strongest baseline.

/pdf/a-convolutional-neural-network-for-modelling-sentences-3vzv81a42n.pdf

A Convolutional Neural Network for Modelling Sentences

第12次 아시아ㆍ太平洋 矯政局長會議 參加記

Computer vision has benefited from initializing multiple deep layers with weights pretrained on large supervised training sets like ImageNet. Natural language processing (NLP) typically sees initialization of only the lowest layer of deep models with pretrained word vectors. In this paper, we use a deep LSTM encoder from an attentional sequence-to-sequence model trained for machine translation (MT) to contextualize word vectors. We show that adding these context vectors (CoVe) improves performance over using only unsupervised word and character vectors on a wide variety of common NLP tasks: sentiment analysis (SST, IMDb), question classification (TREC), entailment (SNLI), and question answering (SQuAD). For fine-grained sentiment analysis and entailment, CoVe improves performance of our baseline models to the state of the art.

/pdf/learned-in-translation-contextualized-word-vectors-243q3gzaw1.pdf

Learned in translation: contextualized word vectors

Question Answering (QA) is undoubtedly a growing field of current research in Artificial Intelligence. Question classification, a QA subtask, aims to associate a category to each question, typically representing the semantic class of its answer. This step is of major importance in the QA process, since it is the basis of several key decisions. For instance, classification helps reducing the number of possible answer candidates, as only answers matching the question category should be taken into account. This paper presents and evaluates a rule-based question classifier that partially founds its performance in the detection of the question headword and in its mapping into the target category through the use of WordNet. Moreover, we use the rule-based classifier as a features' provider of a machine learning-based question classifier. A detailed analysis of the rule-base contribution is presented. Despite using a very compact feature space, state of the art results are obtained.

From symbolic to sub-symbolic information in question classification

THE MENTOR automatically generates multiple-choice tests from a given text. This tool aims at 
supporting the dialogue system of the FalaComigo project, as one of FalaComigo’s goals is the 
interaction with tourists through questions/answers and quizzes about their visit. In a minimally 
supervised learning process and by leveraging the redundancy and linguistic variability of the Web, 
THE MENTOR learns lexico-syntactic patterns using a set of question/answer seeds. Afterwards, 
these patterns are used to match the sentences from which new questions (and answers) can be 
generated. Finally, several ﬁlters are applied in order to discard low quality items. In this paper we 
detail the question generation task as performed by T- Mand evaluate its performance.

Question Generation based on Lexico-Syntactic Patterns Learned from the Web

The answer determines the success of a Question-Answering (QA) system. In redundancy-based QA systems, a common approach is to extract the candidate answers from the information sources and select the most frequent answers as the final answers. However, this strategy has some pitfalls. For instance, if a system is not able to detect equivalences between the candidate answers, their frequencies might be erroneously calculated. Moreover, the user who posed the question should also be taken into account when answering: different persons require different (correct) answers. This can involve the use of suitable vocabulary and/or information details. In these situations, the generation of a response can be a more suitable strategy, instead of the extraction and direct retrieval of the answer from the information sources. The present survey targets the state of the art in the answering task in QA under three different lines of research. First, we present several works that focus on relating candidate answers. Then, we recover the concept of cooperative answer – a correct, useful, and non-misleading answer – and we bring up attempts to address cooperative answering. Finally, we investigate the research community endeavors on response generation. We will also present our perspective on each of these three topics throughout this paper.

https://www.cambridge.org/core/services/aop-cambridge-core/content/view/S1351324911000350

When the answer comes into question in question-answering: survey and open issues

When developing a conversational agent, there is often an urgent need to have a prototype available in order to test the application with real users. A Wizard of Oz is a possibility, but sometimes the agent should be simply deployed in the environment where it will be used. Here, the agent should be able to capture as many interactions as possible and to understand how people react to failure. In this paper, we focus on the rapid development of a natural language understanding module by non experts. Our approach follows the learning paradigm and sees the process of understanding natural language as a classification problem. We test our module with a conversational agent that answers questions in the art domain. Moreover, we show how our approach can be used by a natural language interface to a cinema database.

Towards the rapid development of a natural language understanding module

Linguistic patterns reflect the regularities of Natural Language and their applicability is acknowledged in several Natural Language Processing tasks Particularly, in the task of Question Generation, many systems depend on patterns to generate questions from text The approach we follow relies on patterns that convey lexical, syntactic and semantic information, automatically learned from large-scale corpora

In this paper we discuss the impact of varying several parameters during pattern learning and matching in the Question Generation task In particular, we introduce semantics (by means of named entities) in our lexico-syntactic patterns We evaluate and compare the number and quality of the learned patterns and the matched text segments Also, we detail the influence of the patterns in the generation of natural language questions

Ana Cristina Mendes

Papers

From symbolic to sub-symbolic information in question classification

Question Generation based on Lexico-Syntactic Patterns Learned from the Web

When the answer comes into question in question-answering: survey and open issues

Towards the rapid development of a natural language understanding module

Exploring linguistically-rich patterns for question generation