Showing papers on "Natural language published in 2015"

A large annotated corpus for learning natural language inference

Abstract: Understanding entailment and contradiction is fundamental to understanding natural language, and inference about entailment and contradiction is a valuable testing ground for the development of semantic representations. However, machine learning research in this area has been dramatically limited by the lack of large-scale resources. To address this, we introduce the Stanford Natural Language Inference corpus, a new, freely available collection of labeled sentence pairs, written by humans doing a novel grounded task based on image captioning. At 570K pairs, it is two orders of magnitude larger than all other resources of its type. This increase in scale allows lexicalized classifiers to outperform some sophisticated existing entailment models, and it allows a neural network-based model to perform competitively on natural language inference benchmarks for the first time.

Teaching machines to read and comprehend

Abstract: Teaching machines to read natural language documents remains an elusive challenge. Machine reading systems can be tested on their ability to answer questions posed on the contents of documents that they have seen, but until now large scale training and test datasets have been missing for this type of evaluation. In this work we define a new methodology that resolves this bottleneck and provides large scale supervised reading comprehension data. This allows us to develop a class of attention based deep neural networks that learn to read real documents and answer complex questions with minimal prior knowledge of language structure.

A Critical Review of Recurrent Neural Networks for Sequence Learning

Abstract: Countless learning tasks require dealing with sequential data. Image captioning, speech synthesis, and music generation all require that a model produce outputs that are sequences. In other domains, such as time series prediction, video analysis, and musical information retrieval, a model must learn from inputs that are sequences. Interactive tasks, such as translating natural language, engaging in dialogue, and controlling a robot, often demand both capabilities. Recurrent neural networks (RNNs) are connectionist models that capture the dynamics of sequences via cycles in the network of nodes. Unlike standard feedforward neural networks, recurrent networks retain a state that can represent information from an arbitrarily long context window. Although recurrent neural networks have traditionally been dicult to train, and often contain millions of parameters, recent advances in network architectures, optimization techniques, and parallel computation have enabled successful large-scale learning with them. In recent years, systems based on long short-term memory (LSTM) and bidirectional (BRNN) architectures have demonstrated ground-breaking performance on tasks as varied as image captioning, language translation, and handwriting recognition. In this survey, we review and synthesize the research that over the past three decades rst yielded and then made practical these powerful learning models. When appropriate, we reconcile conicting notation and nomenclature. Our goal is to provide a selfcontained explication of the state of the art together with a historical perspective and references to primary research.

A large annotated corpus for learning natural language inference

Abstract: Understanding entailment and contradiction is fundamental to understanding natural language, and inference about entailment and contradiction is a valuable testing ground for the development of semantic representations. However, machine learning research in this area has been dramatically limited by the lack of large-scale resources. To address this, we introduce the Stanford Natural Language Inference corpus, a new, freely available collection of labeled sentence pairs, written by humans doing a novel grounded task based on image captioning. At 570K pairs, it is two orders of magnitude larger than all other resources of its type. This increase in scale allows lexicalized classifiers to outperform some sophisticated existing entailment models, and it allows a neural network-based model to perform competitively on natural language inference benchmarks for the first time.

Convolutional Neural Network Architectures for Matching Natural Language Sentences

Abstract: Semantic matching is of central importance to many natural language tasks \cite{bordes2014semantic,RetrievalQA}. A successful matching algorithm needs to adequately model the internal structures of language objects and the interaction between them. As a step toward this goal, we propose convolutional neural network models for matching two sentences, by adapting the convolutional strategy in vision and speech. The proposed models not only nicely represent the hierarchical structures of sentences with their layer-by-layer composition and pooling, but also capture the rich matching patterns at different levels. Our models are rather generic, requiring no prior knowledge on language, and can hence be applied to matching tasks of different nature and in different languages. The empirical study on a variety of matching tasks demonstrates the efficacy of the proposed model on a variety of matching tasks and its superiority to competitor models.

Translating Videos to Natural Language Using Deep Recurrent Neural Networks

Abstract: Solving the visual symbol grounding problem has long been a goal of artificial intelligence. The field appears to be advancing closer to this goal with recent breakthroughs in deep learning for natural language grounding in static images. In this paper, we propose to translate videos directly to sentences using a unified deep neural network with both convolutional and recurrent structure. Described video datasets are scarce, and most existing methods have been applied to toy domains with a small vocabulary of possible words. By transferring knowledge from 1.2M+ images with category labels and 100,000+ images with captions, our method is able to create sentence descriptions of open-domain videos with large vocabularies. We compare our approach with recent work using language generation metrics, subject, verb, and object prediction accuracy, and a human evaluation.

Teaching Machines to Read and Comprehend

Abstract: Teaching machines to read natural language documents remains an elusive challenge. Machine reading systems can be tested on their ability to answer questions posed on the contents of documents that they have seen, but until now large scale training and test datasets have been missing for this type of evaluation. In this work we define a new methodology that resolves this bottleneck and provides large scale supervised reading comprehension data. This allows us to develop a class of attention based deep neural networks that learn to read real documents and answer complex questions with minimal prior knowledge of language structure.

Ask Your Neurons: A Neural-Based Approach to Answering Questions about Images

Abstract: We address a question answering task on real-world images that is set up as a Visual Turing Test. By combining latest advances in image representation and natural language processing, we propose Neural-Image-QA, an end-to-end formulation to this problem for which all parts are trained jointly. In contrast to previous efforts, we are facing a multi-modal problem where the language output (answer) is conditioned on visual and natural language input (image and question). Our approach Neural-Image-QA doubles the performance of the previous best approach on this problem. We provide additional insights into the problem by analyzing how much information is contained only in the language part for which we provide a new human baseline. To study human consensus, which is related to the ambiguities inherent in this challenging task, we propose two novel metrics and collect additional answers which extends the original DAQUAR dataset to DAQUAR-Consensus.

Ask Your Neurons: A Neural-based Approach to Answering Questions about Images

Abstract: We address a question answering task on real-world images that is set up as a Visual Turing Test. By combining latest advances in image representation and natural language processing, we propose Neural-Image-QA, an end-to-end formulation to this problem for which all parts are trained jointly. In contrast to previous efforts, we are facing a multi-modal problem where the language output (answer) is conditioned on visual and natural language input (image and question). Our approach Neural-Image-QA doubles the performance of the previous best approach on this problem. We provide additional insights into the problem by analyzing how much information is contained only in the language part for which we provide a new human baseline. To study human consensus, which is related to the ambiguities inherent in this challenging task, we propose two novel metrics and collect additional answers which extends the original DAQUAR dataset to DAQUAR-Consensus.

Jointly Modeling Embedding and Translation to Bridge Video and Language

Abstract: Automatically describing video content with natural language is a fundamental challenge of multimedia. Recurrent Neural Networks (RNN), which models sequence dynamics, has attracted increasing attention on visual interpretation. However, most existing approaches generate a word locally with given previous words and the visual content, while the relationship between sentence semantics and visual content is not holistically exploited. As a result, the generated sentences may be contextually correct but the semantics (e.g., subjects, verbs or objects) are not true. This paper presents a novel unified framework, named Long Short-Term Memory with visual-semantic Embedding (LSTM-E), which can simultaneously explore the learning of LSTM and visual-semantic embedding. The former aims to locally maximize the probability of generating the next word given previous words and visual content, while the latter is to create a visual-semantic embedding space for enforcing the relationship between the semantics of the entire sentence and visual content. Our proposed LSTM-E consists of three components: a 2-D and/or 3-D deep convolutional neural networks for learning powerful video representation, a deep RNN for generating sentences, and a joint embedding model for exploring the relationships between visual content and sentence semantics. The experiments on YouTube2Text dataset show that our proposed LSTM-E achieves to-date the best reported performance in generating natural sentences: 45.3% and 31.0% in terms of BLEU@4 and METEOR, respectively. We also demonstrate that LSTM-E is superior in predicting Subject-Verb-Object (SVO) triplets to several state-of-the-art techniques.

To block or not to block, that is the question: students' perceptions of blocks-based programming

Abstract: Blocks-based programming tools are becoming increasingly common in high-school introductory computer science classes. Such contexts are quite different than the younger audience and informal settings where these tools are more often used. This paper reports findings from a study looking at how high school students view blocks-based programming tools, what they identify as contributing to the perceived ease-of-use of such tools, and what they see as the most salient differences between blocks-based and text-based programming. Students report that numerous factors contribute to making blocks-based programming easy, including the natural language description of blocks, the drag-and-drop composition interaction, and the ease of browsing the language. Students also identify drawbacks to blocks-based programming compared to the conventional text-based approach, including a perceived lack of authenticity and being less powerful. These findings, along with the identified differences between blocks-based and text-based programming, contribute to our understanding of the suitability of using such tools in formal high school settings and can be used to inform the design of new, and revision of existing, introductory programming tools.

Polylanguaging in Superdiversity

Abstract: Humankind is a languaging species. This means that as human beings we use language to achieve our goals. Every time we use language, we change the world a little bit. We do so by using language with other human beings, language is in other words social. In this paper we challenge one of the most widely held views of language as a social, human phenomenon, namely that “language” can be separated into different “languages”, such as “Russian”, “Latin”, and “Greenlandic”. Our paper is based on a recently developed sociolinguistic understanding that this view of language can not be upheld on the basis of linguistic criteria. “Languages” are abstractions, they are sociocultural or ideological constructions which match real-life use of language poorly. This means that sociolinguistics – the study of language as a social phenomenon - must work at another level of analysis with real-life language use. The first part of our paper presents such analyses of observed language use among adolescents in superdiverse societies. We show that the level of a linguistic feature is better suited as the basis for analysis of language use than the level of “a language”. In the second part of the paper we present our concept of polylanguaging which denotes the way in which speakers use features associated with different “languages” – even when they know very little of these “languages”. We use the level of (linguistic) features as the basis for understanding language use, and we claim that features are socioculturally associated with “languages”. Both features individually and languages are socioculturally associated with values, meanings, speakers, etc. This means that we can deal with the connection between features and languages, and in the analyses in the first part we do exactly that.

Learning to Generate Pseudo-Code from Source Code Using Statistical Machine Translation (T)

Abstract: Pseudo-code written in natural language can aid the comprehension of source code in unfamiliar programming languages. However, the great majority of source code has no corresponding pseudo-code, because pseudo-code is redundant and laborious to create. If pseudo-code could be generated automatically and instantly from given source code, we could allow for on-demand production of pseudo-code without human effort. In this paper, we propose a method to automatically generate pseudo-code from source code, specifically adopting the statistical machine translation (SMT) framework. SMT, which was originally designed to translate between two natural languages, allows us to automatically learn the relationship between source code/pseudo-code pairs, making it possible to create a pseudo-code generator with less human effort. In experiments, we generated English or Japanese pseudo-code from Python statements using SMT, and find that the generated pseudo-code is largely accurate, and aids code understanding.

Why Only Us: Language and Evolution

Abstract: We are born crying, but those cries signal the first stirring of language. Within a year or so, infants master the sound system of their language; a few years after that, they are engaging in conversations. This remarkable, species-specific ability to acquire any human language -- "the language faculty" -- raises important biological questions about language, including how it has evolved. This book by two distinguished scholars -- a computer scientist and a linguist -- addresses the enduring question of the evolution of language. Robert Berwick and Noam Chomsky explain that until recently the evolutionary question could not be properly posed, because we did not have a clear idea of how to define "language" and therefore what it was that had evolved. But since the Minimalist Program, developed by Chomsky and others, we know the key ingredients of language and can put together an account of the evolution of human language and what distinguishes us from all other animals. Berwick and Chomsky discuss the biolinguistic perspective on language, which views language as a particular object of the biological world; the computational efficiency of language as a system of thought and understanding; the tension between Darwin's idea of gradual change and our contemporary understanding about evolutionary change and language; and evidence from nonhuman animals, in particular vocal learning in songbirds.

Question-Answer Driven Semantic Role Labeling: Using Natural Language to Annotate Natural Language

Abstract: This paper introduces the task of questionanswer driven semantic role labeling (QA-SRL), where question-answer pairs are used to represent predicate-argument structure. For example, the verb “introduce” in the previous sentence would be labeled with the questions “What is introduced?”, and “What introduces something?”, each paired with the phrase from the sentence that gives the correct answer. Posing the problem this way allows the questions themselves to define the set of possible roles, without the need for predefined frame or thematic role ontologies. It also allows for scalable data collection by annotators with very little training and no linguistic expertise. We gather data in two domains, newswire text and Wikipedia articles, and introduce simple classifierbased models for predicting which questions to ask and what their answers should be. Our results show that non-expert annotators can produce high quality QA-SRL data, and also establish baseline performance levels for future work on this task.

Bimodal Modelling of Source Code and Natural Language

Abstract: We consider the problem of building probabilistic models that jointly model short natural language utterances and source code snippets. The aim is to bring together recent work on statistical modelling of source code and work on bimodal models of images and natural language. The resulting models are useful for a variety of tasks that involve natural language and source code. We demonstrate their performance on two retrieval tasks: retrieving source code snippets given a natural language query, and retrieving natural language descriptions given a source code query (i.e., source code captioning). Experiments show there to be promise in this direction, and that modelling the structure of source code improves performance.

DataTone: Managing Ambiguity in Natural Language Interfaces for Data Visualization

Abstract: Answering questions with data is a difficult and time-consuming process. Visual dashboards and templates make it easy to get started, but asking more sophisticated questions often requires learning a tool designed for expert analysts. Natural language interaction allows users to ask questions directly in complex programs without having to learn how to use an interface. However, natural language is often ambiguous. In this work we propose a mixed-initiative approach to managing ambiguity in natural language interfaces for data visualization. We model ambiguity throughout the process of turning a natural language query into a visualization and use algorithmic disambiguation coupled with interactive ambiguity widgets. These widgets allow the user to resolve ambiguities by surfacing system decisions at the point where the ambiguity matters. Corrections are stored as constraints and influence subsequent queries. We have implemented these ideas in a system, DataTone. In a comparative study, we find that DataTone is easy to learn and lets users ask questions without worrying about syntax and proper question form.

Universal Principles in the Repair of Communication Problems.

Abstract: There would be little adaptive value in a complex communication system like human language if there were no ways to detect and correct problems. A systematic comparison of conversation in a broad sample of the world’s languages reveals a universal system for the real-time resolution of frequent breakdowns in communication. In a sample of 12 languages of 8 language families of varied typological profiles we find a system of ‘other-initiated repair’, where the recipient of an unclear message can signal trouble and the sender can repair the original message. We find that this system is frequently used (on average about once per 1.4 minutes in any language), and that it has detailed common properties, contrary to assumptions of radical cultural variation. Unrelated languages share the same three functionally distinct types of repair initiator for signalling problems and use them in the same kinds of contexts. People prefer to choose the type that is the most specific possible, a principle that minimizes cost both for the sender being asked to fix the problem and for the dyad as a social unit. Disruption to the conversation is kept to a minimum, with the two-utterance repair sequence being on average no longer that the single utterance which is being fixed. The findings, controlled for historical relationships, situation types and other dependencies, reveal the fundamentally cooperative nature of human communication and offer support for the pragmatic universals hypothesis: while languages may vary in the organization of grammar and meaning, key systems of language use may be largely similar across cultural groups. They also provide a fresh perspective on controversies about the core properties of language, by revealing a common infrastructure for social interaction which may be the universal bedrock upon which linguistic diversity rests.

Semantics-Aware Content-Based Recommender Systems.

Abstract: Content-based recommender systems (CBRSs) rely on item and user descriptions (content) to build item representations and user profiles that can be effectively exploited to suggest items similar to those a target user already liked in the past. Most content-based recommender systems use textual features to represent items and user profiles, hence they suffer from the classical problems of natural language ambiguity. This chapter presents a comprehensive survey of semantic representations of items and user profiles that attempt to overcome the main problems of the simpler approaches based on keywords. We propose a classification of semantic approaches into top-down and bottom-up. The former rely on the integration of external knowledge sources, such as ontologies, encyclopedic knowledge and data from the Linked Data cloud, while the latter rely on a lightweight semantic representation based on the hypothesis that the meaning of words depends on their use in large corpora of textual documents. The chapter shows how to make recommender systems aware of semantics to realize a new generation of content-based recommenders.

Man-machine interaction method and system based on artificial intelligence

Abstract: A man-machine interaction method and system based on artificial intelligence. The man-machine interaction method based on artificial intelligence comprises the following steps: receiving input information input by a user through an application terminal (S1); acquiring intent information about the user according to the input information about the user, and distributing the input information to at least one interaction service subsystem according to the intent information (S2); receiving a return result returned by the at least one interaction service subsystem (S3); and generating a user return result according to the return result in a pre-set decision strategy, and providing the user return result to the user (S4). In the method and system, the man-machine interaction system is anthropomorphic instead of being instrumentalized, and the user can obtain relaxed and pleased interaction experience in the intelligent interaction process through chat, research and other service. The search in the form of keywords is improved into natural language based search, the user can express demands through flexible and free natural languages, and a multi-round interaction process is closer to the interaction experience among humans.

Language to Code: Learning Semantic Parsers for If-This-Then-That Recipes

Abstract: Using natural language to write programs is a touchstone problem for computational linguistics. We present an approach that learns to map natural-language descriptions of simple “if-then” rules to executable code. By training and testing on a large corpus of naturally-occurring programs (called “recipes”) and their natural language descriptions, we demonstrate the ability to effectively map language to code. We compare a number of semantic parsing approaches on the highly noisy training data collected from ordinary users, and find that loosely synchronous systems perform best.

The Utility and Application of Mixed-Effects Models in Second Language Research

Abstract: Second language acquisition researchers often face particular challenges when attempting to generalize study findings to the wider learner population. For example, language learners constitute a heterogeneous group, and it is not always clear how a study’s findings may generalize to other individuals who may differ in terms of language background and proficiency, among many other factors. In this paper, we provide an overview of how mixed-effects models can be used to help overcome these and other issues in the field of second language acquisition. We provide an overview of the benefits of mixed-effects models and a practical example of how mixed-effects analyses can be conducted. Mixed-effects models provide second language researchers with a powerful statistical tool in the analysis of a variety of different types of data.

Learning to interpret natural language commands through human-robot dialog

Abstract: Intelligent robots frequently need to understand requests from naive users through natural language. Previous approaches either cannot account for language variation, e.g., keyword search, or require gathering large annotated corpora, which can be expensive and cannot adapt to new variation. We introduce a dialog agent for mobile robots that understands human instructions through semantic parsing, actively resolves ambiguities using a dialog manager, and incrementally learns from human-robot conversations by inducing training data from user paraphrases. Our dialog agent is implemented and tested both on a web interface with hundreds of users via Mechanical Turk and on a mobile robot over several days, tasked with understanding navigation and delivery requests through natural language in an office environment. In both contexts, We observe significant improvements in user satisfaction after learning from conversations.

Visual Madlibs: Fill in the Blank Description Generation and Question Answering

Abstract: In this paper, we introduce a new dataset consisting of 360,001 focused natural language descriptions for 10,738 images. This dataset, the Visual Madlibs dataset, is collected using automatically produced fill-in-the-blank templates designed to gather targeted descriptions about: people and objects, their appearances, activities, and interactions, as well as inferences about the general scene or its broader context. We provide several analyses of the Visual Madlibs dataset and demonstrate its applicability to two new description generation tasks: focused description generation, and multiple-choice question-answering for images. Experiments using joint-embedding and deep learning methods show promising results on these tasks.

Semantics-Aware Content-Based Recommender Systems

Abstract: Content-based recommender systems (CBRSs) rely on item and user descriptions (content) to build item representations and user profiles that can be effectively exploited to suggest items similar to those a target user already liked in the past. Most content-based recommender systems use textual features to represent items and user profiles, hence they suffer from the classical problems of natural language ambiguity. This chapter presents a comprehensive survey of semantic representations of items and user profiles that attempt to overcome the main problems of the simpler approaches based on keywords. We propose a classification of semantic approaches into top-down and bottom-up. The former rely on the integration of external knowledge sources, such as ontologies, encyclopedic knowledge and data from the Linked Data cloud, while the latter rely on a lightweight semantic representation based on the hypothesis that the meaning of words depends on their use in large corpora of textual documents. The chapter shows how to make recommender systems aware of semantics to realize a new generation of content-based recommenders.

Automatically Solving Number Word Problems by Semantic Parsing and Reasoning

Abstract: This paper presents a semantic parsing and reasoning approach to automatically solving math word problems. A new meaning representation language is designed to bridge natural language text and math expressions. A CFG parser is implemented based on 9,600 semi-automatically created grammar rules. We conduct experiments on a test set of over 1,500 number word problems (i.e., verbally expressed number problems) and yield 95.4% precision and 60.2% recall.

Eye movements in code reading: relaxing the linear order

Abstract: Code reading is an important skill in programming. Inspired by the linearity that people exhibit while natural language text reading, we designed local and global gaze-based measures to characterize linearity (left-to-right and top-to-bottom) in reading source code. Unlike natural language text, source code is executable and requires a specific reading approach. To validate these measures, we compared the eye movements of novice and expert programmers who were asked to read and comprehend short snippets of natural language text and Java programs. Our results show that novices read source code less linearly than natural language text. Moreover, experts read code less linearly than novices. These findings indicate that there are specific differences between reading natural language and source code, and suggest that non-linear reading skills increase with expertise. We discuss the implications for practitioners and educators.

The CoNLL-2015 Shared Task on Shallow Discourse Parsing

Abstract: The CoNLL-2015 Shared Task is on Shallow Discourse Parsing, a task focusing on identifying individual discourse relations that are present in a natural language text. A discourse relation can be expressed explicitly or implicitly, and takes two arguments realized as sentences, clauses, or in some rare cases, phrases. Sixteen teams from three continents participated in this task. For the first time in the history of the CoNLL shared tasks, participating teams, instead of running their systems on the test set and submitting the output, were asked to deploy their systems on a remote virtual machine and use a web-based evaluation platform to run their systems on the test set. This meant they were unable to actually see the data set, thus preserving its integrity and ensuring its replicability. In this paper, we present the task definition, the training and test sets, and the evaluation protocol and metric used during this shared task. We also summarize the different approaches adopted by the participating teams, and present the evaluation results. The evaluation data sets and the scorer will serve as a benchmark for future research on shallow discourse parsing.