Showing papers on "Natural language understanding published in 2014"

Jumping NLP Curves: A Review of Natural Language Processing Research [Review Article]

Abstract: Natural language processing (NLP) is a theory-motivated range of computational techniques for the automatic analysis and representation of human language. NLP research has evolved from the era of punch cards and batch processing (in which the analysis of a sentence could take up to 7 minutes) to the era of Google and the likes of it (in which millions of webpages can be processed in less than a second). This review paper draws on recent developments in NLP research to look at the past, present, and future of NLP technology in a new light. Borrowing the paradigm of `jumping curves? from the field of business management and marketing prediction, this survey article reinterprets the evolution of NLP research as the intersection of three overlapping curves-namely Syntactics, Semantics, and Pragmatics Curves- which will eventually lead NLP research to evolve into natural language understanding.

EmoSenticSpace: a novel framework for affective common-sense reasoning

Abstract: Emotions play a key role in natural language understanding and sensemaking. Pure machine learning usually fails to recognize and interpret emotions in text accurately. The need for knowledge bases that give access to semantics and sentics (the conceptual and affective information) associated with natural language is growing exponentially in the context of big social data analysis. To this end, this paper proposes EmoSenticSpace, a new framework for affective common-sense reasoning that extends WordNet-Affect and SenticNet by providing both emotion labels and polarity scores for a large set of natural language concepts. The framework is built by means of fuzzy c-means clustering and support-vector-machine classification, and takes into account a number of similarity measures, including point-wise mutual information and emotional affinity. EmoSenticSpace was tested on three emotion-related natural language processing tasks, namely sentiment analysis, emotion recognition, and personality detection. In all cases, the proposed framework outperforms the state-of-the-art. In particular, the direct evaluation of EmoSenticSpace against psychological features provided in the benchmark ISEAR dataset shows a 92.15% agreement.

A natural language planner interface for mobile manipulators

Abstract: Natural language interfaces for robot control aspire to find the best sequence of actions that reflect the behavior intended by the instruction. This is difficult because of the diversity of language, variety of environments, and heterogeneity of tasks. Previous work has demonstrated that probabilistic graphical models constructed from the parse structure of natural language can be used to identify motions that most closely resemble verb phrases. Such approaches however quickly succumb to computational bottlenecks imposed by construction and search the space of possible actions. Planning constraints, which define goal regions and separate the admissible and inadmissible states in an environment model, provide an interesting alternative to represent the meaning of verb phrases. In this paper we present a new model called the Distributed Correspondence Graph (DCG) to infer the most likely set of planning constraints from natural language instructions. A trajectory planner then uses these planning constraints to find a sequence of actions that resemble the instruction. Separating the problem of identifying the action encoded by the language into individual steps of planning constraint inference and motion planning enables us to avoid computational costs associated with generation and evaluation of many trajectories. We present experimental results from comparative experiments that demonstrate improvements in efficiency in natural language understanding without loss of accuracy.

Plan, Activity, and Intent Recognition: Theory and Practice

Abstract: Plan recognition, activity recognition, and intent recognition together combine and unify techniques from user modeling, machine vision, intelligent user interfaces, human/computer interaction, autonomous and multi-agent systems, natural language understanding, and machine learning. Plan, Activity, and Intent Recognition explains the crucial role of these techniques in a wide variety of applications including: personal agent assistants, computer and network security, opponent modeling in games and simulation systems, coordination in robots and software agents, web e-commerce and collaborative filtering, dialog modeling, video surveillance, smart homes In this book, follow the history of this research area and witness exciting new developments in the field made possible by improved sensors, increased computational power, and new application areas. Combines basic theory on algorithms for plan/activity recognition along with results from recent workshops and seminars Explains how to interpret and recognize plans and activities from sensor data Provides valuable background knowledge and assembles key concepts into one guide for researchers or students studying these disciplines

Spoken Language Understanding for Natural Interaction: The Siri Experience

Abstract: Recent advances in software integration and efforts toward more personalization and context awareness have brought closer the long-standing vision of the ubiquitous intelligent personal assistant. This has become particularly salient in the context of smartphones and electronic tablets, where natural language interaction has the potential to considerably enhance mobile experience. Far beyond merely offering more options in terms of user interface, this trend may well usher in a genuine paradigm shift in man-machine communication. This contribution reviews the two major semantic interpretation frameworks underpinning natural language interaction, along with their respective advantages and drawbacks. It then discusses the choices made in Siri, Apple’s personal assistant on the iOS platform, and speculates on how the current implementation might evolve in the near future to best mitigate any downside.

BEETLE II: Deep natural language understanding and automatic feedback generation for intelligent tutoring in basic electricity and electronics

Abstract: Within STEM domains, physics is considered to be one of the most difficult topics to master, in part because many of the underlying principles are counter-intuitive. Effective teaching methods rely on engaging the student in active experimentation and encouraging deep reasoning, often through the use of self-explanation. Supporting such instructional approaches poses a challenge for developers of Intelligent Tutoring Systems. We describe a system that addresses this challenge by teaching conceptual knowledge about basic electronics and electricity through guided experimentation with a circuit simulator and reflective dialogue to encourage effective self-explanation. The Basic Electricity and Electronics Tutorial Learning Environment (BEETLE II) advances the state of the art in dynamic adaptive feedback generation and natural language processing (NLP) by extending symbolic NLP techniques to support unrestricted student natural language input in the context of a dynamically changing simulation environment in a moderately complex domain. This allows contextually-appropriate feedback to be generated “on the fly” without requiring curriculum designers to anticipate possible student answers and manually author multiple feedback messages. We present the results of a system evaluation. Our curriculum is highly effective, achieving effect sizes of 1.72 when comparing pre- to post-test learning gains from our system to those of a no-training control group. However, we are unable to demonstrate that dynamically generated feedback is superior to a non-NLP feedback condition. Evaluation of interpretation quality demonstrates its link with instructional effectiveness, and provides directions for future research and development.

Natural language understanding (NLU) processing based on user-specified interests

Abstract: Methods and apparatus for natural language understanding (NLU) processing based on user-specified interests. Information specifying a weight for each of a plurality of domains is received via a user interface. The plurality of domains each relates to a potential area of interest for the user, and the weight for a domain from among the plurality of domains indicates a level of interest for the user in the domain. A ranking classifier used to rank NLU hypotheses generated by an NLU engine is trained using training data from which features are, at least in part, based on the information specifying a weight for each of the plurality of domains.

Tackling Winograd Schemas by formalizing relevance theory in knowledge graphs

Abstract: We study disambiguating of pronoun references in Winograd Schemas, which are part of the Winograd Schema Challenge, a proposed replacement for the Turing test. In particular we consider sentences where the pronoun can be resolved to both antecedents without semantic violations in world knowledge, that means for both readings of the sentence there is a possible consistent world. Nevertheless humans will strongly prefer one answer, which can be explained by pragmatic effects described in Relevance Theory. We state formal optimization criteria based on principles of Relevance Theory in a simplification of Roger Schank's graph framework for natural language understanding. We perform experiments using Answer Set Programming and report the usefulness of our criteria for disambiguation and their sensitivity to parameter variations.

Extending domain coverage of language understanding systems via intent transfer between domains using knowledge graphs and search query click logs

Abstract: This paper proposes a new technique to enable Natural Language Understanding (NLU) systems to handle user queries beyond their original semantic schemas defined by intents and slots. Knowledge graph and search query logs are used to extend NLU system’s coverage by transferring intents from other domains to a given domain. The transferred intents as well as existing intents are then applied to a set of new slots that they are not trained with. The knowledge graph and search click logs are used to determine whether the new slots (i.e. entities) or their attributes in the graph can be used together with transfered intents without re-training the underlying NLU models with the expanded (i.e. with new intents and slots) schema. Experimental results show that the proposed technique can in fact be used in extending NLU system’s domain coverage in fulfilling the user’s request.

Improving domain-independent cloud-based speech recognition with domain-dependent phonetic post-processing

Abstract: Automatic speech recognition (ASR) technology has been developed to such a level that off-the-shelf distributed speech recognition services are available (free of cost), which allow researchers to integrate speech into their applications with little development effort or expert knowledge leading to better results compared with previously used open-source tools. Often, however, such services do not accept language models or grammars but process free speech from any domain. While results are very good given the enormous size of the search space, results frequently contain out-of-domain words or constructs that cannot be understood by subsequent domain-dependent natural language understanding (NLU) components. We present a versatile post-processing technique based on phonetic distance that integrates domain knowledge with open-domain ASR results, leading to improved ASR performance. Notably, our technique is able to make use of domain restrictions using various degrees of domain knowledge, ranging from pure vocabulary restrictions via grammars or N-Grams to restrictions of the acceptable utterances. We present results for a variety of corpora (mainly from human-robot interaction) where our combined approach significantly outperforms Google ASR as well as a plain open-source ASR solution.

Building the next generation of personal digital Assistants

Abstract: Voice-based digital Assistants such as Apple's Siri and Google's Now are currently booming. Yet, despite their promise of being context-aware and adapted to a user's preferences and very distinct needs, truly personal assistants are still missing. In this paper we highlight some of the challenges in building personalized speech-operated assistive technology and propose a number of research and development directions we have undertaken in order to solve them. In particular we focus on natural language understanding and dialog management aspects as we believe that these parts of the technology pipeline require the biggest amount of augmentation.

A framework for learning semantic maps from grounded natural language descriptions

Abstract: This paper describes a framework that enables robots to efficiently learn human-centric models of their environment from natural language descriptions. Typical semantic mapping approaches are limited to augmenting metric maps with higher-level properties of the robot's surroundings (e.g. place type, object locations) that can be inferred from the robot's sensor data, but do not use this information to improve the metric map. The novelty of our algorithm lies in fusing high-level knowledge that people can uniquely provide through speech with metric information from the robot's low-level sensor streams. Our method jointly estimates a hybrid metric, topological, and semantic representation of the environment. This semantic graph provides a common framework in which we integrate information that the user communicates (e.g. labels and spatial relations) with metric observations from low-level sensors. Our algorithm efficiently maintains a factored distribution over semantic graphs based upon the stream of natural language and low-level sensor information. We detail the means by which the framework incorporates knowledge conveyed by the user's descriptions, including the ability to reason over expressions that reference yet unknown regions in the environment. We evaluate the algorithm's ability to learn human-centric maps of several different environments and analyze the knowledge inferred from language and the utility of the learned maps. The results demonstrate that the incorporation of information from free-form descriptions increases the metric, topological, and semantic accuracy of the recovered environment model.

Graph-Based Methods for Natural Language Processing and Understanding—A Survey and Analysis

Abstract: This survey and analysis presents the functional components, performance, and maturity of graph-based methods for natural language processing and natural language understanding and their potential for mature products. Resulting capabilities from the methods surveyed include summarization, text entailment, redundancy reduction, similarity measure, word sense induction and disambiguation, semantic relatedness, labeling (e.g., word sense), and novelty detection. Estimated scores for accuracy, coverage, scalability, and performance are derived from each method. This survey and analysis, with tables and bar graphs, offers a unique abstraction of functional components and levels of maturity from this collection of graph-based methodologies.

Effective and robust natural language understanding for human-robot interaction

Abstract: Robots are slowly becoming part of everyday life, as they are being marketed for commercial applications (viz. telepresence, cleaning or entertainment). Thus, the ability to interact with non-expert users is becoming a key requirement. Even if user utterances can be efficiently recognized and transcribed by Automatic Speech Recognition systems, several issues arise in translating them into suitable robotic actions. In this paper, we will discuss both approaches providing two existing Natural Language Understanding workflows for Human Robot Interaction. First, we discuss a grammar based approach: it is based on grammars thus recognizing a restricted set of commands. Then, a data driven approach, based on a free-from speech recognizer and a statistical semantic parser, is discussed. The main advantages of both approaches are discussed, also from an engineering perspective, i.e. considering the effort of realizing HRI systems, as well as their reusability and robustness. An empirical evaluation of the proposed approaches is carried out on several datasets, in order to understand performances and identify possible improvements towards the design of NLP components in HRI.

Wikification and Beyond: The Challenges of Entity and Concept Grounding

Abstract: Contextual disambiguation and grounding of concepts and entities in natural language are essential to progress in many natural language understanding tasks and fundamental to many applications. Wikification aims at automatically identifying concept mentions in text and linking them to referents in a knowledge base (KB) (e.g., Wikipedia). Consider the sentence, "The Times report on Blumenthal (D) has the potential to fundamentally reshape the contest in the Nutmeg State.". A Wikifier should identify the key entities and concepts and map them to an encyclopedic resource (e.g., “D” refers to Democratic Party, and “the Nutmeg State” refers to Connecticut. Wikification benefits end-users and Natural Language Processing (NLP) systems. Readers can better comprehend Wikified documents as information about related topics is readily accessible. For systems, a Wikified document elucidates concepts and entities by grounding them in an encyclopedic resource or an ontology. Wikification output has improved NLP down-stream tasks, including coreference resolution, user interest discovery , recommendation and search. This task has received increased attention in recent years from the NLP and Data Mining communities, partly fostered by the U.S. NIST Text Analysis Conference Knowledge Base Population (KBP) track, and several versions of it has been studied. These include Wikifying all concept mentions in a single text document; Wikifying a cluster of co-referential named entity mentions that appear across documents (Entity Linking), and Wikifying a whole document to a single concept. Other works relate this task to coreference resolution within and across documents and in the context of multiple text genres. 2 Content Overview

Natural Language Query

Abstract: The invention allows end users to obtain immediate, accurate information from structured databases without writing complex database query commands. The invention allows two different, but synchronized, methods of end user information requests: Spoken or typed Natural Language requests, and a Visual Request Specification method. Furthermore, the invention provides a user the means of “teaching the system the correct interpretation” when an information request was misunderstood by the invention's Natural Language Understanding module.

Modeling the user state for context-aware spoken interaction in ambient assisted living

Abstract: Ambient Assisted Living (AAL) systems must provide adapted services easily accessible by a wide variety of users. This can only be possible if the communication between the user and the system is carried out through an interface that is simple, rapid, effective, and robust. Natural language interfaces such as dialog systems fulfill these requisites, as they are based on a spoken conversation that resembles human communication. In this paper, we enhance systems interacting in AAL domains by means of incorporating context-aware conversational agents that consider the external context of the interaction and predict the user's state. The user's state is built on the basis of their emotional state and intention, and it is recognized by means of a module conceived as an intermediate phase between natural language understanding and dialog management in the architecture of the conversational agent. This prediction, carried out for each user turn in the dialog, makes it possible to adapt the system dynamically to the user's needs. We have evaluated our proposal developing a context-aware system adapted to patients suffering from chronic pulmonary diseases, and provide a detailed discussion of the positive influence of our proposal in the success of the interaction, the information and services provided, as well as the perceived quality.

NLU training with user corrections to engine annotations

Abstract: Techniques for training a natural language understanding (NLU) engine may include generating a first annotation of free-form text documenting a healthcare patient encounter and a link between the first annotation and a corresponding portion of the text, using the NLU engine. A second annotation of the text and a link between the second annotation and a corresponding portion of the text may be received from a human user. The first annotation and its corresponding link may be merged with the second annotation and its corresponding link. Training data may be provided to the engine in the form of the text and the merged annotations and links.

Vishit: A Visualizer for Hindi Text

Abstract: We outline the design of a visualizer, named Vishit, for texts in the Hindi language. The Hindi language is lingua franca in many states of India where people speak different languages. The visualized text serves as a universal language where seamless communication is needed by many people who speak different languages and have different cultures. Vishit consists of the following three major processing steps: language processing, knowledge base creation and scene generation. Initial results from the Vishit prototype are encouraging.

Man-machine conversation system

Abstract: The invention discloses a man-machine conversation system applied to an advanced service robot. The man-machine conversation system is characterized in that a voice recognition module, a natural language understanding module, a background service processing module, a natural language generating module and a voice generating module are included and are respectively and independently connected with a conversation management module and used for bidirectional data transmission. The man-machine conversation system has the advantages that the advanced service robot can directly communicate with the human language, the influence caused by a traditional communication way on communication efficiency is avoided, the convenience of man-machine communication is greatly improved, and the aim that the service robot directly listens to human voice instructions is achieved.

Benchmarking Applied Semantic Inference: The PASCAL Recognising Textual Entailment Challenges

Abstract: Identifying that the same meaning is expressed by, or can be inferred from, various language expressions is a major challenge for natural language understanding applications such as information extraction, question answering and automatic summarization. Dagan and Glickman [5] proposed Textual Entailment, the task of deciding whether a target text follows from a source text, as a unifying framework for modeling language variability, which has often been addressed in an application-specific manner. In this paper we describe the series of benchmarks developed for the textual entailment recognition task, known as the PASCAL RTE Challenges. As a concrete example, we describe in detail the second RTE challenge, in which our methodology was consolidated, and served as a basis for the subsequent RTE challenges. The impressive success of these challenges established textual entailment as an active research area in natural language processing, attracting a growing community of researchers.

Syntactic Parser Assisted Semantic Rule Inference

Abstract: Natural language understanding (NLU) engines perform better when they are trained with large amounts of data. However, a large amount of data is not always available. Embodiments of the present invention overcome this problem by generating annotated data for use in a NLU system. An example embodiment generates annotated data by parsing an input annotated phrase, generating a syntactic tree reflecting a grammatical structure of the parsed phrase, and generating one or more alternative versions of the input annotated phrase based on the syntactic tree. Alignment between expressions and corresponding annotations in the annotated phrase are preserved in the one or more alternative versions generated to ensure intention of the input annotated phrase is maintained.

Situated Incremental Natural Language Understanding using a Multimodal, Linguistically-driven Update Model

Abstract: A common site of language use is interactive dialogue between two people situated together in shared time and space In this paper, we present a statistical model for understanding natural human language that works incrementally (ie, does not wait until the end of an utterance to begin processing), and is grounded by linking semantic entities with objects in a shared space We describe our model, show how a semantic meaning representation is grounded with properties of real-world objects, and further show that it can ground with embodied, interactive cues such as pointing gestures or eye gaze

Latest Advances in Inductive Logic Programming

Abstract: This book represents a selection of papers presented at the Inductive Logic Programming (ILP) workshop held at Cumberland Lodge, Great Windsor Park. The collection marks two decades since the first ILP workshop in 1991. During this period the area has developed into the main forum for work on logic-based machine learning. The chapters cover a wide variety of topics, ranging from theory and ILP implementations to state-of-the-art applications in real-world domains. The international contributors represent leaders in the field from prestigious institutions in Europe, North America and Asia. Graduate students and researchers in this field will find this book highly useful as it provides an up-to-date insight into the key sub-areas of implementation and theory of ILP. For academics and researchers in the field of artificial intelligence and natural sciences, the book demonstrates how ILP is being used in areas as diverse as the learning of game strategies, robotics, natural language understanding, query search, drug design and protein modelling.Readership: Graduate students and researchers in the field of ILP, and academics and researchers in the fields of artificial intelligence and natural sciences.

System and method for localized error detection of recognition results

Abstract: A system, method and computer-readable storage devices are disclosed for using targeted clarification (TC) questions in dialog systems in a multimodal virtual agent system (MVA) providing access to information about movies, restaurants, and musical events. In contrast with open-domain spoken systems, the MVA application covers a domain with a fixed set of concepts and uses a natural language understanding (NLU) component to mark concepts in automatically recognized speech. Instead of identifying an error segment, localized error detection (LED) identifies which of the concepts are likely to be present and correct using domain knowledge, automatic speech recognition (ASR), and NLU tags and scores. If at least concept is identified to be present but not correct, the TC component uses this information to generate a targeted clarification question. This approach computes probability distributions of concept presence and correctness for each user utterance, which can apply to automatic learning for clarification policies.

Language-independent ensemble approaches to metaphor identification

Abstract: True natural language understanding requires the ability to identify and understand metaphorical utterances, which are ubiquitous in human communication of all kinds. At present, however, even the problem of identifying metaphors in arbitrary text is very much an unsolved problem, let alone analyzing their meaning. Furthermore, no current methods can be transferred to new languages without the development of extensive language-specific knowledge bases and similar semantic resources. In this paper, we present a new language-independent ensemble-based approach to identifying linguistic metaphors in natural language text. The system's architecture runs multiple corpus-based metaphor identification algorithms in parallel and combines their results. The architecture allows easy integration of new metaphor identification schemes as they are developed. This new approach achieves state-of-the-art results over multiple languages and represents a significant improvement over existing methods for this problem.

The singularity and the state of the art in artificial intelligence

Abstract: The state of the art in automating basic cognitive tasks, including vision and natural language understanding, is far below human abilities. Real-world reasoning, which is an unavoidable part of many advanced forms of computer vision and natural language understanding, is particularly difficult. This suggests that the advent of computers with superhuman general intelligence is not imminent. The possibility of attaining a singularity by computers that lack these abilities is discussed briefly. When I was invited to contribute an article on the subject of “The Singularity”, my initial reaction was “I fervently pray that I don’t live to see it.” Parenthetically, I may say the same of the immortality singularity. I am 55 years old, as of the time of writing, and, compared to most of humanity, I live in circumstances much more comfortable than anything I have earned; but I am hoping not to be here 50 years hence; and certainly hoping not to be here 100 years hence. Whether I will view this question with the same sang froid as the time comes closer remains to be seen. However, my personal preferences are largely irrelevant. What I primarily want to do in this essay on the singularity is to discuss the one aspect of the question where I can pretend to any kind of expertise: the state of the art in artificial intelligence (AI) and the challenges in achieving humanlevel performance on AI tasks. I will argue that these do not suggest that computers will attain an intelligence singularity any time in the near future. I will then, much more conjecturally, discuss whether or not an intelligence singularity might be able to sidestep these challenges. 1 Artificial Intelligence What is artificial intelligence? There are numerous different definitions, with different slants (see [9] chap. 1 for a review and discussion). However, the definition that I prefer is this: There are a number of cognitive tasks that people do easily — often, indeed, with no conscious thought at all — but that it is extremely hard to program on computers. Archetypal examples are vision, natural language understanding, and “real-world reasoning”; I will elaborate on this last below. Artificial intelligence, as I define it, is the study of getting computers to carry out these tasks. Now, it can be objected that this definition is inherently unfair to the computers. If you define AI as “problems that are hard for computers,” then it is no surprise that these are hard for computers.