Showing papers on "Natural language understanding published in 2009"

The Anatomy of A.L.I.C.E.

Abstract: This paper is a technical presentation of Artificial Linguistic Internet Computer Entity (ALICE) and Artificial Intelligence Markup Language (AIML), set in context by historical and philosophical ruminations on human consciousness ALICE, the first AIML-based personality program, won the Loebner Prize as “the most human computer” at the annual Turing Test contests in 2000, 2001, and 2004 The program, and the organization that develops it, is a product of the world of free software More than 500 volunteers from around the world have contributed to her development This paper describes the history of ALICE and AIML-free software since 1995, noting that the theme and strategy of deception and pretense upon which AIML is based can be traced through the history of Artificial Intelligence research This paper goes on to show how to use AIML to create robot personalities like ALICE that pretend to be intelligent and selfaware The paper winds up with a survey of some of the philosophical literature on the question of consciousness We consider Searle’s Chinese Room, and the view that natural language understanding by a computer is impossible We note that the proposition “consciousness is an illusion” may be undermined by the paradoxes it apparently implies We conclude that ALICE does pass the Turing Test, at least, to paraphrase Abraham Lincoln, for some of the people some of the time

Natural Language Understanding

Abstract: I focus on three characteristics of natural language understanding systems that incorporate the properties that make humans able to understand language naturally. The first characteristic of such systems is that they handle recursion. A second property of these systems is that they process abstract hierarchical structures, and they are not limited to the processing of strings of characters or keywords. A third characteristic is that they connect physical forms and interpretations.

Recognizing textual entailment: Rational, evaluation and approaches

Abstract: The goal of identifying textual entailment – whether one piece of text can be plausibly inferred from another – has emerged in recent years as a generic core problem in natural language understanding. Work in this area has been largely driven by the PASCAL Recognizing Textual Entailment (RTE) challenges, which are a series of annual competitive meetings. The current work exhibits strong ties to some earlier lines of research, particularly automatic acquisition of paraphrases and lexical semantic relationships and unsupervised inference in applications such as question answering, information extraction and summarization. It has also opened the way to newer lines of research on more involved inference methods, on knowledge representations needed to support this natural language understanding challenge and on the use of learning methods in this context. RTE has fostered an active and growing community of researchers focused on the problem of applied entailment. This special issue of the JNLE provides an opportunity to showcase some of the most important work in this emerging area.

Interpreting written how-to instructions

Abstract: Written instructions are a common way of teaching people how to accomplish tasks on the web. However, studies have shown that written instructions are difficult to follow, even for experienced users. A system that understands human-written instructions could guide users through the process of following the directions, improving completion rates and enhancing the user experience. While general natural language understanding is extremely difficult, we believe that in the limited domain of howto instructions it should be possible to understand enough to provide guided help in a mixed-initiative environment. Based on a qualitative analysis of instructions gathered for 43 web-based tasks, we have formalized the problem of understanding and interpreting how-to instructions. We compare three different approaches to interpreting instructions: a keyword-based interpreter, a grammar-based interpreter, and an interpreter based on machine learning and information extraction. Our empirical results demonstrate the feasibility of automated how-to instruction understanding.

Towards Natural Language Understanding of Partial Speech Recognition Results in Dialogue Systems

Abstract: We investigate natural language understanding of partial speech recognition results to equip a dialogue system with incremental language processing capabilities for more realistic human-computer conversations. We show that relatively high accuracy can be achieved in understanding of spontaneous utterances before utterances are completed.

EA NLU: Practical Language Understanding for Cognitive Modeling

Abstract: This paper presents an approach to creating flexible general-logic representations from language for use in high-level reasoning tasks in cognitive modeling These representations are grounded in a large-scale ontology and emphasize the need for semantic breadth at the cost of syntactic breadth The task-independent interpretation process allows task-specific pragmatics to guide the interpretation process In the context of a particular cognitive model, we discuss our use of limited abduction for interpretation and show results of its performance

Automated text-based messaging interaction using natural language understanding technologies

Abstract: Automated text-based messaging interaction using natural language understanding technologies enables text-based messages to be received from users and interpreted by a self-service application platform so that the self-service application platform can respond to the text-based messages in an automated manner. The characters and strings of characters contained within the text message are interpreted to extract words, which are then processed using a natural language understanding engine to determine the content of the text-based message. The content is used to generate a response message from static and/or dynamic grammars to automate the process of interacting with a user via text-based messages. Multiple text-based message formats are supported, including text messages transmitted using Short Messaging Service (SMS), instant messaging, chat, and e-mail.

Bringing Role and Reference Grammar to natural language understanding

Abstract: Financial support for this research has been provided by the DGI, Spanish Ministry of Education and Science, grant FFI2008-05035-C02-01/FILO. The research has been co-financed through FEDER funds.

Applications of Statistical Machine Translation Approaches to Spoken Language Understanding

Abstract: In this paper, we investigate two statistical methods for spoken language understanding based on statistical machine translation. The first approach employs the source-channel paradigm, whereas the other uses the maximum entropy framework. Starting with an annotated corpus, we describe the problem of natural language understanding as a translation from a source sentence to a formal language target sentence. We analyze the quality of different alignment models and feature functions and show that the direct maximum entropy approach outperforms the source channel-based method. Furthermore, we investigate how both methods perform if the input sentences contain speech recognition errors. Finally, we investigate a new approach to combine speech recognition and spoken language understanding. For this purpose, we employ minimum error rate training which directly optimizes the final evaluation criterion. By combining all knowledge sources in a log-linear way, we show that we can decrease both the word error rate and the slot error rate. Experiments were carried out on two German inhouse corpora for spoken dialogue systems.

Role of natural language understanding in voice local search.

Abstract: Speak4it is a voice-enabled local search system currently available for iPhone devices. The natural language understanding (NLU) component is one of the key technology modules in this system. The role of NLU in voice-enabled local search is twofold: (a) parse the automatic speech recognition (ASR) output (1-best and word lattices) into meaningful segments that contribute to high-precision local search, and (b) understand user’s intent. This paper is concerned with the first task of NLU. In previous work, we had presented a scalable approach to parsing, which is built upon text indexing and search framework, and can also parse ASR lattices. In this paper, we propose an algorithm to improve the baseline by extracting the “subjects” of the query. Experimental results indicate that lattice-based query parsing outperforms ASR 1-best based parsing by 2.1% absolute and extracting subjects in the query improves the robustness of search.

Joint Inference for Natural Language Processing

Abstract: In recent decades, researchers in natural language processing have made great progress on well-defined subproblems such as part-of-speech tagging, phrase chunking, syntactic parsing, named-entity recognition, coreference and semantic-role labeling. Better models, features, and learning algorithms have allowed systems to perform many of these tasks with 90% accuracy or better. However, success in integrated, end-to-end natural language understanding remains elusive. I contend that the chief reason for this failure is that errors cascade and accumulate through a pipeline of naively chained components. For example, if we naively use the single most likely output of a part-of-speech tagger as the input to a syntactic parser, and those parse trees as the input to a coreference system, and so on, errors in each step will propagate to later ones: each components 90% accuracy multiplied through six components becomes only 53%. Consider, for instance, the sentence "I know you like your mother." If a part-of-speech tagger deterministically labels "like" as a verb, then certain later syntactic and semantic analysis will be blocked from alternative interpretations, such as "I know you like your mother (does)." The part-of-speech tagger needs more syntactic and semantic information to make this choice. Consider also the classic example "The boy saw the man with the telescope." No single correct syntactic parse of this sentence is possible in isolation. Correct interpretation requires the integration of these syntactic decisions with semantics and context. Humans manage and resolve ambiguity by unified, simultaneous consideration of morphology, syntax, semantics, pragmatics and other contextual information. In statistical modeling such unified consideration is known as joint inference. The need for joint inference appears not only in natural language processing, but also in information integration, computer vision, robotics and elsewhere. All of these applications require integrating evidence from multiple sources, at multiple levels of abstraction. I believe that joint inference is one of the most fundamentally central issues in all of artificial intelligence. In this talk I will describe work in probabilistic models that perform joint inference across multiple components of an information processing pipeline in order to avoid the brittle accumulation of errors. I will survey work in exact inference, variational inference and Markov-chain Monte Carlo methods. We will discuss various approaches that have been applied to natural language processing, and hypothesize about why joint inference has helped in some cases, and not in others. I will then focus on our recent work at University of Massachusetts in large-scale conditional random fields with complex relational structure. In a single factor graph we seamlessly integrate multiple subproblems, using our new probabilistic programming language to compactly express complex, mutable variable-factor structure both in first-order logic as well as in more expressive Turing-complete imperative procedures. We avoid unrolling this graphical model by using Markov-chain Monte Carlo for inference, and make inference more efficient with learned proposal distributions. Parameter estimation is performed by SampleRank, which avoids complete inference as a subroutine by learning simply to correctly rank successive states of the Markov-chain. Joint work with Aron Culotta, Michael Wick, Rob Hall, Khashayar Rohanimanesh, Karl Schultz, Sameer Singh, Charles Sutton and David Smith.

Method for building a natural language understanding model for a spoken dialog system

Abstract: A method of generating a natural language model for use in a spoken dialog system is disclosed. The method comprises using sample utterances and creating a number of hand crafted rules for each call-type defined in a labeling guide. A first NLU model is generated and tested using the hand crafted rules and sample utterances. A second NLU model is built using the sample utterances as new training data and using the hand crafted rules. The second NLU model is tested for performance using a first batch of labeled data. A series of NLU models are built by adding a previous batch of labeled data to training data and using a new batch of labeling data as test data to generate the series of NLU models with training data that increases constantly. If not all the labeling data is received, the method comprises repeating the step of building a series of NLU models until all labeling data is received. After all the training data is received, at least once, the method comprises building a third NLU model using all the labeling data, wherein the third NLU model is used in generating the spoken dialog service.

Toward Human Level Machine Intelligence – Is it Achievable? The Need for a Paradigm Shift

Abstract: Officially, AI was born in 1956. Since then, very impressive progress has been made in many areas – but not in the realm of human level machine intelligence. Anyone who has been forced to use a dumb automated customer service system will readily agree. The Turing Test lies far beyond. Today, no machine can pass the Turing Test and none is likely to do so in the foreseeable future. During much of its early history, AI was rife with exaggerated expectations. A headline in an article published in the late forties of last century was headlined, “Electric brain capable of translating foreign languages is being built.” Today, more than half a century later, we do have translation software, but nothing that can approach the quality of human translation. Clearly, achievement of human level machine intelligence is a challenge that is hard to meet. Humans have many remarkable capabilities; there are two that stand out in importance. First, the capability to reason, converse and make rational decisions in an environment of imprecision, uncertainty, incompleteness of information, partiality of truth and possibility. And second, the capability to perform a wide variety of physical and mental tasks without any measurements and any computations. A prerequisite to achievement of human level machine intelligence is mechanization of these capabilities and, in particular, mechanization of natural language understanding. In my view, mechanization of these capabilities is beyond the reach of the armamentarium of AI – an armamentarium which in large measure is based on classical, Aristotelian, bivalent logic and bivalent-logic-based probability theory. To make significant progress toward achievement of human level machine intelligence a paradigm shift is needed. More specifically, what is needed is an addition to the armamentarium of AI of two methodologies: (a) a nontraditional methodology of computing with words (CW) or more generally, NL-Computation; and (b) a countertraditional methodology which involves a progression from computing with numbers to computing with words. The centerpiece of these methodologies is the concept of precisiation of meaning. Addition of these methodologies to AI would be an important step toward the achievement of human level machine intelligence and its applications in decision-making, pattern recognition, analysis of evidence, diagnosis and assessment of causality. Such applications have a position of centrality in our infocentric society.

State-of-the-art NLP Approaches to Coreference Resolution: Theory and Practical Recipes

Abstract: The identification of different nominal phrases in a discourse as used to refer to the same (discourse) entity is essential for achieving robust natural language understanding (NLU). The importance of this task is directly amplified by the field of Natural Language Processing (NLP) currently moving towards high-level linguistic tasks requiring NLU capabilities such as e.g. recognizing textual entailment. This tutorial aims at providing the NLP community with a gentle introduction to the task of coreference resolution from both a theoretical and an application-oriented perspective. Its main purposes are: (1) to introduce a general audience of NLP researchers to the core ideas underlying state-of-the-art computational models of coreference; (2) to provide that same audience with an overview of NLP applications which can benefit from coreference information.

A pragmatic approach to computational narrative understanding

Abstract: Narrative understanding is a hard problem for artificial intelligence that requires deep semantic understanding of natural language and broad world knowledge. Early research in this area stalled due to the difficulty of knowledge engineering and a trend in the field towards robustness at the expense of depth. This work explores how a practical integration of more recent resources and theories for natural language understanding can perform deep semantic interpretation of narratives when guided by specific pragmatic constraints. It shows how cognitive models can provide pragmatic context for narrative understanding in terms of well-defined reasoning tasks, and how those tasks can be used to guide interpretation and evaluate understanding. This work presents an implemented system, EA NLU, which has been used to interpret narrative text input to cognitive modeling simulations. EA NLU integrates existing large-scale knowledge resources with a controlled grammar and a compositional semantic interpretation process to generate highly expressive logical representations of sentences. Delayed disambiguation and representations from dynamic logic are used to separate this compositional process from a querydriven discourse interpretation process that is guided by pragmatic concerns and uses world knowledge. By isolating explicit points of ambiguity and using limited evidential abduction, this query-driven process can automatically identify the disambiguation choices that entail relevant interpretations. This work shows how this approach maintains computational tractability without sacrificing expressive power. EA NLU is evaluated through a series of experiments with two cognitive models, showing that it is capable of meeting the deep reasoning requirements those models pose, and that the constraints provided by the models can effectively guide the interpretation process. By enforcing consistent interpretation principles, EA NLU benefits the cognitive modeling experiments by reducing the opportunities for tailoring the input. This work also explores the use of a theory of narrative functions as a heuristic guide to interpretation in EA NLU. In contrast to potentially global task-specific queries, these narrative functions can be inferred on a sentence-by-sentence basis, providing incremental disambiguation. This method is evaluated by interpreting a set of Aesop's fables, and showing that the interpretations are sufficient to capture the intended lesson of each fable.

Natural Language Understanding and Assessment

Abstract: INTRODUCTION Natural language understanding and assessment is a subset of natural language processing (NLP). The primary purpose of natural language understanding algorithms is to convert written or spoken human language into representations that can be manipulated by computer programs. Complex learning environments such as intelligent tutoring systems (ITSs) often depend on natural language understanding for fast and accurate interpretation of human language so that the system can respond intelligently in natural language. These ITSs function by interpreting the meaning of student input, assessing the extent to which it manifests learning, and generating suitable feedback to the learner. To operate effectively, systems need to be fast enough to operate in the real time environments of ITSs. Delays in feedback caused by computational processing run the risk of frustrating the user and leading to lower engagement with the system. At the same time, the accuracy of assessing student input is critical because inaccurate feedback can potentially compromise learning and lower the student's motivation and metacognitive awareness of the learning goals of the system (Millis et al., 2007). As such, student input in ITSs requires an assessment approach that is fast enough to operate in real time but accurate enough to provide appropriate evaluation. One of the ways in which ITSs with natural language understanding verify student input is through matching. In some cases, the match is between the user input and a pre-selected stored answer to a question, solution to a problem, misconception, or other form of benchmark response. In other cases, the system evaluates the degree to which the student input varies from a complex representation or a dynamically computed structure. The computation of matches and similarity metrics are limited by the fidelity and flexibility of the computational linguistics modules. The major challenge with assessing natural language input is that it is relatively unconstrained and rarely follows brittle rules in its computation of spelling, syntax, and semantics (McCarthy et al., 2007). Researchers who have developed tutorial dialogue systems in natural language have explored the accuracy of matching students' written input to targeted knowledge. Examples of these systems are AutoTutor and Why-Atlas, which such statistical and word overlap algorithms can boast much success. However, over short dialogue exchanges (such as those in ITSs), the accuracy of interpretation can be seriously compromised without a deeper level of lexico-syntactic textual assessment (McCarthy et al., 2007). Such a lexico-syntactic approach, entailment evaluation, is presented in …

Linguistic text mining for problem reports

Abstract: This paper describes a linguistic text mining tool for analyzing problem reports in aerospace engineering and safety organizations. The Semantic Trend Analysis Tool (STAT) helps analysts find and review recurrences, similarities and trends in problem reports. The tool is being used to analyze engineering discrepancy reports at NASA Johnson Space Center. The tool has been augmented with a statistical natural language parser that also resolves parsing gaps and identifies verb arguments and adjuncts. The tool uses an aerospace ontology augmented with features of taxonomies and thesauruses. The ontology defines hierarchies of problem types, equipment types and function types. STAT uses the output of the parser and the aerospace ontology to identify words and phrases in problem report descriptions that refer to types of hazards, equipment damage, performance deviations or functional impairments. Tool performance has been evaluated on 120 problem descriptions from problem reports, with encouraging results.

Instance-based Metaphor Comprehension and Generation

Abstract: In natural language,simile,always considered as a marked metaphor,is easier to recognize and provides an ideal knowledge source for metaphor comprehension and generationThis paper proposed a strategy to acquire large-scale simile instances with Web search engine,and to construct a simile knowledge-base automaticallyBased on the simile knowledge-base,the source domains of Chinese metaphors were studied and an instance-based method for metaphor comprehension and generation was put forwardExperiment results show that the method achieves high precisions for both comprehension and generation,and has a good expandabilityIn addition,the collocational relations in the knowledge-base are useful for many other natural language processing tasks

Annotation of joint projects and information states in human-NPC dialogues

Abstract: We present a corpus of human-NPC interactions in a virtual environment. The corpus has been obtained through a Wizard-of-Oz experiment simulating a scenario where the user furnishes a room with the help of a virtual interior designer. With the aim of extracting useful information for the development of a dialogue model, an annotation scheme and representation format have been designed. The unit of annotation is the minimal joint project. Minimal joint projects are represented as feature-structures containing information on their goals, the information state shared by the dialogue participants and the actions composing the projects. The representation format is suitable for describing dialogues independently of the task and domain and can serve as representation of dialogue states in dialogue models. A methodology for the generation of project representations relying on manual and automatic annotation has also been developed. The annotated corpus allows for the automatic extraction of dialogue transitions and delivers useful information for the development of the natural language understanding and generation modules.

Statistical methods in natural language understanding and spoken dialogue systems

Abstract: Modern automatic spoken dialogue systems cover a wide range of applications. There are systems for hotel reservations, restaurant guides, systems for travel and timetable information, as well as systems for automatic telephone-banking services. Building the different components of a spoken dialogue system and combining them in an optimal way such that a reasonable dialogue becomes possible is a complex task because during the course of a dialogue, the system has to deal with uncertain information. In this thesis, we use statistical methods to model and combine the system’s components. Statistical methods provide a well-founded theory for modeling systems where decisions have to be made under uncertainty. Starting from Bayes’ decision rule, we define and evaluate various statistical models for these components, which comprise speech recognition, natural language understanding, and dialogue management. The problem of natural language understanding is described as a special machine translation problem where a source sentence is translated into a formal language target sentence consisting of concepts. For this, we define and evaluate two models. The first model is a generative model based on the source-channel paradigm. Because the word context plays an important role in natural language understanding tasks, we use a phrasebased translation system in order to take local context dependencies into account. The second model is a direct model based on the maximum entropy framework and works similar to a tagger. For the direct model, we define several feature functions that capture dependencies between words and concepts. Both methods have the advantage that only source-target pairs in the form of input-output sentences must be provided for training. Thus, there is no need to generate grammars manually, which significantly reduces the costs of building dialogue systems for new domains. Furthermore, we propose and investigate a framework based on minimum error rate training that results in a tighter coupling between speech recognition and language understanding. This framework allows for an easy integration of multiple knowledge sources by minimizing the overall error criterion. Thus, it is possible to add language understanding features to the speech recognition framework and thus to minimize the word error rate, or to add speech recognition features to the language understanding framework and thus to minimize the slot error rate. Finally, we develop a task-independent dialogue manager using trees as the fundamental data structure. Based on a cost function, the dialogue manager chooses the next dialogue action with minimal costs. The design and the task-independence of the dialogue manager leads to a strict separation of a given application and the operations performed by the dialogue manager, which simplifies porting an existing dialogue system to a new domain. We report results from a field test in which the dialogue manager was able to choose the optimal dialogue action in 90% of the decisions. We investigate techniques for error handling based on confidence measures defined for speech recognition and language understanding. Furthermore, we investigate the overall performance of the dialogue system when confidence measures from speech recognition and natural language understanding are incorporated into the dialogue strategy. Experiments have been carried out on the TelDir database, which is a German in-house telephone directory assistance corpus, and on the Taba database, which is a German in-house train time scheduling task.

The Use of Scripts Based on Conceptual Dependency Primitives for the Operation of Service Mobile Robots

Abstract: This paper describes a Human-Robot interaction subsystem that is part of a robotics architecture, the ViRbot, used to control the operation of service mobile robots. The Human/Robot Interface subsystem consists of tree modules: Natural Language Understanding, Speech Generation and Robot's Facial Expressions. To demonstrate the utility of this Human-Robot interaction subsystem it is presented a set of applications that allows a user to command a mobile robot through spoken commands. The mobile robot accomplish the required commands using an actions planner and reactive behaviors. In the ViRbot architecture the actions planner module uses Conceptual Dependency (CD) primitives as the base for representing the problem domain. After a command is spoken a CD representation of it is generated, a rule base system takes this CD representation, and using the state of the environment generates other subtasks represented by CDs to accomplish the command. In this paper is also presented how to represent context through scripts. Using scripts it is easy to make inferences about events for which there are incomplete information or are ambiguous. Scripts serve to encode common sense knowledge. Scripts are also used to fill the gaps between seemingly unrelated events.

Methods of Rule Acquisition in the TextLearner System.

Abstract: This paper describes the TextLearner prototype, a knowledgeacquisition program that represents the culmination of the DARPA-IPTO-sponsored Reading Learning Comprehension seedling program, an effort to determine the feasibility of autonomous knowledge acquisition through the analysis of text. Built atop the Cyc Knowledge Base and implemented almost entirely in the formal representation language of CycL, TextLearner is an anomaly in the way of Natural Language Understanding programs. The system operates by generating an information-rich model of its target document, and uses that model to explore learning opportunities. In particular, TextLearner generates and evaluates hypotheses, not only about the content of the target document, but about how to interpret unfamiliar natural language constructions. This paper focuses on this second capability and describes four algorithms TextLearner uses to acquire rules for interpreting text.

Computer-aided natural language annotation

Abstract: The present invention uses a natural language understanding system that is currently being trained to assist in annotating training data for training that natural language understanding system. Unannotated training data is provided to the system and the system proposes annotations to the training data. The user is offered an opportunity to confirm or correct the proposed annotations, and the system is trained with the corrected or verified annotations.

Evaluating the Potential Utility of ASR N-Best Lists for Incremental Spoken Dialogue Systems

Abstract: The potential of using ASR n-best lists for dialogue systems has often been recognised (if less often realised): it is often the case that even when the top-ranked hypothesis is erroneous, a better one can be found at a lower rank. In this paper, we describe metrics for evaluating whether the same potential carries over to incremental dialogue systems, where ASR output is consumed and reacted upon while speech is still ongoing. We show that even small N can provide an advantage for semantic processing, at a cost of a computational overhead. Index Terms: dialogue systems, speech recognition, natural language understanding, incrementality

Intelligent Decision Support System Based on Natural Language Understanding

Abstract: In this paper, we simply introduced the traditional decision support system and its characteristics and limitations. And then some new types of intelligent decision support system are recommended, such as GDSS, GDSS, 3IDSS and IDSSKD. And at last IDSS based on natural language understanding are described in detail. Some techniques and methods of IDSSNLU are introduced detailedly. The prospects of IDSS based on natural language understanding is put foreword. Keywords-intelligent decision support system, knowledge discovery, natural language understanding

Verbal polysemy resolution through contextualized clustering of arguments

Abstract: Natural language is characterized by a high degree of polysemy, and the majority of content words accept multiple interpretations. However, this does not significantly complicate natural language understanding. Native speakers rely on context to assign the correct sense to each word in an utterance. NLP applications, such as automated word sense disambiguation, require the ability to identify correctly context elements that activate each sense. Our goal in this work is to address the problem of contrasting semantics of the arguments as the source of meaning differentiation for the predicate. We investigate different factors that influence the way sense differentiation for predicates is accomplished in composition and develop a method for identifying semantically diverse arguments that activate the same sense of a polysemous predicate. The method targets specifically polysemous verbs, with an easy extension to other polysemous words. The proposed unsupervised learning method is completely automatic and relies exclusively on distributional information, intentionally eschewing the use of human-constructed knowledge sources and annotated data. We develop the notion of selectional equivalence for polysemous predicates and propose a method for contextualizing the representation of a lexical item with respect to the particular context provided by the predicate. We also present the first attempt at developing a sense-annotated data set that targets sense distinctions dependent predominantly on semantics of a single argument as the source of disambiguation for the predicate. We analyze the difficulties involved in doing semantic annotation for such task. We examine different types of relations within sense inventories and give a qualitative analysis of the effects they have on decisions made by the annotators, as well as annotator error. The developed data set is used to evaluate the quality of the proposed clustering method. The output is adapted for evaluation within a standard sense induction paradigm. We use several evaluation measures to assess different aspects of the algorithm’s performance. Relative to the baselines, we outperform the best systems in the recent SEMEVAL sense induction task (Agirre et al., 2007) on two out of three measures. We also discuss further extensions and possible uses for the proposed automatic algorithm, including the identification of selectional behavior of complex nominals (Pustejovsky, 1995) and the disambiguation of noun phrases with semantically weak head nouns.

Resolving references and identifying existing knowledge in a memory based parser

Abstract: Learning by reading systems, designed to acquire episodic (instance based) knowledge, ultimately have to integrate that knowledge into an underlying memory. In order to effectively integrate new knowledge with existing knowledge such a system needs to be able to resolve references to the instances (agents, locations, events, etc.) it is reading about with those already existing in memory. This is necessary to extend existing memory structures, and to avoid incorrectly producing duplicate memories. Direct Memory Access Parsing (DMAP) leverages existing knowledge and performs reference resolution and memory integration in the early stages of parsing natural language text. By performing incremental memory integration our system can reduce the number of ambiguous sentence interpretations and coreference mappings it will explore in-depth, however this savings is currently canceled out by the run-time cost of reference resolution algorithm. This paper supports the continued investigation of this line of research, which is to identify and evaluate the extent to which semantic and episodic memory can facilitate natural language understanding, especially when used early in the language understanding process.

Natural language understanding in controlled virtual environments

Abstract: ion Yes Yes Yes Yes Yes Indexing method Word Word Word and Sense Word and Sense Word and Sense 115424 synsets 8869 senses 3400 senses Coverage 41100 entries ? 355224 head words 7463 head words 3600 head words Linguistic Corpus Psycholinguistic Corpus Psycholinguistic Sense origin intuition Corpus examples knowledge examples knowledge Compatibility Poor Poor Good Poor Poor Table 3.5: Differences between Word Sense Resources CHAPTER 3. TECHNOLOGY FOUNDATIONS 80 "Paula hit the ball" Example 6: VerbNet Frame Example CHAPTER 3. TECHNOLOGY FOUNDATIONS 81 3.9.2 Overview of Commonly Used WSD Techniques In the broadest sense, there are two types of WSD systems: supervised and unsupervised. A prototypical supervised WSD system must rely on sense-labelled training examples as its primary source of disambiguation information, as a result, the quality and quantity of the training data have always been the bottleneck of the supervised WSD systems. In contrast, a prototypical unsupervised WSD system do not rely on sense-labelled training data and instead rely on machine readable dictionaries, thesauruses and unlabelled corpora as its main sources of disambiguation information.11 From the coverage’s point of view, unsupervised WSD systems tend to be able to handle much greater number of polysemous words than supervised systems because of the lack of sense-labelled data. However, from the performance’s point of view, supervised systems tend to perform much better than unsupervised systems on words for which training data is available. For the purpose of this research, the choice of WSD system will depend on the nature of the movie scripts, and this issue will be discussed in detail in chapter ?? where the movie script corpus is described. General Context Based Word Sense Disambiguation The most common source of disambiguation features for this particular type of WSD system is the surrounding words of the ambiguous word, (Yarowsky 1995), (Yarowsky 1993), (Gale et al. 1992). Using just the surrounding words, it is possible to create the following disambiguation features: 1. n-grams of the lemmatized surrounding words within a window of K work tokens to the left and right of the target polysemous word 2. n-grams of the Part of Speech (POS) tags of the surrounding words within a window of K work tokens to the left and right of the target polysemous word To illustrate the n-grams related features, consider the ambiguous word bank in the sentence “I went to the bank to apply for a home loan. ”. If the window size is chosen to be 5 words, i.e. 5 words to the left of bank and 5 words to the right of bank, then the following n-gram features could be extracted: uni-gram (“SENTENCE START”), (“I”), (“go”), (“to”), (“the”), (“to”), (“apply”), (“for”), (“a”), (“home”), (“loan”) bi-gram (“SENTENCE START”, “I”), (“I”, “go”), (“go”, “to”), (“to”, “the”), (“to”, “apply”), (“apply”, “for”), (“for”, “a”), (“a”, “home”), (“home”, “loan”) However, it is also possible for an unsupervised WSD system to use labelled training examples. CHAPTER 3. TECHNOLOGY FOUNDATIONS 82 It is also possible to include the relative positions of the n-grams as part of the feature. For example, a uni-gram feature of (“apply”) could become (“apply”, +) to indicate it is to the right of the target word, or even (“apply”, 2) to indicate it is the second word token to the right of the target word. With other commonly available natural language processing tools, it would not be difficult to include higher-level features such as: 1. WordNet taxonomy information for the surrounding words 2. Chunking information 3. Parse tree information 4. Named Entity (NE) information 5. Anaphora information Most general-purpose WSD systems would employ a particular combination of empirical models, and train classifiers with a set of features similar to the ones mentioned above. Examples of such WSD systems include Yarowsky (1995), Ng and Lee (1996) and Cabezas et al. (2001). On top of the features, two additional heuristics are often used in general purpose WSD systems to provide additional disambiguation information when the genre or topic of the sentence containing the ambiguous word is available. The first is the one sense per collocation (OSPC) heuristic (Yarowsky 1993), it states that if a word tokens c is found to occur within the context of a particular sense si of a polysemous word w, then it is highly likely that whenever c co-occurs with w, si will be w’s correct sense. For example, consider the polysemous word plant with the following two senses: plant1: buildings for carrying on industrial labour plant2: a living organism lacking the power of locomotion The first sense of plant is likely to co-occur with words such as “factory”, “worker”, “engineer” and “machinery”; whereas the second sense would likely to co-occur with words such as “flower”, “root”, and “leaf”. As a result, these words tend to have sufficient disambiguation power to determine the correct sense within given contexts. The second heuristic is one sense per discourse (OSPD) (Gale et al. 1992), it states that polysemous words tend to exhibit only one sense in a given discourse. For example, consider the noun plant in the excerpt taken from a news article12 shown in Figure 7. The word “engineer” is a good The title of this article is “A Worker Recalls the Chernobyl Disaster”, it was written by ANNA MELNICHUK and published by The Associated Press, on Tuesday, April 25, 2006; 2:13 PM CHAPTER 3. TECHNOLOGY FOUNDATIONS 83 indication that for the first occurrence of plant, the first sense is being used, then with the OSPD heuristic, it can be correctly determined that the first sense also applies to the second occurrence of plant. One seminal WSD paper that exemplifies the use of n-gram based features, the OSPC heuristic and the OSPD heuristic is Yarowsky (1995). This paper introduces a semi-unsupervised system which learns to disambiguate polysemous words from a small number of labelled training data and a large number of unlabelled training data by applying the above two heuristics. Yarowsky’s system starts with a few gold-standard-labelled examples as the seeds of the bootstrapping process. It then trains a classifier based on these seeds, and use it to classify the senses of the unlabelled data. Once this classification is finished, the one-sense-per-discourse heuristic and a probability threshold are then used to select a new set of highly-ranked examples from the classified data. These new examples will then be used to train the next classifier. This cycle of training-analyzing-retraining keeps repeating until either all the data have been labelled or the reanalyzing stage produces no more new highly-ranked examples. Yarowsky’s system is elegant because it does not require large numbers of gold-standard labelled training data but can still achieve results comparable to those that do. The features used in this system are just word collocations of the polysemous words. The evaluation data for Yarowsky’s system consists of 11 polysemous nouns and 1 polysemous verb extracted from a 460 million word corpus,13 with two senses per word. The disambiguation accuracy for all these words ranged between 93.6% to 98.8%, with an average accuracy of 96.5%. However, as all of the test words only have 2 senses, it can be argued that Yarowsky’s disambiguation task may be too easy. Mihalcea and Csomai (2005) describes another general context based WSD system called SenseLearner. Unlike Yarowsky’s system, SenseLearner was semi-supervised system that could handle a much broader number of polysemous words. Even though SenseLearner was a semi-supervised system designed to use as little annotated data as possible, it still requires a certain amount of gold standard training data which comes from Semcor (Miller et al. 1993). SenseLearner makes use of a set of predefined word categories which are groups of words sharing the some common syntactic and semantic properties. Each verb category can handle the disambiguation of a specific group of verbs. With the categories, SenseLearner trains a contextual semantic model and a collocation semantic model which can be used independently for the disambiguation of all the Semcor verbs which are covered by the verb categories. The contextual model treats the lemmas and POS tags of the words immediately surrounding target verb as a bag of words. The collocation model uses the same surrounding words, but treat them as bi-grams, for example, given the target verb say in “the judge said The Yarowsky paper does not mention the name of the corpus. CHAPTER 3. TECHNOLOGY FOUNDATIONS 84 yesterday ...”, the collocations will be judge said and said yesterday. For verbs which are not in Semcor or covered by the word categories, the majority sense as defined by WordNet is used. SenseLearner was evaluated on the English all-words disambiguation tasks of Senseval-2 (Palmer et al. 2001) and it beat the

Evaluation of a domain independent approach to natural language processing for game-like user interfaces

Abstract: Many researchers that develop full software applications in the broad field of natural language processing (NLP) typically implement their single system's components from scratch. While there is nothing wrong with such a methodology from an operational perspective, it typically results in a waste of time. Furthermore, it leads to a substantial diversion of the researchers' efforts from more conceptual and theoretical aspects that could be geared towards the advancement of the state-of-the-art in the field. These main drawbacks call for an implementation approach allowing components' reusability across domains and applications. In that respect, this paper presents an evaluation of a domain independent approach to natural language understanding (NLU) that we have been implementing over the last several years. We have successfully tested and used our approach in three different natural language interfaces to game-like applications, each with its own conversational domains.