Statistical approaches to processing natural language text have become dominant in recent years This foundational text is the first comprehensive introduction to statistical natural language processing (NLP) to appear The book contains all the theory and algorithms needed for building NLP tools It provides broad but rigorous coverage of mathematical and linguistic foundations, as well as detailed discussion of statistical methods, allowing students and researchers to construct their own implementations The book covers collocation finding, word sense disambiguation, probabilistic parsing, information retrieval, and other applications

/pdf/foundations-of-statistical-natural-language-processing-1ou5e6c2q6.pdf

Foundations of Statistical Natural Language Processing

: As a result of this grant, the researchers have now published oil CDROM a corpus of over 4 million words of running text annotated with part-of- speech (POS) tags, with over 3 million words of that material assigned skeletal grammatical structure This material now includes a fully hand-parsed version of the classic Brown corpus About one half of the papers at the ACL Workshop on Using Large Text Corpora this past summer were based on the materials generated by this grant

/pdf/building-a-large-annotated-corpus-of-english-the-penn-4tcny82kaq.pdf

Building a large annotated corpus of English: the penn treebank

Brooks, R.A., Intelligence without representation, Artificial Intelligence 47 (1991) 139159. Artificial intelligence research has foundered on the issue of representation. When intelligence is approached in an incremental manner, with strict reliance on interfacing to the real world through perception and action, reliance on representation disappears. In this paper we outline our approach to incrementally building complete intelligent Creatures. The fundamental decomposition of the intelligent system is not into independent information processing units which must interface with each other via representations. Instead, the intelligent system is decomposed into independent and parallel activity producers which all interface directly to the world through perception and action, rather than interface to each other particularly much. The notions of central and peripheral systems evaporateeverything is both central and peripheral. Based on these principles we have built a very successful series of mobile robots which operate without supervision as Creatures in standard office environments.

Intelligence without Representation

From the Publisher:
This book takes an empirical approach to language processing, based on applying statistical and other machine-learning algorithms to large corpora.Methodology boxes are included in each chapter. Each chapter is built around one or more worked examples to demonstrate the main idea of the chapter. Covers the fundamental algorithms of various fields, whether originally proposed for spoken or written language to demonstrate how the same algorithm can be used for speech recognition and word-sense disambiguation. Emphasis on web and other practical applications. Emphasis on scientific evaluation. Useful as a reference for professionals in any of the areas of speech and language processing.

/pdf/speech-and-language-processing-an-introduction-to-natural-atzaecqfem.pdf

Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition

http://people.csail.mit.edu/brooks/papers/representation.pdf

Intelligence without representation

This paper describes the RUS framework for natural language processing, in which a parser incorporating a substantial ATN grammar for English interacts with a semantic interpreter to simultaneously parse and interpret input. The structure of that interaction is discussed, including the roles played by syntactic and semantic knowledge. Several implementations of the RUS framework are currently in use, sharing the same grammar, but differing in the form of their semantic component. One of these, the PSI-KLONE system, is based on a general object-centered knowledge representation system, called KL-ONE. The operation of PSI-KLONE is described, including its use of KL-ONE to support a general inference process called "incremental description refinement." The last section of the paper discusses several important criteria for knowledge representation systems to be used in syntactic and semantic processing.

https://aaaipress.org/Papers/AAAI/1980/AAAI80-090.pdf

Knowledge representation for syntactic/semantic processing

We present a natural language interface system which is based entirely on trained statistical models. The system consists of three stages of processing: parsing, semantic interpretation, and discourse. Each of these stages is modeled as a statistical process. The models are fully integrated, resulting in an end-to-end system that maps input utterances into meaning representation frames.

A Fully Statistical Approach to Natural Language Interfaces

We describe and evaluate hidden understanding models, a statistical learning approach to natural language understanding. Given a string of words, hidden understanding models determine the most likely meaning for the string. We discuss 1) the problem of representing meaning in this framework, 2) the structure of the statistical model, 3) the process of training the model, and 4) the process of understanding using the model. Finally, we give experimental results, including results on an ARPA evaluation.

Hidden understanding models of natural language

Many different problems can be represented as graphs displayed in the form of node--link diagrams. However, when a graph is large it becomes visually uninterpretable because of the tangle of links. We describe a set of techniques that use motion in an interactive interface to provide effective access to larger graphs. Touching a node with the mouse cursor causes that node and the subgraph of closely connected nodes to oscillate. We argue from perceptual principles that this should be a more effective way of interactively highlighting a subgraph than more conventional static methods. The MEGraph system was developed to gain experience with different forms of motion highlighting. Based on positive feedback, three experiments were carried out to evaluate the effectiveness of motion highlighting for specific tasks. All three showed motion to be more effective than static highlighting, both in increasing the speed of response for a variety of visual queries, and in reducing errors. We argue that motion highlighting can be a valuable technique in applications that require users to understand large graphs.

/pdf/motion-to-support-rapid-interactive-queries-on-node-link-53r608krig.pdf

Motion to support rapid interactive queries on node--link diagrams

For reasons of clarity, a typical node-link diagram statically displayed on paper or a computer screen contains fewer than 30 nodes. However, many problems would benefit if far more complex information could be diagrammed. Following Munzner et al., we suggest that with interactive diagrams this may be possible. We describe an interactive technique whereby a subset of a larger network diagram is highlighted by being set into oscillatory motion when a node is selected with a mouse. The subset is determined by a breadth first search of the underlying graph starting from the selected node. This technique is designed to support visual queries on moderately large node-link diagrams containing up to a few thousand nodes. An experimental evaluation was carried out with networks having 32, 100, 320, 1000, and 3200 nodes respectively, and with four highlighting techniques: static highlighting, motion highlighting, static + motion highlighting, and none. The results show that the interactive highlighting methods support rapid visual queries of nodes in close topological proximity to one another, even for the largest diagrams tested. Without highlighting, error rates were high even for the smallest network that was evaluated. Motion highlighting and static highlighting were equally effective. A second experiment was carried out to evaluate methods for showing two subsets of a larger network simultaneously in such a way that both are clearly distinct. The specific task was to determine if the two subsets had nodes in common. The results showed that this task could be performed rapidly and with few errors if one subset was highlighted using motion and the other was highlighted using a static technique. We discuss the implications for information visualization.

/pdf/supporting-visual-queries-on-medium-sized-node-link-diagrams-4ira6tqmhp.pdf

Robert J. Bobrow

Papers

Knowledge representation for syntactic/semantic processing

A Fully Statistical Approach to Natural Language Interfaces

Hidden understanding models of natural language

Motion to support rapid interactive queries on node--link diagrams

Supporting visual queries on medium-sized node-link diagrams