Connectionist models have had problems representing and applying general knowledge rules that specifically require variables This variable binding problem has barred them from performing the high-

High-level Inferencing in a Connectionist Network

We describe here the theory behind the language comprehension program Wimp. Wimp understands by first finding paths between the open-class words in a sentence using a marker passing, or spreading-activation, technique. This paper is primarily concerned with the "meaning" (or interpretation) of such paths. We argue that they are best thought of as backbones of proofs that the terms (words) at either end of the paths exist in the story and show how viewing paths in this way naturally leads to the kinds of inferences which are normally thought to characterize "understanding." In a companion paper we show how this interpretation also accomplishes much of the work normally expected in the parsing of language (noun-phrase reference, word-sense disambiguation, etc) so we only briefly touch on this topic here. Wimp has been implemented and works on all of the examples herein.

/pdf/a-neat-theory-of-marker-passing-2umutoymj9.pdf

A neat theory of marker passing

The problem of deciding what was implied by a written text, of "reading between the lines" is the problem of inference. To extract proper inferences from a text requires a great deal of general knowledge on the part of the reader. Past approaches have often postulated an algorithm tuned to process a particular kind of knowledge structure (such as a script, or a plan). An alternative, unified approach is proposed. The algorithm recognizes six very general classes of inference, classes that are not dependent on individual knowledge structures, but instead rely on patterns of connectivity between concepts. The complexity has been effectively shifted from the algorithm to the knowledge base; new kinds of knowledge structures can be added without modifying the algorithm.

/pdf/inference-in-text-understanding-3tx0neo1ta.pdf

Inference in text understanding

This study investigated the effects of first language word-level reading skills on the development of English as a second language (ESL) word-level reading skills. A crosslinguistic analysis indicates that native Arabic and Japanese speakers are likely to encounter different types of ESL word-level reading difficulties. Specifically, native Arab speakers are likely to exhibit difficulties with prelexical ESL word recognition processes, whereas native Japanese speakers are likely to exhibit difficulties with on-line ESL word integration processes that integrate words into phrase/clause structures for comprehension. Results from a lexical decision task showed that a group of Japanese ESL learners had significantly faster and more accurate word recognition skills compared to a proficiency-matched Arab ESL group. In contrast, both groups read words within sentences in a sentence reading task at the same speed, though the Arab ESL group was significantly more accurate in integrating words into larger phrase and clause units and comprehending them than the Japanese ESL group. These results indicate that Arab and Japanese ESL students have different word-level reading difficulties, implicating different learning needs and pedagogical interventions for developing ESL reading proficiency.

https://www.cambridge.org/core/services/aop-cambridge-core/content/view/AE9CA05C51DBFFDA6871CD3B9572223E/S014271640300016Xa.pdf/div-class-title-english-word-recognition-and-word-integration-skills-of-native-arabic-and-japanese-speaking-learners-of-english-as-a-second-language-div.pdf

English Word Recognition and Word Integration Skills of Native Arabic- and Japanese-Speaking Learners of English as a Second Language.

This thesis presents NL-Soar, a detailed computational model of human sentence comprehension that accounts for a broad range of psycholinguistic phenomena. NL-Soar provides in-depth accounts of structural ambiguity resolution, garden path effects, unproblematic ambiguities, parsing breakdown on difficult embeddings, unacceptable embeddings, immediacy of interpretation, and the time course of comprehension. The model explains a variety of both modular and interactive effects, and shows how learning can affect ambiguity resolution behavior. In addition to accounting for the qualitative phenomena surrounding parsing breakdown and garden path effects, NL-Soar explains a wide range of contrasts between garden paths and unproblematic ambiguities, and difficult and acceptable embeddings: the theory has been applied in detail to over 100 types of structures representing theses contrasts, with a success rate of about 90%. The account of real-time immediacy includes predictions about the time course of comprehension and a zero-parameter prediction about the average rate of skilled comprehension. Finally, the theory has been successfully applied to a suggestive range of cross-linguistic examples, including constructions from head-final languages such as Japanese. NL-Soar is based on the Soar theory of cognitive architecture, which provides the underlying control structure, memory structures, and learning mechanism. The basic principles of NL-Soar are a result of applying these architectural mechanisms to the task of efficiently comprehending language in real-time. Soar is more than an implementation language for the system: it plays a central theoretical role and accounts for many of the model''s novel empirical predictions.

/pdf/an-architecturally-based-theory-of-human-sentence-3rl9tfpv1x.pdf

An Architecturally-based Theory of Human Sentence Comprehension

Author(s): Granger, Richard H.; Eiselt, Kurt P.; Holbrook, Jennifer K. | Abstract: The past decade of reseatch in Natural Language Processing has universally recognized that, since natural language input is almost always ambiguous with respect to its pragmatic implications, its syntactic parse, and even its lexical analysis (i.e., choice of correct word-sense for an ambiguous word), processing natural language input requires decisions about word meanings, syntactic structure, and pragmatic inferences. The lexical, syntactic, and pragmatic levels of inferencing are not as disparate as they have often been treated in both psychological and artificial intelligence research. In fact, these three levels of analysis interact to form a joint interpretation of text.ATLAST (A Three-level Language Analysis SysTem) is an implemented integration of human language understanding at the lexical, the syntactic, and the pragmatic levels. For psychological validity, ATLAST is based on results of experiments with human subjects. The ATLAST model uses a new architecture which was developed to incorporate three features: spreading activation memory, two-stage syntax, and parallel processing of syntax and semantics. It is also a new framework within which to interpret and tackle unsolved problems through implementation and experimentation.

/pdf/parsing-with-parallelism-a-spreading-activation-model-of-p7rxnq94mo.pdf

Parsing with parallelism : a spreading-activation model of inference processing during text understanding

Author(s): Eiselt, Kurt Paul | Abstract: Solving the mysteries of human language understanding inevitably requires an answer to the question of how the language understander resolves ambiguity, for human language is certainly ambiguous. But ambiguity leads to choices between possible explanations, and choice opens the door for mistakes. Unless we are willing to believe that the human language understander always makes the correct choice, any explanation of ambiguity resolution must be considered incomplete if it does not also account for recovery from an incorrect decision.This dissertation describes a new approach to lexical ambiguity resolution during sentence understanding which is implemented in a program called ATLAST. Many computational models of natural language understanding have dealt with lexical ambiguity resolution, but ATLAST is one of the few models to address the associated problem of error recovery. ATLAST's ability to recover from an incorrect lexical inference decision stems from its ability to retain unchosen word meanings for a period of time after it selects the apparently context-appropriate meaning of an ambiguous word. The short-term retention of possible lexical inferences permits ATLAST to recover from incorrect decisions without backtracking and reprocessing text, and without keeping a record of possible choices indefinitely.The principle of retention provides a solution to the problem of error recovery which is compatible with current psycholinguistic theories of lexical disambiguation. Furthermore, the existence of some form of retention in lexical disambiguation is supported by the results of experiments with human subjects. This dissertation includes a discussion of these results and speculation on how the principle of retention might be extended to account for recovery from erroneous higher-level inference decisions.

Inference processing and error recovery in sentence understanding

Is the human language understander a collection of modular processes operating with relative autonomy, or is it a single integrated process? This ongoing debate has polarized the language processing community, with two fundamentally different types of model posited, and with each camp concluding that the other is wrong. One camp puts forth a model with separate processors and distinct knowledge sources to explain one body of data, and the other proposes a model with a single processor and a homogeneous, monolithic knowledge source to explain the other body of data. In this paper we argue that a hybrid approach which combines a unified processor with separate knowledge sources provides an explanation of both bodies of data, and we demonstrate the feasibility of this approach with the computational model called COMPERE. We believe that this approach brings the language processing community significantly closer to offering humanlike language processing systems.

Having your cake and eating it too: autonomy and interaction in a model of sentence processing

The development of models of human sentence processing has traditionally followed one of two paths. Either the model posited a sequence of processing modules, each with its own task-specific knowledge (e.g., syntax and semantics), or it posited a single processor utilizing different types of knowledge inextricably integrated into a monolithic knowledge base. Our previous work in modeling the sentence processor resulted in a model in which different processing modules used separate knowledge sources but operated in parallel to arrive at the interpretation of a sentence. One highlight of this model is that it offered an explanation of how the sentence processor might recover from an error in choosing the meaning of an ambiguous word. Recent experimental work by Laurie Stowe strongly suggests that the human sentence processor deals with syntactic error recovery using a mechanism very much like that proposed by our model of semantic error recovery. Another way to interpret Stowe's finding is this: the human sentence processor consists of a single unified processing module utilizing multiple independent knowledge sources in parallel. A sentence processor built upon this architecture should at times exhibit behavior associated with modular approaches, and at other times act like an integrated system. In this paper we explore some of these ideas via a prototype computational model of sentence processing called COMPERE, and propose a set of psychological experiments for testing our theories.

A Unified Process Model of Syntactic and Semantic Error Recovery in Sentence Understanding.

Psychological investigations have led to considerable insight into the working of the human language comprehension system. In this article, we look at a set of principles derived from psychological findings to argue for a particular organization of linguistic knowledge along with a particular processing strategy and present a computational model of sentence processing based on those principles. Many studies have shown that human sentence comprehension is an incremental and interactive process in which semantic and other higher-level information interacts with syntactic information to make informed commitments as early as possible at a local ambiguity. Early commitments may be made by using top-down guidance from knowledge of different types, each of which must be applicable independently of others. Further evidence from studies of error recovery and delayed decisions points toward an arbitration mechanism for combining syntactic and semantic information in resolving ambiguities. In order to account for all of the above, we propose that all types of linguistic knowledge must be represented in a common form but must be separable so that they can be applied independently of each other and integrated at processing time by the arbitrator. We present such a uniform representation and a computational model called COMPERE based on the representation and the processing strategy.

Kurt P. Eiselt

Papers

Parsing with parallelism : a spreading-activation model of inference processing during text understanding

Inference processing and error recovery in sentence understanding

Having your cake and eating it too: autonomy and interaction in a model of sentence processing

A Unified Process Model of Syntactic and Semantic Error Recovery in Sentence Understanding.

Uniform Representations for Syntax-Semantics Arbitration.