Connectionist models have had problems representing and applying general knowledge rules that specifically require variables This variable binding problem has barred them from performing the high-

High-level Inferencing in a Connectionist Network

We describe here the theory behind the language comprehension program Wimp. Wimp understands by first finding paths between the open-class words in a sentence using a marker passing, or spreading-activation, technique. This paper is primarily concerned with the "meaning" (or interpretation) of such paths. We argue that they are best thought of as backbones of proofs that the terms (words) at either end of the paths exist in the story and show how viewing paths in this way naturally leads to the kinds of inferences which are normally thought to characterize "understanding." In a companion paper we show how this interpretation also accomplishes much of the work normally expected in the parsing of language (noun-phrase reference, word-sense disambiguation, etc) so we only briefly touch on this topic here. Wimp has been implemented and works on all of the examples herein.

/pdf/a-neat-theory-of-marker-passing-2umutoymj9.pdf

A neat theory of marker passing

The problem of deciding what was implied by a written text, of "reading between the lines" is the problem of inference. To extract proper inferences from a text requires a great deal of general knowledge on the part of the reader. Past approaches have often postulated an algorithm tuned to process a particular kind of knowledge structure (such as a script, or a plan). An alternative, unified approach is proposed. The algorithm recognizes six very general classes of inference, classes that are not dependent on individual knowledge structures, but instead rely on patterns of connectivity between concepts. The complexity has been effectively shifted from the algorithm to the knowledge base; new kinds of knowledge structures can be added without modifying the algorithm.

/pdf/inference-in-text-understanding-3tx0neo1ta.pdf

Inference in text understanding

This study investigated the effects of first language word-level reading skills on the development of English as a second language (ESL) word-level reading skills. A crosslinguistic analysis indicates that native Arabic and Japanese speakers are likely to encounter different types of ESL word-level reading difficulties. Specifically, native Arab speakers are likely to exhibit difficulties with prelexical ESL word recognition processes, whereas native Japanese speakers are likely to exhibit difficulties with on-line ESL word integration processes that integrate words into phrase/clause structures for comprehension. Results from a lexical decision task showed that a group of Japanese ESL learners had significantly faster and more accurate word recognition skills compared to a proficiency-matched Arab ESL group. In contrast, both groups read words within sentences in a sentence reading task at the same speed, though the Arab ESL group was significantly more accurate in integrating words into larger phrase and clause units and comprehending them than the Japanese ESL group. These results indicate that Arab and Japanese ESL students have different word-level reading difficulties, implicating different learning needs and pedagogical interventions for developing ESL reading proficiency.

https://www.cambridge.org/core/services/aop-cambridge-core/content/view/AE9CA05C51DBFFDA6871CD3B9572223E/S014271640300016Xa.pdf/div-class-title-english-word-recognition-and-word-integration-skills-of-native-arabic-and-japanese-speaking-learners-of-english-as-a-second-language-div.pdf

English Word Recognition and Word Integration Skills of Native Arabic- and Japanese-Speaking Learners of English as a Second Language.

This thesis presents NL-Soar, a detailed computational model of human sentence comprehension that accounts for a broad range of psycholinguistic phenomena. NL-Soar provides in-depth accounts of structural ambiguity resolution, garden path effects, unproblematic ambiguities, parsing breakdown on difficult embeddings, unacceptable embeddings, immediacy of interpretation, and the time course of comprehension. The model explains a variety of both modular and interactive effects, and shows how learning can affect ambiguity resolution behavior. In addition to accounting for the qualitative phenomena surrounding parsing breakdown and garden path effects, NL-Soar explains a wide range of contrasts between garden paths and unproblematic ambiguities, and difficult and acceptable embeddings: the theory has been applied in detail to over 100 types of structures representing theses contrasts, with a success rate of about 90%. The account of real-time immediacy includes predictions about the time course of comprehension and a zero-parameter prediction about the average rate of skilled comprehension. Finally, the theory has been successfully applied to a suggestive range of cross-linguistic examples, including constructions from head-final languages such as Japanese. NL-Soar is based on the Soar theory of cognitive architecture, which provides the underlying control structure, memory structures, and learning mechanism. The basic principles of NL-Soar are a result of applying these architectural mechanisms to the task of efficiently comprehending language in real-time. Soar is more than an implementation language for the system: it plays a central theoretical role and accounts for many of the model''s novel empirical predictions.

/pdf/an-architecturally-based-theory-of-human-sentence-3rl9tfpv1x.pdf

An Architecturally-based Theory of Human Sentence Comprehension

Author(s): Granger, Richard H.; Eiselt, Kurt P.; Holbrook, Jennifer K. | Abstract: The past decade of reseatch in Natural Language Processing has universally recognized that, since natural language input is almost always ambiguous with respect to its pragmatic implications, its syntactic parse, and even its lexical analysis (i.e., choice of correct word-sense for an ambiguous word), processing natural language input requires decisions about word meanings, syntactic structure, and pragmatic inferences. The lexical, syntactic, and pragmatic levels of inferencing are not as disparate as they have often been treated in both psychological and artificial intelligence research. In fact, these three levels of analysis interact to form a joint interpretation of text.ATLAST (A Three-level Language Analysis SysTem) is an implemented integration of human language understanding at the lexical, the syntactic, and the pragmatic levels. For psychological validity, ATLAST is based on results of experiments with human subjects. The ATLAST model uses a new architecture which was developed to incorporate three features: spreading activation memory, two-stage syntax, and parallel processing of syntax and semantics. It is also a new framework within which to interpret and tackle unsolved problems through implementation and experimentation.

/pdf/parsing-with-parallelism-a-spreading-activation-model-of-p7rxnq94mo.pdf

Parsing with parallelism : a spreading-activation model of inference processing during text understanding

Is the human language understander a collection of modular processes operating with relative autonomy, or is it a single integrated process? This ongoing debate has polarized the language processing community, with two fundamentally different types of model posited, and with each camp concluding that the other is wrong. One camp puts forth a model with separate processors and distinct knowledge sources to explain one body of data, and the other proposes a model with a single processor and a homogeneous, monolithic knowledge source to explain the other body of data. In this paper we argue that a hybrid approach which combines a unified processor with separate knowledge sources provides an explanation of both bodies of data, and we demonstrate the feasibility of this approach with the computational model called COMPERE. We believe that this approach brings the language processing community significantly closer to offering humanlike language processing systems.

Having your cake and eating it too: autonomy and interaction in a model of sentence processing

The development of models of human sentence processing has traditionally followed one of two paths. Either the model posited a sequence of processing modules, each with its own task-specific knowledge (e.g., syntax and semantics), or it posited a single processor utilizing different types of knowledge inextricably integrated into a monolithic knowledge base. Our previous work in modeling the sentence processor resulted in a model in which different processing modules used separate knowledge sources but operated in parallel to arrive at the interpretation of a sentence. One highlight of this model is that it offered an explanation of how the sentence processor might recover from an error in choosing the meaning of an ambiguous word. Recent experimental work by Laurie Stowe strongly suggests that the human sentence processor deals with syntactic error recovery using a mechanism very much like that proposed by our model of semantic error recovery. Another way to interpret Stowe's finding is this: the human sentence processor consists of a single unified processing module utilizing multiple independent knowledge sources in parallel. A sentence processor built upon this architecture should at times exhibit behavior associated with modular approaches, and at other times act like an integrated system. In this paper we explore some of these ideas via a prototype computational model of sentence processing called COMPERE, and propose a set of psychological experiments for testing our theories.

A Unified Process Model of Syntactic and Semantic Error Recovery in Sentence Understanding.

Sentence processing in understanding: interaction and integration of knowledge sources

Is the human language understander a collection of modular processes operating with relative autonomy, or is it a single integrated process? This ongoing debate has polarized the language processing community, with two fundamentally different types of model posited, and with each camp concluding that the other is wrong. One camp puts forth a model with separate processors and distinct knowledge sources to explain one body of data, and the other proposes a model with a single processor and a homogeneous, monolithic knowledge source to explain the other body of data. In this paper we argue that a hybrid approach which combines a unified processor with separate knowledge sources provides an explanation of both bodies of data, and we demonstrate the feasibility of this approach with the computational model called COMPERE. We believe that this approach brings the language processing community significantly closer to offering human-like language processing systems.

Jennifer K. Holbrook

Papers

Parsing with parallelism : a spreading-activation model of inference processing during text understanding

Having your cake and eating it too: autonomy and interaction in a model of sentence processing

A Unified Process Model of Syntactic and Semantic Error Recovery in Sentence Understanding.

Sentence processing in understanding: interaction and integration of knowledge sources

Having Your Cake and Eating It Too: Autonomy and Interaction in a Model of Sentence Processing