A Neurolinguistic Model of Grammatical Construction Processing
Summary (5 min read)
INTRODUCTION
- One of the long-term quests of cognitive neuroscience has been to link functional aspects of language processing to its underlying neurophysiology, that is, to understand how neural mechanisms allow the mapping of the surface structure of a sentence onto a conceptual representation of its meaning.
- The current research takes this multidisciplinary approach within the theoretical framework associated with grammatical constructions.
- These mappings vary along a continuum of complexity.
- The universal grammar is there from the outset, and the challenge is to understand how it got there.
- This issue is partially addressed in the current research.
Theoretical Framework for Grammatical Constructions
- Figure 1C illustrates a more complex construction that contains an embedded relative phrase.
- Part of the response to this lies in the specific ways in which function and content words are structurally organized in distinct sentence types.
- The general idea here is that these constructions are templates into which a variety of open class elements (nouns, verbs, etc.) can be inserted in order to express novel meanings.
- Several major issues can be raised with respect to this characterization.
Implementing Grammatical Constructions in a Neurocomputational Model
- Determining the construction type for a given sentence consists in analyzing or recoding the sentence as a sequence of open class and closed class elements, and then performing sequence recognition on this recoded sequence.
- The reordering in the form–meaning mapping in Figure 1B can be characterized as the abstract structure ABCD- Dominey, Hoen, and Inui 2089 BACD where A-D represent variable slots.
- Given the initial validation of this model in the neurophysiological and neuropsychological studies cited above, the authors can now proceed with a more detailed study of how this can contribute to an understanding of the possible implementation of grammatical constructions, as illustrated in Figure 2.
- As the sentence is processed word by word, a process of lexical categorization identifies open and closed class words.
- In particular, the authors propose that the formation of the ConstructionIndex as a neural pattern of activity will rely on sequence processing in recurrent cortical networks, and that the retrieval of the FormToMeaning component will rely on a corticostriatal associative memory.
Goals of the Current Study
- The authors have previously demonstrated that the model in Figure 2 can learn a variety of constructions that can then be used in different language interaction contexts (Dominey & Boucher, 2005a, b; Dominey, 2000).
- The proposed model of grammatical construction processing should accommodate different types of languages.
- That is, rather than looking for patterns at the level of the entire sentence or ConstructionIndex, the model could apply exactly the same mechanisms to identify subpatterns within the sentence.
- This will decompose the sentence into the principal and the relative phrases, with the outcome that both will more likely correspond to existing constructions .
- This type of segmentation approach has been explored by Miikkulainen (1996), discussed below.
METHODS
- In the first set of experiments (Experiment 1A–C) for each of the three languages English, Japanese, and French, respectively, the authors will expose the model to a set of <sentence, meaning> pairs in a training phase.
- The authors then validate that the model has learned these sentences by presenting the sentences alone and comparing the 2092 Journal of Cognitive Neuroscience Volume 18, Number 12 generated meanings with the actual meanings.
- The authors compare the meaning generated for each sentence with its actual meaning.
- In the second set of experiments (Experiment 2A–D) the authors attempt to determine whether pattern matching within a sentence can be used to extract relative phrases.
- The net result is the capability to handle structurally novel sentences by decomposing them into their phrasal components in a construction grammar setting.
Model Overview
- The model is trained on <sentence, meaning> pairs and then is tested with new sentences to generate the appropriate meanings.
- Open class words are directed to the open class array (OCA), and are represented by an empty slot in the ConstructionIndex.
- Given the FormToMeaning mapping for the current sentence, the authors can now store it in the ConstructionInventory, associated with the corresponding ConstructionIndex for that sentence.
- Now let us consider how the system could accommodate complex sentences as in Figure 1C (and the relative and conjoined sentences in Table 4).
- Thus, the authors now present the generalized compositional solution.
Processing Compositional Structure of Relative Phrases
- As described above, the model will account for sentences with relative phrases by treating the entire sentence as a distinct construction, mapping the open class elements onto two distinct instances of the SceneEventArray as indicated in Figure 1C.
- The pseudocode in Table 2 describes the process for extracting relative phrases.
- Likewise, the authors demonstrated a hybrid neural network’s ability to fill working memory ‘‘slots’’ with specific sequence elements and then modulate the retrieval of the slot contents to transform these variable sequences for abstract sequence processing (Dominey & Ramus, 2000; Dominey et al., 1998) and syntactic comprehension (Dominey et al., 2003; Dominey, 2002).
- For each of these test sentences, the model successfully identified the corresponding construction, applied the corresponding sentence-to-meaning mapping, and generated the correct meaning.
- An example of this word order flexibility of Japanese with respect to English is illustrated in Table 5 with the English passive ditransitive forms that can be.
Language English French Japanese
- Dominey, Hoen, and Inui 2095 expressed in four different common manners in Japanese (Constructions 9–12).
- Indeed, the model successfully discriminates between all of the 26 construction types based on the ConstructionIndex unique to each construction type, and associates the correct FormToMeaning mapping with each of them.
- This demonstration with Japanese is an important validation that at least for this subset of constructions, the construction-based model is applicable both to fixed word order languages such as English, as well as free word order languages such as Japanese.
- This also provides further validation for the proposal of Bates et al. (1982) and MacWhinney (1982) that thematic roles are indicated by a constellation of cues including grammatical markers and word order.
- As for the English and Japanese studies, the model was trained on a set of French sentences (illustrated in Table 6) that were paired with their meanings.
Example Sentences and Meanings Grammatical Constructions
- Push(block, cylinder) Verb(agent, object) 2. The cylinder was pushed by the block.
- The block gave the cylinder to the moon.
- Give(block, cylinder, moon) Verb(agent, object, recipient) 4. (Dat passive) Give(block, cylinder, moon) Action1(agent1, object2, recipient3).
Dual-event Relative Constructions
- The block that pushed the cylinder touched the moon.
- Agent1 that verb1ed object2 verb2ed object3.
- The cylinder that was pushed by the block gave the cat to the dog.
Dual-event Conjoined Constructions
- The block pushed the cylinder and the moon.
- The moon and the block were given to the cylinder by the cat.
- In the previous experiments, for each of the three languages, a limited set of sentences including those with relative phrases was learned, based on a noncompositional mechanism.
- For these sentences, the verbs in the main and relative phrases in the OpenClassArray became associated with the event components of two distinct copies of the SceneEventArray, corresponding to constructions like that in Figure 1C.
The chief pushed the boy and the block.
- For the English experiments, the model was first trained on five sentence types corresponding to the active and passive transitive and dative constructions (1–4 in Table 4), and the relative ‘‘The boy the dog hit,’’ along with their corresponding meaning representations.
- Similarly for Sentence 2, application of the phrase structure extraction yielded three phrases: ‘‘The dog bit the man,’’ ‘‘The cat hit the dog,’’ and ‘‘The boy saw the cat,’’ for each of which the corresponding meanings were extracted based on the previously learned constructions.
- Cat wo relverb boy ga dog ni-yotte osareta Girl-ga girl-wo bit dog-ga bit cat-wo chased boywo saw.
Psychological Reality of the Model
- The first principle inherent in the model is that instead of representing <sentence, meaning> mappings in terms of a generative grammar, they are represented directly in a structured inventory of grammatical constructions that are nothing more than these mappings (Tomasello, 2003; Goldberg, 1995).
- The essential innate capabilities the authors postulate are mechanisms for (1) abstract (variable based) structure mapping and for 2100 Journal of Cognitive Neuroscience Volume 18, Number 12 (2) pattern-based embedded structure processing.
- What has been lacking, however, is a specification of how grammatical constructions can be combined in order to generate new constructions on the fly, using these same sequence processing mechanisms.
- This approach should extend to accommodate more generalized relative phrase structure, as well as prepositional and other categories of embedded phrases.
- Thus, when variations in word order are used in order to place the pragmatic focus on a particular item, as in the Japanese examples 9–12 in Table 5, distinct constructions will be employed to capture these differences.
Division of Labor in the Construction Inventory
- Construction grammar assumes no strict division between the lexicon and syntax in that both lexical and syntactic constructions pair form with meaning (see Goldberg, 1995).
- Subsequent developmental categorization of an open class category of concrete nouns would allow verb-based constructions such as ‘‘Gimme X.’’.
- A pertinent account of such distributional analysis is presented in Cartwright and Brent (1997).
- The current model represents what the authors consider a necessary refinement that is wholly consistent with construction grammar, and indeed provides insight into a consistent explanation for compositionality.
Temporal Dynamics and Resolution of Detail
- The theoretical foundation of the model is that the sentence-to-meaning mapping is performed based on the configuration of closed class elements (the ConstructionIndex) in the sentence (consistent with Bates et al., 1982; MacWhinney, 1982).
- In the current implementation, to reduce processing complexity, the authors wait until the entire sentence has been seen before extracting the corresponding mapping.
- Miyamoto (2002) observed that in some cases, marking on the noun phrases indicates phrase boundaries prior to the arrival of the verb, so that the detection of the relative phrase structure can actually precede the nonfinal verb (Miyamoto, 2002).
- In general, if delay is decreased, then the required information must come from an alternative source (such as the double o constraint).
- Thus, in building up the ConstructionIndex incrementally, when such information is not available then the incremental processing will induce temporary failures including well-documented garden path effects.
Underlying Neurophysiology
- Part of the importance of the current approach is that the underlying primitive functions that provide this construction processing capability can be linked in a plausible manner to known neurophysiological systems.
- The resulting pattern of cortical activity projects to the caudate nucleus of the striatum, and via these modifiable corticostriatal synapses, retrieves the appropriate FormToMeaning mapping that is then implemented in the frontal transformation processing network that includes BA 44.
- The authors have simulated how a recurrent cortical network can encode the sequential structure of closed and open class words in a sentence, corresponding to the ConstructionIndex, in order to retrieve the appropriate FormToMeaning mapping based on corticostriatal associative memory (Dominey et al., 2003; Dominey, 2002, 2005a, 2005b).
- This result is consistent with previous fMRI studies that showed activation of these areas in sentence comprehension tasks (e.g., Kuperberg et al., 2003; Baumgaertner, Weiller, & Büchel, 2002), particularly for sequential or structural processing aspects in sentences (Newman, Just, Keller, Roth, & Carpenter, 2003; Dapretto & Bookheimer, 1999; Kang, Constable, Gore, & Avrutin, 1999; Inui et al., 1998).
- This yielded a profile of sentence comprehension performance highly correlated (R2 = .89) with that of the nine agrammatic patients the authors studied (Dominey, 2002).
Extraction of Relative Phrase Structure
- Part of the novelty of their approach is that the extraction of relative phrase structure relies on these same capabilities.
- Each extracted phrase is processed by the construction model, and then the remainder of the original sequence is processed.
- At the extreme, this is related to lexically driven approaches including that of Vosse and Kempen (2000), lexical functional grammar (LFG; Bresnan, 2001), and head-driven phrase structure grammar (HPSG; Pollard & Sag, 1994), in which lexical entries include type descriptions that form a hierarchy that, combined with grammatical rules, allows for unificationbased parsing.
- One source of variability is the structure of the noun phrase.
Acknowledgments
- This study was supported in part by the HFSP Organization, the French ACI NIC and TTT Projects, LAFMI, and the ESF.
- Reprint requests should be sent to Peter Ford Dominey, Institut des Sciences Cognitives, CNRS UMR 5015, 67, Boulevard Pinel, 69675 Bron Cedex, France, or via e-mail: dominey@isc.cnrs.fr and www.isc.cnrs.fr/.
Did you find this useful? Give us your feedback
Citations
9 citations
Cites background from "A Neurolinguistic Model of Grammati..."
...ResPars proposes to model how the human brain processes sentences and is inspired from several studies in neuroscience [12], [17]....
[...]
8 citations
Cites background from "A Neurolinguistic Model of Grammati..."
...Such approaches are often more inspired by brain mechanisms involved in language processing [10], [12]....
[...]
...ResPars proposes to model how the human brain processes sentences and is inspired from several studies in neuroscience [10], [12]....
[...]
8 citations
Cites background from "A Neurolinguistic Model of Grammati..."
...Based on an analysis of the resulting contact sequence, we could identify for each of five different event types a unique contact sequence [9]....
[...]
...In the specific task we pose, for a given video sequence of a human operator performing actions with different objects (see Fig....
[...]
...Interestingly, while this is a severe limitation in the long term, it appears to be a developmental step for human children (of about 2 years of age) on their way to more adult-like generative performance [4,42]....
[...]
8 citations
Additional excerpts
...The neural parser proposes to model how the human brain processes sentences and is inspired from several studies in neuroscience [5], [16]....
[...]
8 citations
References
3,600 citations
3,382 citations
2,807 citations
1,760 citations
1,493 citations