scispace - formally typeset

Journal ArticleDOI

A Neurolinguistic Model of Grammatical Construction Processing

01 Dec 2006-Journal of Cognitive Neuroscience (MIT Press)-Vol. 18, Iss: 12, pp 2088-2107

TL;DR: The objective of the current research is to outline a functional description of grammatical construction processing based on principles of psycholinguistics, develop a model of how these functions can be implemented in human neurophysiology, and demonstrate the feasibility of the resulting model in processing languages of typologically diverse natures, that is, English, French, and Japanese.

AbstractOne of the functions of everyday human language is to communicate meaning. Thus, when one hears or reads the sentence, “John gave a book to Mary,” some aspect of an event concerning the transfer of possession of a book from John to Mary is (hopefully) transmitted. One theoretical approach to language referred to as construction grammar emphasizes this link between sentence structure and meaning in the form of grammatical constructions. The objective of the current research is to (1) outline a functional description of grammatical construction processing based on principles of psycholinguistics, (2) develop a model of how these functions can be implemented in human neurophysiology, and then (3) demonstrate the feasibility of the resulting model in processing languages of typologically diverse natures, that is, English, French, and Japanese. In this context, particular interest will be directed toward the processing of novel compositional structure of relative phrases. The simulation results are discussed in the context of recent neurophysiological studies of language processing.

Topics: Construction grammar (60%), Grammatical construction (59%), Grammar (57%), Psycholinguistics (57%), Sentence (56%)

Summary (5 min read)

INTRODUCTION

  • One of the long-term quests of cognitive neuroscience has been to link functional aspects of language processing to its underlying neurophysiology, that is, to understand how neural mechanisms allow the mapping of the surface structure of a sentence onto a conceptual representation of its meaning.
  • The current research takes this multidisciplinary approach within the theoretical framework associated with grammatical constructions.
  • These mappings vary along a continuum of complexity.
  • The universal grammar is there from the outset, and the challenge is to understand how it got there.
  • This issue is partially addressed in the current research.

Theoretical Framework for Grammatical Constructions

  • Figure 1C illustrates a more complex construction that contains an embedded relative phrase.
  • Part of the response to this lies in the specific ways in which function and content words are structurally organized in distinct sentence types.
  • The general idea here is that these constructions are templates into which a variety of open class elements (nouns, verbs, etc.) can be inserted in order to express novel meanings.
  • Several major issues can be raised with respect to this characterization.

Implementing Grammatical Constructions in a Neurocomputational Model

  • Determining the construction type for a given sentence consists in analyzing or recoding the sentence as a sequence of open class and closed class elements, and then performing sequence recognition on this recoded sequence.
  • The reordering in the form–meaning mapping in Figure 1B can be characterized as the abstract structure ABCD- Dominey, Hoen, and Inui 2089 BACD where A-D represent variable slots.
  • Given the initial validation of this model in the neurophysiological and neuropsychological studies cited above, the authors can now proceed with a more detailed study of how this can contribute to an understanding of the possible implementation of grammatical constructions, as illustrated in Figure 2.
  • As the sentence is processed word by word, a process of lexical categorization identifies open and closed class words.
  • In particular, the authors propose that the formation of the ConstructionIndex as a neural pattern of activity will rely on sequence processing in recurrent cortical networks, and that the retrieval of the FormToMeaning component will rely on a corticostriatal associative memory.

Goals of the Current Study

  • The authors have previously demonstrated that the model in Figure 2 can learn a variety of constructions that can then be used in different language interaction contexts (Dominey & Boucher, 2005a, b; Dominey, 2000).
  • The proposed model of grammatical construction processing should accommodate different types of languages.
  • That is, rather than looking for patterns at the level of the entire sentence or ConstructionIndex, the model could apply exactly the same mechanisms to identify subpatterns within the sentence.
  • This will decompose the sentence into the principal and the relative phrases, with the outcome that both will more likely correspond to existing constructions .
  • This type of segmentation approach has been explored by Miikkulainen (1996), discussed below.

METHODS

  • In the first set of experiments (Experiment 1A–C) for each of the three languages English, Japanese, and French, respectively, the authors will expose the model to a set of <sentence, meaning> pairs in a training phase.
  • The authors then validate that the model has learned these sentences by presenting the sentences alone and comparing the 2092 Journal of Cognitive Neuroscience Volume 18, Number 12 generated meanings with the actual meanings.
  • The authors compare the meaning generated for each sentence with its actual meaning.
  • In the second set of experiments (Experiment 2A–D) the authors attempt to determine whether pattern matching within a sentence can be used to extract relative phrases.
  • The net result is the capability to handle structurally novel sentences by decomposing them into their phrasal components in a construction grammar setting.

Model Overview

  • The model is trained on <sentence, meaning> pairs and then is tested with new sentences to generate the appropriate meanings.
  • Open class words are directed to the open class array (OCA), and are represented by an empty slot in the ConstructionIndex.
  • Given the FormToMeaning mapping for the current sentence, the authors can now store it in the ConstructionInventory, associated with the corresponding ConstructionIndex for that sentence.
  • Now let us consider how the system could accommodate complex sentences as in Figure 1C (and the relative and conjoined sentences in Table 4).
  • Thus, the authors now present the generalized compositional solution.

Processing Compositional Structure of Relative Phrases

  • As described above, the model will account for sentences with relative phrases by treating the entire sentence as a distinct construction, mapping the open class elements onto two distinct instances of the SceneEventArray as indicated in Figure 1C.
  • The pseudocode in Table 2 describes the process for extracting relative phrases.
  • Likewise, the authors demonstrated a hybrid neural network’s ability to fill working memory ‘‘slots’’ with specific sequence elements and then modulate the retrieval of the slot contents to transform these variable sequences for abstract sequence processing (Dominey & Ramus, 2000; Dominey et al., 1998) and syntactic comprehension (Dominey et al., 2003; Dominey, 2002).
  • For each of these test sentences, the model successfully identified the corresponding construction, applied the corresponding sentence-to-meaning mapping, and generated the correct meaning.
  • An example of this word order flexibility of Japanese with respect to English is illustrated in Table 5 with the English passive ditransitive forms that can be.

Language English French Japanese

  • Dominey, Hoen, and Inui 2095 expressed in four different common manners in Japanese (Constructions 9–12).
  • Indeed, the model successfully discriminates between all of the 26 construction types based on the ConstructionIndex unique to each construction type, and associates the correct FormToMeaning mapping with each of them.
  • This demonstration with Japanese is an important validation that at least for this subset of constructions, the construction-based model is applicable both to fixed word order languages such as English, as well as free word order languages such as Japanese.
  • This also provides further validation for the proposal of Bates et al. (1982) and MacWhinney (1982) that thematic roles are indicated by a constellation of cues including grammatical markers and word order.
  • As for the English and Japanese studies, the model was trained on a set of French sentences (illustrated in Table 6) that were paired with their meanings.

Example Sentences and Meanings Grammatical Constructions

  • Push(block, cylinder) Verb(agent, object) 2. The cylinder was pushed by the block.
  • The block gave the cylinder to the moon.
  • Give(block, cylinder, moon) Verb(agent, object, recipient) 4. (Dat passive) Give(block, cylinder, moon) Action1(agent1, object2, recipient3).

Dual-event Relative Constructions

  • The block that pushed the cylinder touched the moon.
  • Agent1 that verb1ed object2 verb2ed object3.
  • The cylinder that was pushed by the block gave the cat to the dog.

Dual-event Conjoined Constructions

  • The block pushed the cylinder and the moon.
  • The moon and the block were given to the cylinder by the cat.
  • In the previous experiments, for each of the three languages, a limited set of sentences including those with relative phrases was learned, based on a noncompositional mechanism.
  • For these sentences, the verbs in the main and relative phrases in the OpenClassArray became associated with the event components of two distinct copies of the SceneEventArray, corresponding to constructions like that in Figure 1C.

The chief pushed the boy and the block.

  • For the English experiments, the model was first trained on five sentence types corresponding to the active and passive transitive and dative constructions (1–4 in Table 4), and the relative ‘‘The boy the dog hit,’’ along with their corresponding meaning representations.
  • Similarly for Sentence 2, application of the phrase structure extraction yielded three phrases: ‘‘The dog bit the man,’’ ‘‘The cat hit the dog,’’ and ‘‘The boy saw the cat,’’ for each of which the corresponding meanings were extracted based on the previously learned constructions.
  • Cat wo relverb boy ga dog ni-yotte osareta Girl-ga girl-wo bit dog-ga bit cat-wo chased boywo saw.

Psychological Reality of the Model

  • The first principle inherent in the model is that instead of representing <sentence, meaning> mappings in terms of a generative grammar, they are represented directly in a structured inventory of grammatical constructions that are nothing more than these mappings (Tomasello, 2003; Goldberg, 1995).
  • The essential innate capabilities the authors postulate are mechanisms for (1) abstract (variable based) structure mapping and for 2100 Journal of Cognitive Neuroscience Volume 18, Number 12 (2) pattern-based embedded structure processing.
  • What has been lacking, however, is a specification of how grammatical constructions can be combined in order to generate new constructions on the fly, using these same sequence processing mechanisms.
  • This approach should extend to accommodate more generalized relative phrase structure, as well as prepositional and other categories of embedded phrases.
  • Thus, when variations in word order are used in order to place the pragmatic focus on a particular item, as in the Japanese examples 9–12 in Table 5, distinct constructions will be employed to capture these differences.

Division of Labor in the Construction Inventory

  • Construction grammar assumes no strict division between the lexicon and syntax in that both lexical and syntactic constructions pair form with meaning (see Goldberg, 1995).
  • Subsequent developmental categorization of an open class category of concrete nouns would allow verb-based constructions such as ‘‘Gimme X.’’.
  • A pertinent account of such distributional analysis is presented in Cartwright and Brent (1997).
  • The current model represents what the authors consider a necessary refinement that is wholly consistent with construction grammar, and indeed provides insight into a consistent explanation for compositionality.

Temporal Dynamics and Resolution of Detail

  • The theoretical foundation of the model is that the sentence-to-meaning mapping is performed based on the configuration of closed class elements (the ConstructionIndex) in the sentence (consistent with Bates et al., 1982; MacWhinney, 1982).
  • In the current implementation, to reduce processing complexity, the authors wait until the entire sentence has been seen before extracting the corresponding mapping.
  • Miyamoto (2002) observed that in some cases, marking on the noun phrases indicates phrase boundaries prior to the arrival of the verb, so that the detection of the relative phrase structure can actually precede the nonfinal verb (Miyamoto, 2002).
  • In general, if delay is decreased, then the required information must come from an alternative source (such as the double o constraint).
  • Thus, in building up the ConstructionIndex incrementally, when such information is not available then the incremental processing will induce temporary failures including well-documented garden path effects.

Underlying Neurophysiology

  • Part of the importance of the current approach is that the underlying primitive functions that provide this construction processing capability can be linked in a plausible manner to known neurophysiological systems.
  • The resulting pattern of cortical activity projects to the caudate nucleus of the striatum, and via these modifiable corticostriatal synapses, retrieves the appropriate FormToMeaning mapping that is then implemented in the frontal transformation processing network that includes BA 44.
  • The authors have simulated how a recurrent cortical network can encode the sequential structure of closed and open class words in a sentence, corresponding to the ConstructionIndex, in order to retrieve the appropriate FormToMeaning mapping based on corticostriatal associative memory (Dominey et al., 2003; Dominey, 2002, 2005a, 2005b).
  • This result is consistent with previous fMRI studies that showed activation of these areas in sentence comprehension tasks (e.g., Kuperberg et al., 2003; Baumgaertner, Weiller, & Büchel, 2002), particularly for sequential or structural processing aspects in sentences (Newman, Just, Keller, Roth, & Carpenter, 2003; Dapretto & Bookheimer, 1999; Kang, Constable, Gore, & Avrutin, 1999; Inui et al., 1998).
  • This yielded a profile of sentence comprehension performance highly correlated (R2 = .89) with that of the nine agrammatic patients the authors studied (Dominey, 2002).

Extraction of Relative Phrase Structure

  • Part of the novelty of their approach is that the extraction of relative phrase structure relies on these same capabilities.
  • Each extracted phrase is processed by the construction model, and then the remainder of the original sequence is processed.
  • At the extreme, this is related to lexically driven approaches including that of Vosse and Kempen (2000), lexical functional grammar (LFG; Bresnan, 2001), and head-driven phrase structure grammar (HPSG; Pollard & Sag, 1994), in which lexical entries include type descriptions that form a hierarchy that, combined with grammatical rules, allows for unificationbased parsing.
  • One source of variability is the structure of the noun phrase.

Acknowledgments

  • This study was supported in part by the HFSP Organization, the French ACI NIC and TTT Projects, LAFMI, and the ESF.
  • Reprint requests should be sent to Peter Ford Dominey, Institut des Sciences Cognitives, CNRS UMR 5015, 67, Boulevard Pinel, 69675 Bron Cedex, France, or via e-mail: dominey@isc.cnrs.fr and www.isc.cnrs.fr/.

Did you find this useful? Give us your feedback

...read more

Content maybe subject to copyright    Report

A Neurolinguistic Model of Grammatical
Construction Processing
Peter Ford Dominey
1
, Michel Hoen
1
, and Toshio Inui
2
Abstract
& One of the functions of everyday human language is to com-
municate meaning. Thus, when one hears or reads the sen-
tence, ‘‘John gave a book to Mary,’’ some aspect of an event
concerning the transfer of possession of a book from John to
Mary is (hopefully) transmitted. One theoretical approach to
language referred to as construction grammar emphasizes this
link between sentence structure and meaning in the form of
grammatical constructions. The objective of the current re-
search is to (1) outline a functional description of grammatical
construction processing based on principles of psycholinguis-
tics, (2) develop a model of how these functions can be imple-
mented in human neurophysiology, and then (3) demonstrate
the feasibility of the resulting model in processing languages of
typologically diverse natures, that is, English, French, and Japa-
nese. In this context, particular interest will be directed toward
the processing of novel compositional structure of relative
phrases. The simulation results are discussed in the context of
recent neurophysiological studies of language processing. &
INTRODUCTION
One of the long-term quests of cognitive neuroscience
has been to link functional aspects of language process-
ing to its underlying neurophysiology, that is, to under-
stand how neural mechanisms allow the mapping of the
surface structure of a sentence onto a conceptual repre-
sentation of its meaning. The successful pursuit of this
objective will likely prove to be a multidisciplinary endeav-
or that requires cooperation between theoretical, devel-
opmental, and neurological approaches to the study of
language, as well as contributions from computational
modeling that can eventually validate proposed hypothe-
ses. The current research takes this multidisciplinary
approach within the theoretical framework associated
with grammatical constructions. The essential distinc-
tion within this context is that language is considered to
consist of a structured inventory of mappings between
the surface forms of utterances and meanings, referred
to as grammatical constructions (see Goldberg, 1995).
These mappings vary along a continuum of complexity.
At one end are single words and fixed ‘‘holophrases’’
such as ‘‘Gimme that’’ that are processed as unparsed
‘‘holistic’’ items (see Tomasello, 2003). At the other
extreme are complex abstract argument constructions
that allow the use of sentences like this one. In between
are the workhorses of everyday language, abstract argu-
ment constructions that allow the expression of spatio-
temporal events that are basic to human experience
including active transitive (e.g., John took the car) and
ditransitive (e.g., Mary gave my mom a new recipe)
constructions (Goldberg, 1995).
In this context, the ‘‘usage-based’’ perspective holds
that the infant begins language acquisition by learning
very simple constructions in a progressive development
of processing complexity, with a substantial amount of
early ground that can be covered with relatively mod-
est computational resources (Clark, 2003; Tomasello,
2003). This is in contrast with the ‘‘continuity hypothe-
sis’’ issued from the generative grammar philosophy,
which holds that the full range of syntactic complexity
is available in the form of a universal grammar and is
used at the outset of language learning (see Tomasello,
2000, and comments on the continuity hypothesis de-
bate). In this generative context, the universal grammar
is there from the outset, and the challenge is to under-
stand how it got there. In the usage-based construction
approach, initial processing is simple and becomes
increasingly complex, and the challenge is to explain
the mechanisms that allow full productivity and com-
positionality (composing new constructions from exist-
ing ones). This issue is partially addressed in the current
research.
Theoretical Framework for
Grammatical Constructions
If grammatical constructions are mappings from sen-
tence structure to meaning, then the system must be
capable of (1) identifying the type of grammatical con-
struction for a given sentence and (2) using the identi-
fied construction and its corresponding mapping to
1
CNRS UMR 5015, France,
2
Kyoto University, Japan
D 2006 Massachusetts Institute of Technology Journal of Cognitive Neuroscience 18:12, pp. 2088–2107

extract the meaning of the sentence. Interestingly, this
corresponds to Townsend and Bever’s (2001) two steps
of syntactic processing, that is, (i) parsing of the sen-
tence into phrasal constituents and words (requiring
access to lexical categories and word order analysis) and
(ii) subsequent analysis of phrasal structure and long-
distance dependencies by means of syntactic rules.
Figure 1A illustrates an example of an active transi-
tive sentence and its mapping onto a representation of
meaning. The generalized representation of the corre-
sponding active transitive construction is depicted in
Figure 1B. The ‘‘slots’’ depicted by the NPs and V can
be instantiated by different nouns and verbs in order to
generate an open set of new sentences. For each sentence
corresponding to this construction type, the mapping of
sentence to meaning is provided by the construction.
Figure 1C illustrates a more complex construction that
contains an embedded relative phrase. In this context, a
central issue in construction grammar will concern how
the potential diversity of constructions are identified.
Part of the response to this lies in the specific ways in
which function and content words are structurally orga-
nized in distinct sentence types. Function words (also
referred to as closed class words because of their limited
number in any given language), including determiners,
prepositions, and auxiliary verbs, play a role in defining
the grammatical structure of a sentence (i.e., in specify-
ing who did what to whom) although they carry little
semantic content. Content words (also referred to as
open class words because of their essentially unlimited
number) play a more central role in contributing pieces
of meaning that are inserted into the grammatical
structure of the sentence. Thus, returning to our exam-
ples in Figure 1, the thematic roles for the content
words are determined by their relative position in the
sentences with respect to the other content words and
with respect to the function words. Although this is the
case in English, Bates, McNew, MacWhinney, Devescovi,
and Smith (1982) and MacWhinney (1982) have made
the case more generally, stating that across human lan-
guages, the grammatical structure of sentences is spec-
ified by a combination of cues including word order,
grammatical function words (and or grammatical mark-
ers attached to the word roots), and prosodic structure.
The general idea here is that these constructions are
templates into which a variety of open class elements
(nouns, verbs, etc.) can be inserted in order to express
novel meanings. Part of the definition of a construction
is the mapping between slots or variables in the tem-
plate and the corresponding semantic roles in the
meaning, as illustrated in Figure 1. In this context, a
substantial part of the language faculty corresponds to a
structured set of such sentence-to-meaning mappings,
and these mappings are stored and retrieved based on
the patterns of structural markers (i.e., word order and
function word patterns) unique to each grammatical
construction type. Several major issues can be raised
with respect to this characterization. The issues that we
address in the current research are as follows: (1) Can
this theoretical characterization be mapped onto human
functional neuroanatomy in a meaningful manner and
(2) can the resulting system be demonstrated to account
for a restricted subset of human language phenomena
in a meaningful manner?
Implementing Grammatical Constructions
in a Neurocomputational Model
Interestingly, this characterization of grammatical con-
structions can be reformulated into a type of sequence
learning problem, if we consider a sentence as a se-
quence of words. Determining the construction type for
a given sentence consists in analyzing or recoding the
sentence as a sequence of open class and closed class
elements, and then performing sequence recognition on
this recoded sequence. Dominey (1995) and Dominey,
Arbib, and Joseph (1995) have demonstrated how such
sequence recognition can be performed by a recurrent
prefrontal cortical network (a ‘‘temporal recurrent net-
work’’ [TRN]) that encodes sequential structure (see
also Dominey 1998a, b). Then corticostriatal connec-
tions allow the association of different categories of
sequences represented in the recurrent network with
different behavioral responses.
Once the construction type has thus been identified,
the corresponding mapping of open class elements
onto their semantic roles must be retrieved and per-
formed. This mapping corresponds to what we have
called ‘‘abstract structure’’ processing (Dominey, Lelekov,
Ventre-Dominey, & Jeannerod, 1998). In this context, the
reordering in the form–meaning mapping in Figure 1B
can be characterized as the abstract structure ABCD-
Figure 1. Grammatical construction overview. (A) Specific example
of a sentence-to-meaning mapping. (B) Generalized representation
of the construction. (C) Sentence-to-meaning mapping for sentence
with relativized phrase. (D) Compositional sentence-to-meaning
mapping for sentence with relative phrase in which the relative
phrase has been extracted.
Dominey, Hoen, and Inui 2089

BACD where A-D represent variable slots. In order to
accommodate such abstract structures, rather than rep-
resenting sequences of distinct elements, we modified
the recurrent network model to represent sequences of
variables corresponding to prefrontal working memory
elements (Dominey et al., 1998).
Concretely, from a developmental perspective, we
demonstrated that the resulting abstract recurrent net-
work (ARN) could simulate human infant performance
in distinguishing between abstract structures such as
ABB versus AAB (Dominey & Ramus, 2000) as described
by Marcus et al. (1999). In the grammatical construction
analog, these abstract structures correspond to the
mapping of word order in the sentence onto semantic
arguments in the meaning as illustrated in Figure 1. We
thus demonstrated how the dual TRN/ARN system could
be used for sentence comprehension. The sequence of
closed class elements defining the construction was
processed by the TRN (corresponding to a recurrent
corticocortical network). Then, via modifiable cortico-
striatal synapses, the resulting pattern of cortical activity
recovered the corresponding abstract structure for re-
ordering the open class elements into the appropriate
semantic argument order by the ARN (Dominey, Hoen,
Blanc, & Lelekov-Boissard, 2003; Dominey, 2002).
The model essentially predicted a common neuro-
physiological basis for abstract mappings of the form
BHM-HMB and form-to-meaning mappings such as ‘‘The
ball was hit by Mary’’: Hit(Mary, ball). Concretely, we
predicted that brain lesions in the left perisylvian cortex
that produce syntactic comprehension deficits would
produce correlated impairments in abstract sequence
processing. The first test of this prediction was thus to
compare performance in aphasic patients on these two
types of behavior. We observed a highly significant
correlation between performance on syntactic compre-
hension and abstract structure processing in aphasic
patients, as well as in schizophrenic patients (Dominey
et al., 2003; Lelekov et al., 2000). We reasoned that if this
correlation was due to a shared brain mechanism, then
training in one of the tasks should transfer to improve-
ment on the other. Based on this prediction, we subse-
quently demonstrated that reeducation with specific
nonlinguistic abstract structures transferred to improved
comprehension for the analogous sentence types in
a group of aphasic patients (Hoen et al., 2003). In or-
der to begin to characterize the underlying shared neu-
ral mechanisms, Hoen and Dominey (2000) measured
brain activity with event-related potentials for abstract
sequences in which the mapping was specified by a
‘‘function symbol’’ analogous to function words in sen-
tences. Thus, in the sequences ABCxBAC and ABCzABC,
the two function symbols x and z indicate two distinct
structure mappings. We thus observed a left anterior
negativity (LAN) in response to the function symbols,
analogous to the LAN observed in response to gram-
matical function words during sentence processing
(Hoen & Dominey, 2000), again suggesting a shared
neural mechanism. In order to neuroanatomically local-
ize this shared mechanism, we performed a related set
of brain imagery experiments comparing sentence and
sequence processing using functional magnetic reso-
nance imaging (fMRI). We observed that a common cor-
tical network including Brodmann’s area (BA) 44 was
involved in the processing of sentences and abstract
structure in nonlinguistic sequences, whereas BA 45
was exclusively activated in sentence processing, corre-
sponding to insertion of lexical semantic content into
the transformation mechanism (Hoen, Pachot-Clouard,
Segebarth, & Dominey, 2006).
This computational neuroscience approach allowed
the projection of the construction grammar framework
onto the corticostriatal system with two essential prop-
erties: first, construction identification by corticostriatal
sequence recognition, and second, structure mapping
based on the manipulation of sequences of frontal cor-
tical working memory elements (Dominey & Hoen,
2006). Given the initial validation of this model in the
neurophysiological and neuropsychological studies cited
above, we can now proceed with a more detailed study
of how this can contribute to an understanding of the
possible implementation of grammatical constructions,
as illustrated in Figure 2.
As the sentence is processed word by word, a process
of lexical categorization identifies open and closed class
words. This is not unrealistic, as newborns can perform
this categorization (Shi, Werker, & Morgan, 1999), and
several neural network studies have demonstrated lexi-
cal categorization of this type based on prosodic cues
(Blanc, Dodane, & Dominey, 2003; Shi et al., 1999). In
the current implementation, only nouns and verbs are
recognized as open class words, with the modification of
these by adjectives and adverbs left for now as a future
issue. The meanings of the open class words are re-
trieved from the lexicon (not addressed here, but see
Dominey & Boucher, 2005; Roy, 2002; Dominey, 2000;
Siskind, 1996) and these referent meanings are stored in
a working memory called the PredictedReferentsArray.
The next crucial step is the mapping of these referent
meanings onto the appropriate components of the
meaning structure. In Figure 2, this corresponds to
the mapping from the PredictedReferentsArray onto
the meaning coded in the SceneEventArray. As seen in
Figure 2A and B, this mapping varies depending on the
construction type. Thus, the system must be able to
store and retrieve different FormToMeaning mappings
appropriate for different sentence types, corresponding
to distinct grammatical constructions.
During the lexical categorization process, the struc-
ture of the sentence is recoded based on the local
structure of open and closed class words, in order to
yield a ConstructionIndex that will be unique to each
construction type, corresponding to the cue ensemble
of Bates et al. (1982) in the general case. The closed
2090 Journal of Cognitive Neuroscience Volume 18, Number 12

class words are explicitly represented in the Construc-
tionIndex, whereas open class words are represented as
slots that can take open class words as arguments. The
ConstructionIndex is thus a global representation of the
sentence structure. Again, the requirement is that every
different grammatical construction type should yield a
unique ConstructionIndex. This ConstructionIndex can
then be used as an index into an associative memory to
store and retrieve the correct FormToMeaning mapping.
We have suggested that this mechanism relies on
recurrent cortical networks and corticostriatal process-
ing (Dominey et al., 2003). In particular, we propose that
the formation of the ConstructionIndex as a neural
pattern of activity will rely on sequence processing in
recurrent cortical networks, and that the retrieval of the
FormToMeaning component will rely on a corticostriatal
associative memory. Finally, the mapping from form to
meaning will take place in the frontal cortical region
including BAs 44, 46, and 6. This corresponds to the
SceneEventArray, consistent with observations that
event meaning is represented in this BA 44 pars oper-
cularis region, when events are being visually observed
(Buccino et al., 2004), and when their descriptions are
listened to (Tettamanti et al., 2005). Hoen et al. (2006)
provided strong evidence that for both the processing of
grammatical and nonlinguistic structure processing
rules, this frontal cortical region including BAs 44, 46,
and 6 was activated (see Figure 3), indicating its role in
linguistic and nonlinguistic structural mapping.
The proposed role of basal ganglia in rule storage
and retrieval is somewhat related to the procedural
component of Ullman’s (2001, 2004, 2006) grammar pro-
cessing model in which grammatical rules are encoded
in specific (but potentially domain independent) chan-
nels of the corticostriatal system. Longworth, Keenan,
Barker, Marslen-Wilson, and Tyler (2005) indicate a
more restricted, non-language-specific role of the stria-
tum in language in the selection of the appropriate
mapping in the late integration processes of language
comprehension. Neuropsychological evidence for the
role of the striatum in such rule extraction has been
provided in patients with Huntington’s disease (a form
of striatal dysfunction) that were impaired in rule appli-
cation in three domains: morphology, syntax, and arith-
metic (Teichmann et al., 2005). These data are thus
consistent with the hypothesis that the striatum is
Figure 2. Structure-mapping
architecture. (A) Passive
sentence processing: Step 1,
lexical categorization—
open and closed class words
directed to OpenClassArray
and ConstructionIndex,
respectively. Step 2, open
class words in OpenClassArray
are translated to their
referent meanings via the
WordToReferent mapping.
Insertion of this referent
semantic content into
the Predicted Referents
Array (PRA) is realized in
pars triangularis BA 45.
Step 3, PRA elements are
mapped onto their roles
in the SceneEventArray
by the FormToMeaning
mapping specific to each
sentence type. Step 4,
This mapping is retrieved
from ConstructionInventory
(a corticostriatal associative
memory) via the Construction-
Index (a corticocortical
recurrent network) that
encodes the closed and
open class word patterns
that characterize each
grammatical construction type.
The structure mapping process
is associated with activation of
pars opercularis BA 44. In the
current implementation, neural network associative memory for the ConstructionInventory is replaced by a procedural lookup table. (B) Active
sentence. Note difference in ConstructionIndex and in FormToMeaning.
Dominey, Hoen, and Inui 2091

involved in the computational application of rules in-
cluding our proposed mappings.
In contrast, the integration of lexicosemantic con-
tent into this structure processing machinery, filling of
the PredictedReferentsArray, corresponds to a more
language-specific ventral stream mechanism that culmi-
nates in the pars triangularis (BA 45) of the ventral pre-
motor area, consistent with the declarative component
of Ullman’s (2004) model. In this context, Hoen et al.
(2006) observed that when the processing of sentences
and nonlinguistic sequences was compared, BA 45 was
activated exclusively by the sentence processing.
Goals of the Current Study
We have previously demonstrated that the model in
Figure 2 can learn a variety of constructions that can
then be used in different language interaction contexts
(Dominey & Boucher, 2005a, b; Dominey, 2000). How-
ever, from a functional perspective, a model of gram-
matical construction processing should additionally
address two distinct challenges of (1) cross-linguistic
validity and (2) compositionality, which we will address
in the current study.
Although the examples used above have been pre-
sented in English, the proposed model of grammatical
construction processing should accommodate different
types of languages. Thus, the first objective will be to
demonstrate that the proposed system is capable of
learning grammatical constructions in English, French,
and Japanese. Whereas English and French are relatively
similar in their linguistic structure, Japanese is some-
what different in that it allows more freedom in word
order, with information about thematic roles encoded
in case markers.
Regarding the challenge of compositionality, as pre-
sented above, it appears that for each different type of
sentence, the system must have a distinct construction.
Considering Figure 1C, this indicates that every time a
noun is expanded into a relative phrase, a new construc-
tion will be required. This is undesirable for two reasons:
First it imposes a large number of similar constructions to
be stored, and second, it means that if a sentence occurs
with a relative phrase in a new location, the model will
fail to understand that sentence without first having a
<sentence, meaning> pair from which it can learn the
mapping. Alternatively, the model could process con-
structions within constructions in a compositional man-
ner. Thus, the sentence ‘‘The dog that chased the cat bit
Mary’’ could be decomposed into its constituent phrases
‘‘The dog chased the cat’’ and ‘‘The dog bit Mary.’’ The
goal is to determine whether the model as presented
above can accommodate this kind of compositionality.
During the processing of multiple sentences with this
type of embedded relative clause, it will repeatedly
occur that the pattern ‘‘agent that action object’’ will
occur, and will map onto the meaning component
action(agent, object). The same kind of pattern finding
that allows the association of a ConstructionIndex with
the corresponding FormToMeaning mapping could also
work on such patterns that reoccur within sentences.
That is, rather than looking for patterns at the level of
the entire sentence or ConstructionIndex, the model
could apply exactly the same mechanisms to identify
subpatterns within the sentence. This would allow the
system to generalize over these occurrences such that
when this pattern occurs in a new location in a sentence,
it can be extracted. This will decompose the sentence
into the principal and the relative phrases, with the out-
come that both will more likely correspond to existing
constructions (see Figure 1D). Clearly, this will not work
in all cases of new relative clauses, but more importantly,
it will provide a compositional capability for the system
to accommodate a subclass of the possible new sentence
types. This type of segmentation approach has been
explored by Miikkulainen (1996), discussed below.
METHODS
In the first set of experiments (Experiment 1A–C) for each
of the three languages English, Japanese, and French,
respectively, we will expose the model to a set of <sen-
tence, meaning> pairs in a training phase. We then
validate that the model has learned these sentences
by presenting the sentences alone and comparing the
Figure 3. Comparison of brain activation in sentence processing
and nonlinguistic sequence mapping tasks. Subjects read visually
presented sentences and nonlinguistic sequences presented one
word/element at a time, and performed grammaticality/correctness
judgments after each, responding by button press. Red areas
indicate regions activated by both tasks, including a prefrontal
network that involves BAs 44, 46, 9, and 6. Green areas indicate
cortical regions activated exclusively in the sentence processing
task, including BAs 21, 22, 47, and 45. From Hoen et al. (2006)
with permission.
2092 Journal of Cognitive Neuroscience Volume 18, Number 12

Citations
More filters

Journal ArticleDOI
TL;DR: This review systematically surveys both current ways of generating/adapting the reservoirs and training different types of readouts, and offers a natural conceptual classification of the techniques, which transcends boundaries of the current ''brand-names'' of reservoir methods.
Abstract: Echo State Networks and Liquid State Machines introduced a new paradigm in artificial recurrent neural network (RNN) training, where an RNN (the reservoir) is generated randomly and only a readout is trained. The paradigm, becoming known as reservoir computing, greatly facilitated the practical application of RNNs and outperformed classical fully trained RNNs in many tasks. It has lately become a vivid research field with numerous extensions of the basic idea, including reservoir adaptation, thus broadening the initial paradigm to using different methods for training the reservoir and the readout. This review systematically surveys both current ways of generating/adapting the reservoirs and training different types of readouts. It offers a natural conceptual classification of the techniques, which transcends boundaries of the current ''brand-names'' of reservoir methods, and thus aims to help in unifying the field and providing the reader with a detailed ''map'' of it.

1,735 citations


Journal ArticleDOI
TL;DR: Networks involving the temporal cortex and the inferior frontal cortex with a clear left lateralization were shown to support syntactic processes, whereas less lateralized temporo-frontal networks subserve semantic processes.
Abstract: Language processing is a trait of human species. The knowledge about its neurobiological basis has been increased considerably over the past decades. Different brain regions in the left and right hemisphere have been identified to support particular language functions. Networks involving the temporal cortex and the inferior frontal cortex with a clear left lateralization were shown to support syntactic processes, whereas less lateralized temporo-frontal networks subserve semantic processes. These networks have been substantiated both by functional as well as by structural connectivity data. Electrophysiological measures indicate that within these networks syntactic processes of local structure building precede the assignment of grammatical and semantic relations in a sentence. Suprasegmental prosodic information overtly available in the acoustic language input is processed predominantly in a temporo-frontal network in the right hemisphere associated with a clear electrophysiological marker. Studies with patients suffering from lesions in the corpus callosum reveal that the posterior portion of this structure plays a crucial role in the interaction of syntactic and prosodic information during language processing.

1,073 citations


Cites methods from "A Neurolinguistic Model of Grammati..."

  • ...Based on their data, they proposed the following functional subdivision of Broca’s area and its adjacent regions in the prefrontal cortex (54, 121): superior and posterior regions (BA 6/9/46/44) are engaged in sequential and structural aspects of processing, whereas anterior and inferior regions (BA 11/47/45) are implicated in context information insertion into structural matrixes selected by the upper regions....

    [...]


Journal ArticleDOI
TL;DR: The results showed that listening to action-related sentences activates a left fronto-parieto-temporal network that includes the pars opercularis of the inferior frontal gyrus (Broca's area), as well as the inferior parietal lobule, the intraparietal sulcus, and the posterior middle temporal gyrus.
Abstract: Observing actions made by others activates the cortical circuits responsible for the planning and execution of those same actions This observation–execution matching system (mirror-neuron system) is thought to play an important role in the understanding of actions made by others In an fMRI experiment, we tested whether this system also becomes active during the processing of action-related sentences Participants listened to sentences describing actions performed with the mouth, the hand, or the leg Abstract sentences of comparable syntactic structure were used as control stimuli The results showed that listening to action-related sentences activates a left fronto-parieto-temporal network that includes the pars opercularis of the inferior frontal gyrus (Broca's area), those sectors of the premotor cortex where the actions described are motorically coded, as well as the inferior parietal lobule, the intraparietal sulcus, and the posterior middle temporal gyrus These data provide the first direct evidence that listening to sentences that describe actions engages the visuomotor circuits which subserve action execution and observation

941 citations


Journal ArticleDOI
TL;DR: Cognitive developmental robotics aims to provide new understanding of how human's higher cognitive functions develop by means of a synthetic approach that developmentally constructs cognitive functions through interactions with the environment, including other agents.
Abstract: Cognitive developmental robotics (CDR) aims to provide new understanding of how human's higher cognitive functions develop by means of a synthetic approach that developmentally constructs cognitive functions. The core idea of CDR is ldquophysical embodimentrdquo that enables information structuring through interactions with the environment, including other agents. The idea is shaped based on the hypothesized development model of human cognitive functions from body representation to social behavior. Along with the model, studies of CDR and related works are introduced, and discussion on the model and future issues are argued.

487 citations


Journal ArticleDOI
TL;DR: No studies have satisfied all the criteria necessary to support the more specific neurobiological claims made by the two hypotheses-namely, that each level of verb meaning is associated with mirror neurons in the pertinent brain regions, which would require demonstrating that within those regions the same neuronal populations are engaged during (a) the linguistic processing of particular motor features of verb mean, (b) the execution of actions with the corresponding motor features, and (c) the observation ofactions with the equivalent motor features.
Abstract: Verbs have two separate levels of meaning. One level reflects the uniqueness of every verb and is called the "root". The other level consists of a more austere representation that is shared by all the verbs in a given class and is called the "event structure template". We explore the following hypotheses about how, with specific reference to the motor features of action verbs, these two distinct levels of semantic representation might correspond to two distinct levels of the mirror neuron system. Hypothesis 1: Root-level motor features of verb meaning are partially subserved by somatotopically mapped mirror neurons in the left primary motor and/or premotor cortices. Hypothesis 2: Template-level motor features of verb meaning are partially subserved by representationally more schematic mirror neurons in Brodmann area 44 of the left inferior frontal gyrus. Evidence has been accumulating in support of the general neuroanatomical claims made by these two hypotheses-namely, that each level of verb meaning is associated with the designated cortical areas. However, as yet no studies have satisfied all the criteria necessary to support the more specific neurobiological claims made by the two hypotheses-namely, that each level of verb meaning is associated with mirror neurons in the pertinent brain regions. This would require demonstrating that within those regions the same neuronal populations are engaged during (a) the linguistic processing of particular motor features of verb meaning, (b) the execution of actions with the corresponding motor features, and (c) the observation of actions with the corresponding motor features.

183 citations


Cites methods from "A Neurolinguistic Model of Grammati..."

  • ...This is analogous to Dominey et al.’s (2006) structure mapping process, since both...

    [...]


References
More filters

Book
01 Jan 1994
TL;DR: This book presents the most complete exposition of the theory of head-driven phrase structure grammar, introduced in the authors' "Information-Based Syntax and Semantics," and demonstrates the applicability of the HPSG approach to a wide range of empirical problems.
Abstract: This book presents the most complete exposition of the theory of head-driven phrase structure grammar (HPSG), introduced in the authors' "Information-Based Syntax and Semantics." HPSG provides an integration of key ideas from the various disciplines of cognitive science, drawing on results from diverse approaches to syntactic theory, situation semantics, data type theory, and knowledge representation. The result is a conception of grammar as a set of declarative and order-independent constraints, a conception well suited to modelling human language processing. This self-contained volume demonstrates the applicability of the HPSG approach to a wide range of empirical problems, including a number which have occupied center-stage within syntactic theory for well over twenty years: the control of "understood" subjects, long-distance dependencies conventionally treated in terms of "wh"-movement, and syntactic constraints on the relationship between various kinds of pronouns and their antecedents. The authors make clear how their approach compares with and improves upon approaches undertaken in other frameworks, including in particular the government-binding theory of Noam Chomsky.

3,544 citations


Book
15 Mar 1995
Abstract: Acknowledgments 1: Introduction 2: The Interaction between Verbs and Constructions 3: Relations among Constructions 4: On Linking 5: Partial Productivity 6: The English Ditransitive Construction 7: The English Caused-Motion Construction 8: The English Resultative Construction 9: The Way Construction 10: Conclusion Notes Bibliography Index

3,370 citations


Book
01 Jan 2003
Abstract: Drawing together a vast body of empirical research in cognitive science, linguistics, and developmental psychology, Michael Tomasello demonstrates that we don't need a self-contained "language instinct" to explain how children learn language. Their linguistic ability is interwoven with other cognitive abilities. Tomasello argues that the essence of language is its symbolic dimension, which rests on the uniquely human ability to comprehend intention. Grammar emerges as the speakers of a language create linguistic constructions out of recurring sequences of symbols, children pick up these patterns in the buzz of words they hear around them. Constructing a Language offers a compellingly argued, psychologically sound new vision for the study of language acquisition.

2,803 citations


Journal ArticleDOI
TL;DR: This review argues that sentence processing is supported by a temporo-frontal network, within this network, temporal regions subserve aspects of identification and frontal regions the building of syntactic and semantic relations.
Abstract: Functional dissociations within the neural basis of auditory sentence processing are difficult to specify because phonological, syntactic and semantic information are all involved when sentences are perceived. In this review I argue that sentence processing is supported by a temporo-frontal network. Within this network, temporal regions subserve aspects of identification and frontal regions the building of syntactic and semantic relations. Temporal analyses of brain activation within this network support syntax-first models because they reveal that building of syntactic structure precedes semantic processes and that these interact only during a later stage.

1,636 citations


Journal ArticleDOI
TL;DR: It is proposed that "language" disorders, such as specific language impairment and non-fluent and fluent aphasia, may be profitably viewed as impairments primarily affecting one or the other brain system, and suggested a new neurocognitive framework for the study of lexicon and grammar.
Abstract: The structure of the brain and the nature of evolution suggest that, despite its uniqueness, language likely depends on brain systems that also subserve other functions. The declarative/procedural (DP) model claims that the mental lexicon of memorized word-specific knowledge depends on the largely temporal-lobe substrates of declarative memory, which underlies the storage and use of knowledge of facts and events. The mental grammar, which subserves the rule-governed combination of lexical items into complex representations, depends on a distinct neural system. This system, which is composed of a network of specific frontal, basal-ganglia, parietal and cerebellar structures, underlies procedural memory, which supports the learning and execution of motor and cognitive skills, especially those involving sequences. The functions of the two brain systems, together with their anatomical, physiological and biochemical substrates, lead to specific claims and predictions regarding their roles in language. These predictions are compared with those of other neurocognitive models of language. Empirical evidence is presented from neuroimaging studies of normal language processing, and from developmental and adult-onset disorders. It is argued that this evidence supports the DP model. It is additionally proposed that "language" disorders, such as specific language impairment and non-fluent and fluent aphasia, may be profitably viewed as impairments primarily affecting one or the other brain system. Overall, the data suggest a new neurocognitive framework for the study of lexicon and grammar.

1,386 citations


Frequently Asked Questions (1)
Q1. What have the authors contributed in "A neurolinguistic model of grammatical construction processing" ?

The simulation results are discussed in the context of recent neurophysiological studies of language processing.