Chomsky hierarchy

About: Chomsky hierarchy is a research topic. Over the lifetime, 601 publications have been published within this topic receiving 31067 citations. The topic is also known as: Chomsky–Schützenberger hierarchy.

Infinite RAAM: A Principled Connectionist Basis for Grammatical Competence

Abstract: Infinite RAAM: A Principled Connectionist Basis for Grammatical Competence Simon Levy, Ofer Melnik and Jordan Pollack levy, melnik, pollack@cs.brandeis.edu Dynamical and Evolutionary Machine Organization Volen Center for Complex Systems, Brandeis University, Waltham, MA 02454, USA February 6, 2000 Abstract This paper presents Infinite RAAM (IRAAM), a new fusion of recurrent neural networks with fractal geometry, allowing us to understand the behavior of these networks as dynamical sys- tems. Our recent work with IRAAMs has shown that they are capable of generating the context-free (non-regular) language for arbitrary values of . This paper expands upon that work, showing that IRAAMs are capable of generating syntac- tically ambiguous languages but seem less capable of gener- ating certain context-free constructions that are absent or dis- favored in natural languages. Together, these demonstrations support our belief that IRAAMs can provide an explanatorily adequate connectionist model of grammatical competence in natural language. Natural Language Issues In an early and extremely influential paper, Noam Chomsky (1956) showed that natural languages (NL’s) cannot be mod- eled by a finite-state automaton, because of the existence of center-embedded constructions. A second and equally im- portant observation from this work was that a minimally ade- quate NL grammar must be ambiguous, assigning more than one structure (interpretation) to some sentences, for example, They are flying planes. The first observation led to the development of Chomsky’s formal hierarchy of languages, based on the computational resources of the machines needed to recognize them. In this hierarchy, Chomsky’s observation about center-embedding is expressed by saying that NL’s are non-regular; i.e., they can- not be generated by having only rules of the form a grammar , where and are non-terminal symbols and is a terminal symbol. Whether NL’s are merely non-regular, belonging in the next, context-free (CF) level of the Chomsky hierarchy, or are more powerful, belonging further up in the hierarchy, became the subject of heated debate (Higginbotham 1984; Postal and Langendoen 1984; Shieber 1985). Non-CF phenomena such as reduplication/copying (Culy 1985) and crossed serial de- pendencies (Bresnan, Kaplan, Peters, and Zaenen 1982) sug- gested that a more powerful approach, using syntactic trans- formations (Chomsky 1957) was called for, but some re- searchers criticized transformations as having arbitrary power and thus failing to constrain the types of languages that could be expressed (Gazdar 1982). Further criticism of the entire formal approach came from observing that even CF gram- mars (CFGs) had the power to generate structures, such as a sequence followed by its mirror image, that did not seem to occur in NL (Manaster-Ramer 1986), or which placed an extraordinary burden on the human parsing mechanism when they did occur (Bach, Brown, and Marslen-Wilson 1986). Connectionism and Natural Language While debates about the complexity of NL were raging, connectionism was beginning to awaken from a fifteen-year sleep. In connectionist models many researchers found a way of embodying flexibility, graceful degradation, and other non-rigid properties that seem to characterize real cognitive systems like NL. This research culminated the publication of a highly controversial paper by Rumelhart and McClel- land (1986) which provided a connectionist account of part of the grammar of English using a feed-forward neural net- work. The paper was soon criticized by more traditional cog- nitive scientists (Fodor and Pylyshyn 1988; Pinker and Prince 1988), who cited the non-generative nature of such connec- tionist models as a fundamental shortcoming of the entire field. Partly in response to these criticisms, many connection- ists have spent the past decade investigating network models which support generativity through recurrent (feedback) con- nections (Lawrence, Giles, and Fong 1998; Rodriguez, Wiles, and Elman 1999; Williams and Zipser 1989). The research we present here is an attempt to contribute to this effort while focusing as strongly as possible on the natural language is- sues described above. Such an attempt faces a number of challenges. First, despite analysis of how a network’s dynamics con- tribute to its generativity, it is often uncertain whether the dynamics can support generation of well-formed strings be- yond a certain length. That is, it is unknown whether the net- work has a true “competence” for the language of which it has learned a few exemplars, or is merely capable of generating a finite, and hence regular, subset of the language. 1 Second, it is often easier to model weak, rather than strong genera- tive capacity, by building networks that generate or recognize strings having certain properties, without assigning any syn- tactic structure to the strings. Third, this lack of syntactic structure inhibits the formulation of an account of syntactic ambiguity in such networks, making them less plausible as models of NL. To be fair, not all connectionists, or cognitive scientists, take seriously the notion that human language has infinite generative ca- pacity. Though we obviously do not have the resources to argue the issue here, we are certain that a model with a provably infinite competence would be more persuasive to the cognitive science com- munity as a whole than would a model without one.

Codes Over Languages

Abstract: This correspondence discusses a restrictive structure of the customary hierarchy of formal languages attained by imposing restrictions on a set of productions and on their Iuse by means of codewords. A coded fuzzy language (CFL) is defined in order gap between formal languages and natural languages. Somte properties of CFLs and their relationships to the restrictive device are studied. A cyclic language of order n is defined to investigate properties of formal and fuzzy languages with regard to classes of recognizers.

Langford strings, formal languages and contextual ambiguity

Abstract: We propose a series of generalizations of Langford strings considered in Combinatorics and discuss the place of the corresponding languages in Chomsky hierarchy, as well as their contextual ambiguity in the sense of algebraic linguistics.

Merge-Generability as the Key Concept of Human Language: Evidence From Neuroscience.

Abstract: Ever since the inception of generative linguistics, various dependency patterns have been widely discussed in the literature, particularly as they pertain to the hierarchy based on "weak generation" - the so-called Chomsky Hierarchy. However, humans can make any possible dependency patterns by using artificial means on a sequence of symbols (e.g., computer programing). The differences between sentences in human language and general symbol sequences have been routinely observed, but the question as to why such differences exist has barely been raised. Here, we address this problem and propose a theoretical explanation in terms of a new concept of "Merge-generability," that is, whether the structural basis for a given dependency is provided by the fundamental operation Merge. In our functional magnetic resonance imaging (fMRI) study, we tested the judgments of noun phrase (NP)-predicate (Pred) pairings in sentences of Japanese, an SOV language that allows natural, unbounded nesting configurations. We further introduced two pseudo-adverbs, which artificially force dependencies that do not conform to structures generated by Merge, i.e., non-Merge-generable; these adverbs enable us to manipulate Merge-generability (Natural or Artificial). By employing this novel paradigm, we obtained the following results. Firstly, the behavioral data clearly showed that an NP-Pred matching task became more demanding under the Artificial conditions than under the Natural conditions, reflecting cognitive loads that could be covaried with the increased number of words. Secondly, localized activation in the left frontal cortex, as well as in the left middle temporal gyrus and angular gyrus, was observed for the [Natural - Artificial] contrast, indicating specialization of these left regions in syntactic processing. Any activation due to task difficulty was completely excluded from activations in these regions, because the Natural conditions were always easier than the Artificial ones. And finally, the [Artificial - Natural] contrast resulted in the dorsal portion of the left frontal cortex, together with wide-spread regions required for general cognitive demands. These results indicate that Merge-generable sentences are processed in these specific regions in contrast to non-Merge-generable sentences, demonstrating that Merge is indeed a fundamental operation, which comes into play especially under the Natural conditions.

The boolean closure of growing context-sensitive languages

Abstract: The class of growing context-sensitive languages (GCSL) is a naturally defined subclass of context-sensitive languages whose membership problem is solvable in polynomial time. GCSL and its deterministic counterpart called Church-Rosser Languages (CRL) complement the Chomsky hierarchy in a natural way [9]. In this paper, the extension of GCSL obtained by closures of this class under the boolean operations are investigated. We show that there exists an infinite intersection hierarchy, answering an open problem from [1]. Further, we compare the expressive power of the boolean closures of GCSL, CRL, CFL and LOGCFL.