On certain formal properties of grammars
TL;DR: A sequence of restrictions that limit grammars first to Turing machines, then to two types of system from which a phrase structure description of the generated language can be drawn, and finally to finite state Markov sources are shown to be increasingly heavy.
Abstract: A grammar can be regarded as a device that enumerates the sentences of a language. We study a sequence of restrictions that limit grammars first to Turing machines, then to two types of system from which a phrase structure description of the generated language can be drawn, and finally to finite state Markov sources (finite automata). These restrictions are shown to be increasingly heavy in the sense that the languages that can be generated by grammars meeting a given restriction constitute a proper subset of those that can be generated by grammars meeting the preceding restriction. Various formulations of phrase structure description are considered, and the source of their excess generative power over finite state sources is investigated in greater detail.
Citations
More filters
Book•
01 Jul 1978TL;DR: This volume intended to serve as a text for upper undergraduate and graduate level students and special emphasis is given to the role of algebraic techniques in formal language theory through a chapter devoted to the fixed point approach to the analysis of context-free languages.
Abstract: From the Publisher:
Formal language theory was fist developed in the mid 1950's in an attempt to develop theories of natural language acquisition. It was soon realized that this theory (particularly the context-free portion) was quite relevant to the artificial languages that had originated in computer science. Since those days, the theory of formal languages has been developed extensively, and has several discernible trends, which include applications to the syntactic analysis of programming languages, program schemes, models of biological systems, and relationships with natural languages.
1,415 citations
Cites background from "On certain formal properties of gra..."
...[2] Chomsky, Noam, "On Certain Formal Properties of Grammar....
[...]
TL;DR: The historical evolution of NLP is described, and common NLP sub-problems in this extensive field are summarized, and possible future directions for NLP are considered.
1,346 citations
TL;DR: A recognition algorithm is exhibited whereby an arbitrary string over a given vocabulary can be tested for containment in a given context-free language and it is shown that it is completed in a number of steps proportional to the “cube” of the number of symbols in the tested string.
Abstract: A recognition algorithm is exhibited whereby an arbitrary string over a given vocabulary can be tested for containment in a given context-free language. A special merit of this algorithm is that it is completed in a number of steps proportional to the “cube” of the number of symbols in the tested string. As a byproduct of the grammatical analysis, required by the recognition algorithm, one can obtain, by some additional processing not exceeding the “cube” factor of computational complexity, a parsing matrix—a complete summary of the grammatical structure of the sentence. It is also shown how, by means of a minor modification of the recognition algorithm, one can obtain an integer representing the ambiguity of the sentence, i.e., the number of distinct ways in which that sentence can be generated by the grammar. The recognition algorithm is then simulated on a Turing Machine. It is shown that this simulation likewise requires a number of steps proportional to only the “cube” of the test string length.
1,075 citations
972 citations
TL;DR: This chapter discusses the several classes of sentence-generating devices that are closely related, in various ways, to the grammars of both natural languages and artificial languages of various kinds.
Abstract: Publisher Summary This chapter discusses the several classes of sentence-generating devices that are closely related, in various ways, to the grammars of both natural languages and artificial languages of various kinds. By a language it simply mean a set of strings in some finite set V of symbols called the vocabulary of the language. By a grammar a set of rules that give a recursive enumeration of the strings belonging to the language. It can be said that the grammar generates these strings. The chapter discusses the aspect of the structural description of a sentence, namely, its subdivision into phrases belonging to various categories. A major concern of the general theory of natural languages is to define the class of possible strings; the class of possible grammars; the class of possible structural descriptions; a procedure for assigning structural descriptions to sentences, given a grammar; and to do all of this in such a way that the structural description assigned to a sentence by the grammar of a natural language will provide the basis for explaining how a speaker of this language would understand this sentence.
819 citations
References
More filters
TL;DR: It is found that no finite-state Markov process that produces symbols with transition from state to state can serve as an English grammar, and the particular subclass of such processes that produce n -order statistical approximations to English do not come closer, with increasing n, to matching the output of anEnglish grammar.
Abstract: We investigate several conceptions of linguistic structure to determine whether or not they can provide simple and "revealing" grammars that generate all of the sentences of English and only these. We find that no finite-state Markov process that produces symbols with transition from state to state can serve as an English grammar. Furthermore, the particular subclass of such processes that produce n -order statistical approximations to English do not come closer, with increasing n , to matching the output of an English grammar. We formalize-the notions of "phrase structure" and show that this gives us a method for describing language which is essentially more powerful, though still representable as a rather elementary type of finite-state process. Nevertheless, it is successful only when limited to a small subset of simple sentences. We study the formal properties of a set of grammatical transformations that carry sentences with phrase structure into new sentences with derived phrase structure, showing that transformational grammars are processes of the same elementary type as phrase-structure grammars; that the grammar of English is materially simplified if phrase structure description is limited to a kernel of simple sentences from which all other sentences are constructed by repeated transformations; and that this view of linguistic structure gives a certain insight into the use and understanding of language.
2,140 citations
"On certain formal properties of gra..." refers background or methods in this paper
...The basic character izat ion theorem for finite automata is proven in Kleene (1956). called "grammatical transformations" (Harris, 1952a, b, 1957; Chomsky, 1956, 1957)....
[...]
...It should be mentioned that there appears to be good evidence that devices of the kinds studied here are not adequate for formulation of a full grammar for a natural language (see Chomsky, 1956, §4; 1957, Chapter 5)....
[...]
...(Cf. Chomsky, 1956, 1957.)...
[...]
...In Chomsky (1956, 1957), an appropriate ~ and ~ (i.e., an appropriate method for determining structural information in a uniform manner from the grammar) are described informally for several types of grammar, including those that will be studied here....
[...]
...It would, for example, be extremely interesting to know whether it is in principle possible to construct a phrase structure grammar for English (even though there is good motivation of other kinds for not doing so)....
[...]
TL;DR: The notion of recursive functions of positive integers has been studied in the context of symbolic logic as discussed by the authors, and it has been shown that such a concept admits of development into a mathematical theory much as the group concept has been developed into a theory of groups.
Abstract: Introduction. Recent developments of symbolic logic have considerable importance for mathematics both with respect to its philosophy and practice. That mathematicians generally are oblivious to the importance of this work of Gödel, Church, Turing, Kleene, Rosser and others as it affects the subject of their own interest is in part due to the forbidding, diverse and alien formalisms in which this work is embodied. Yet, without such formalism, this pioneering work would lose most of its cogency. But apart from the question of importance, these formalisms bring to mathematics a new and precise mathematical concept, that of the general recursive function of Hërbrand-GödelKleene, or its proved equivalents in the developments of Church and Turing. It is the purpose of this lecture to demonstrate by example that this concept admits of development into a mathematical theory much as the group concept has been developed into a theory of groups. Moreover, that stripped of its formalism, such a theory admits of an intuitive development which can be followed, if not indeed pursued, by a mathematician, layman though he be in this formal field. It is this intuitive development of a very limited portion of a sub-theory of the hoped for general theory that we present in this lecture. We must emphasize that, with a few exceptions explicitly so noted, we have obtained formal proofs of all the consequently mathematical theorems here developed informally. Yet the real mathematics involved must lie in the informal development. For in every instance the informal "proof" was first obtained; and once gotten, transforming it into the formal proof turned out to be a routine chore. We shall not here reproduce the formal definition of recursive function of positive integers. A simple example of such a function is an
686 citations
"On certain formal properties of gra..." refers background in this paper
...I am indebted to George A. Miller for several important observations about the systems under consideration here, and to I~. B. Lees for material improvements in presentation. i Following a familiar technical use of the term "generate," cf. Post (1944)....
[...]
TL;DR: A simple structural characterization theorem for finite state languages is established, based on the cyclical structure of the grammar, which shows that the complement of any finite state language formed on a given vocabulary of symbols is also a finite statelanguage.
Abstract: A finite state language is a finite or infinite set of strings (sentences) of symbols (words) generated by a finite set of rules (the grammar), where each rule specifies the state of the system in which it can be applied, the symbol which is generated, and the state of the system after the rule is applied. A number of equivalent descriptions of finite state languages are explored. A simple structural characterization theorem for finite state languages is established, based on the cyclical structure of the grammar. It is shown that the complement of any finite state language formed on a given vocabulary of symbols is also a finite state language, and that the union of any two finite state languages formed on a given vocabulary is a finite state language; i.e., the set of all finite state languages that can be formed on a given vocabulary is a Boolean algebra. Procedures for calculating the number of grammatical strings of any given length are also described.
388 citations
TL;DR: The authors defines a formal relation among sentences, by virtue of which one sentence structure may be called a transform of another sentence structure (e g the active and the passive, or in a different way question and answer) based on comparing the individual co-occurrences of morphemes.
Abstract: This paper defines a formal relation among sentences, by virtue of which one sentence structure may be called a transform of another sentence structure (e g the active and the passive, or in a different way question and answer) The relation is based on comparing the individual co-occurrences of morphemes By investigating the individual co-occurrences (§ 12; § 2) we can characterize the distribution of certain classes which may not be definable in ordinary linguistic terms (e g pronouns, § 26) More important, we can then proceed to define transformation (§ 13), based on two structures having the same set of individual co-occurrences This relation yields unique analyses of certain structures and distinctions which could not be analyzed in ordinary linguistic terms (§ 3) It replaces a large part of the complexities of constituent analysis and sentence structure, at the cost of adding a level to grammatical analysis It also has various analytic and practical applications (§ 57), and can enter into a more algebraic analysis of language structure (§ 52, 4, 6) than is natural for the usual classificatory linguistics A list of English transformations is given in § 4 The main argument can be followed in § 111 (Co-Occurrence Defined), § 12 (Constructional Status), § 13 (Transformation Defined), § 29 (Summary of Constructions), § 39 (Summary of Sentence Sequences), §5 (The Place of Transformations in Linguistic Structure)1
253 citations