Showing papers on "Tree-adjoining grammar published in 1999"

Estimators for Stochastic "Unification-Based" Grammars

Abstract: Log-linear models provide a statistically sound framework for Stochastic "Unification-Based" Grammars (SUBGs) and stochastic versions of other kinds of grammars. We describe two computationally-tractable ways of estimating the parameters of such grammars from a training corpus of syntactic analyses, and apply these to estimate a stochastic version of Lexical-Functional Grammar.

Semiring parsing

Abstract: We synthesize work on parsing algorithms, deductive parsing, and the theory of algebra applied to formal languages into a general system for describing parsers. Each parser performs abstract computations using the operations of a semiring. The system allows a single, simple representation to be used for describing parsers that compute recognition, derivation forests, Viterbi, n-best, inside values, and other values, simply by substituting the operations of different semirings. We also show how to use the same representation, interpreted differently, to compute outside values. The system can be used to describe a wide variety of parsers, including Earley's algorithm, tree adjoining grammar parsing, Graham Harrison Ruzzo parsing, and prefix value computation.

Efficient Parsing for Bilexical Context-Free Grammars and Head Automaton Grammars

Abstract: Several recent stochastic parsers use bilexical grammars, where each word type idiosyncratically prefers particular complements with particular head words. We present O(n4) parsing algorithms for two bilexical formalisms, improving the prior upper bounds of O(n5). For a common special case that was known to allow O(n3) parsing (Eisner, 1997), we present an O(n3) algorithm with an improved grammar constant.

Relating Probabilistic Grammars and Automata

Abstract: Both probabilistic context-free grammars (PCFGs) and shift-reduce probabilistic pushdown automata (PPDAs) have been used for language modeling and maximum likelihood parsing. We investigate the precise relationship between these two formalisms, showing that, while they define the same classes of probabilistic languages, they appear to impose different inductive biases.

Shape Grammars: Six Types

Abstract: The issue of decidability in relation to shape grammars is considered here. Decidability concerns, first, the identification of different types of grammars and, second, the answerability or solvability of questions about these types of grammars. In this paper, the first of these two topics is examined. Six different types of shape grammars are defined by considering different kinds of restrictions on rule format and rule ordering. The effects that these different restrictions have on the generative power, practicality, pedagogical value, and other characteristics of a shape grammar are discussed. In a subsequent paper, “Shape grammars: five questions” (Knight, 1998), the answerabilities of various questions about the types of shape grammars outlined here are explored. The decidability issues addressed in this paper and the subsequent one are key to the practical use of shape grammars in design projects where specific goals and constraints need to be satisfied.

Parsing Minimalist Languages

Abstract: In this paper I will present a formal specification of a recognizer for languages generated by Minimalist Grammars (Stabler, 1997). Minimalist Grammars are simple, formal grammars modeling some important aspects of the kind of grammars developed in the framework of Chomsky's Minimalist Program (Chomsky, 1995). A Minimalist Grammar is defined by a set of lexical items, which varies from language to language, and two universal structure building functions, which are defined on trees or configurations: merge and move. Michaelis (1998) has shown that the set of languages generated by Minimalist Grammars is mildly context-sensitive: it falls properly between the set of context-sensitive languages and the set of context-free languages. Mildly context-sensitive languages are assumed to be appropriately powerful for the description of natural languages (Joshi, 1985). Minimalist Grammars can move material from positions arbitrarily deep inside a sentence. This property contributes to the non-context-free nature of these grammars. Michaelis' equivalence result (Michaelis, 1998) permits a representation of Minimalist Grammars in which the operations of the grammar are characterized in such a way that they are strictly local. In this representation, configurations are reduced to those properties that determine their behavior with regard to the structure building functions merge and move. This representation will be used in this paper to derive a top-down recognizer for languages generated by Minimalist Grammars. The recognizer starts from the assumption that the sentence to be parsed is actually a grammatical sentence, and then tries to disassemble it into lexical items by repeatedly applying the structure building functions merge and move in reverse. The recognizer presented in this paper has the correct-prefix property: it goes through the input sentence from left to right and, in case of an ungrammatical sentence, it will halt at the first word that does not fit into a grammatical structure, i.e., the recognizer will not go beyond a prefix that cannot be extended to a grammatical sentence in the language. Similarly, the parser of the human sentence processor detects an ungrammaticality at the first word that makes a sentence ungrammatical, and for garden path sentences it will generally hesitate at the first word that does not fit into the structure that has been hypothesized for the sentence. This is a computationally advantageous property for a recognizer to have, because it prevents the recognizer from spending any effort on a sentence once it is known that it is ungrammatical. Besides contributing to a deeper understanding of Minimalist Grammars and theories that can be formalized in a similar fashion, e.g. Strict Asymmetry Grammars (e.g. Di Sciullo, 1999), the work reported in this paper may also be relevant for psycholinguistic inquiries. In most psycholinguistic proposals, the operations of the human parser are informally sketched in the context of a small set of example sentences, leaving open the question whether a parser with the desired properties covering the entire language actually exists. Another drawback of some psycholinguistic work is that it is based on simplistic and outdated conceptions of syntactic structure. Having a formal, sound and complete parsing model for Minimalist Grammars may help to remedy these problems.

Optimal code selection in DAGs

Abstract: We extend the tree parsing approach to code selection to DAGs In general, our extension does not produce the optimal code selection for all DAGs (this problem would be NP-complete), but for certain code selection grammars, it does We present a method for checking whether a code selection grammar belongs to this set of DAG-optimal grammars, and use this method to check code selection grammars adapted from lcc: the grammars for the MIPS and SPARC architectures are DAG-optimal, and the code selection grammar for the 386 architecture is almost DAG-optimal

Tabular algorithms for TAG parsing

Abstract: We describe several tabular algorithms for Tree Adjoining Grammar parsing, creating a continuum from simple pure bottom-up algorithms to complex predictive algorithms and showing what transformations must be applied to each one in order to obtain the next one in the continuum.

Extensions of Attribute Grammars for Structured Document Queries

Abstract: Document specification languages like XML, model documents using extended context-free grammars. These differ from standard context-free grammars in that they allow arbitrary regular expressions on the right-hand side of productions. To query such documents, we introduce a new form of attribute grammars (extended AGs) that work directly over extended context-free grammars rather than over standard context-free grammars. Viewed as a query language, extended AGs are particularly relevant as they can take into account the inherent order of the children of a node in a document. We show that two key properties of standard attribute grammars carry over to extended AGs: efficiency of evaluation and decidability of well-definedness. We further characterize the expressiveness of extended AGs in terms of monadic second-order logic and establish the complexity of their non-emptiness and equivalence problem to be complete for EXPTIME. As an application we show that the Region Algebra expressions can be efficiently translated into extended AGs. This translation drastically improves the known upper bound on the complexity of the emptiness and equivalence test for Region Algebra expressions.

Generalized P-Systems

Abstract: We consider a variant of P-systems, a new model for computations using membrane structures and recently introduced by Gheorghe Paun. Using the membranes as a kind of filter for specific objects when transferring them into an inner compartment turns out to be a very powerful mechanism in combination with suitable rules to be applied within the membranes. The model of generalized P-systems, GP-systems for short, considered in this paper allows for the simulation of graph controlled grammars of arbitrary type based on productions working on single objects; for example, the general results we establish in this paper can immediately be applied to the graph controlled versions of context-free string grammars, n-dimensional #-context-free array grammars, and elementary graph grammars.

Deductive systems and grammars: proofs as grammatical structures

Abstract: During the last fifteen years, much of the research of proof theoretical grammars has been focused on their weak generative capacity. This research culminated in Pentus’ theorem, which showed that Lambek grammars generate precisely the context-free languages. However, during the same period of time, research on other grammar formalisms has stressed the importance of “strong generative capacity,” i.e. the derivation or phrase structure trees that grammars assign to strings. The first topic of this thesis is the strong generative capacity of Lambek grammars. The proof theoretic perspective on grammars allows us to consider different notions of what “structure assigned by a Lambek grammar to a string” is taken to mean. For example, we can take any proof tree that establishes that a grammar generates a certain string or only those that are in some normal form. It can be shown that the formal properties of these notions of structure differ. The main result of this part of the thesis is that, although Lambek grammars generate context-free string languages, their derivation trees are more complex than those of context-free grammars. The latter were characterized by Thatcher as coinciding with the local tree languages, while the derivation trees of Lambek grammars include tree languages which are not regular. Even non-associative Lambek grammars, which recently have become more popular variants of categorial grammar, can be used to generate non-local tree languages. However, their normal form tree languages are always regular. Finally, categorial grammars lacking introduction rules have local derivation trees. Thus, there is a genuine hierarchy of proof theoretical grammars with respect to strong generative capacity. Additionally, we consider the semantic aspect of the proof theoretic approach to language, which is given by the correspondence between proof theory and type

A Bootstrap Method for Constructing Local Grammars

Abstract: Local grammars are finite-state grammars or finite-state automata that represent sets of utterances of a natural language. Local grammars have been used to describe a wide variety of sets of strings, ranging from finite sets of words related by prefixation and suffixation to sets of sentences syntactically and semantically related. Formal representations of finite-state grammars are quite varied, although equivalent. We have chosen a particular representation by graphs which is well-adapted to the syntax of natural languages. Using a particular example, we will present in detail a method for constructing local grammars using large corpora.

Grammatical Inference as Unification

Abstract: We propose to set the grammatical inference problem in a logical framework. The search for admissible solutions in a given class of languages is reduced to the problem of unifying a set of terms. This point of view has been already developed in the particular context of categorial grammars, a type of lexicalized grammar. We present the state of the art in this domain and propose several improvements. The case of regular grammars is studied in a second part. We show that rational unification allows to infer such grammars. We give corresponding Prolog programs in both cases. Indeed, one of the aim of this work is to show that "lean" programs are possible for grammatical inference. This approach has been successful in the field of automated theorem proving and we expect to observe the same benefits in grammatical inference : efficiency and extendibility.

The Turing Completeness of Multimodal Categorial Grammars

Abstract: In this paper, we demonstrate that the multimodal categorial grammars are in fact Turing-complete in their weak generative capacity The result follows from a straightforward reduction of generalized rewriting systems to a mixed associative and modal categorial calculus We conclude with a discussion of a restriction to the so-caled weak Sahlqvist lexical rules, for which we can ensure decidability

Chinese numbers, MIX, scrambling, and range concatenation grammars

Abstract: The notion of mild context-sensitivity was formulated in an attempt to express the formal power which is both necessary and sufficient to define the syntax of natural languages. However, some linguistic phenomena such as Chinese numbers and German word scrambling lie beyond the realm of mildly context-sensitive formalisms. On the other hand, the class of range concatenation grammars provides added power w.r.t. mildly context-sensitive grammars while keeping a polynomial parse time behavior. In this report, we show that this increased power can be used to define the above-mentioned linguistic phenomena with a polynomial parse time of a very low degree.

Elimination of left recursion from context-free grammars

Abstract: A method for transforming a first set of rule expressions forming a first grammar to a second set of rule expressions forming a second grammar includes identifying at least one left-recursive category of the first grammar; and applying a left-corner transform to substantially only the left-recursive rule expressions of the first grammar in forming the second grammar.

The Simple Language Generator: Encoding Complex Languages With Simple Grammars

Abstract: : This paper introduces the design and use of the Simple Language Generator (SLG). SLG allows the user to construct small but interesting stochastic context free languages with relative ease. Although context free grammars are convenient for representing natural language syntax, they do not easily support the semantic and pragmatic constraints that make certain combinations of words or structures more likely than others. Context free grammars for languages involving many interacting constraints can become extremely complex and cannot reasonably be written by hand. SLG allows the basic syntax of a grammar to be specified in context free form and constraints to be applied atop this framework in a relatively natural fashion. This combination of grammar and constraints is then converted into a standard stochastic context free grammar for use in generating sentences or in making context dependent likelihood predictions of the sequence of words in a sentence.

On the leftmost derviation in matrix grammars

Abstract: Matrix grammars are one of the classical topics of formal languages, more specifically, regulated rewriting. Although this type of control on the work of context-free grammars is one of the earliest, matrix grammars still raise interesting questions (not to speak about old open problems in this area). One such class of problems concerns the leftmost derivation (in grammars without appearance checking). The main point of this paper is the systematic study of all possibilities of defining leftmost derivation in matrix grammars. Twelve types of such a restriction are defined, only four of which being discussed in literature. For seven of them, we find a proof of a characterization of recursively enumerable languages (by matrix grammars with arbitrary context-free rules but without appearance checking). Other three cases characterize the recursively enumerable languages modulo a morphism and an intersection with a regular language. In this way, we solve nearly all problems listed as open on page 67 of the monograph [7], which can be seen as the main contribution of this paper. Moreover, we find a characterization of the recursively enumerable languages for matrix grammars with the leftmost restriction defined on classes of a given partition of the nonterminal alphabet.

The role of lexicalization and pruning for base noun phrase grammars

Abstract: This paper explores the role of lexicalization and pruning of grammars for base noun phrase identification. We modify our original framework (Cardie & Pierce 1998) to extract lexicalized treebank grammars that assign a score to each potential noun phrase based upon both the part-of-speech tag sequence and the word sequence of the phrase. We evaluate the modified framework on the "simple" and "complex" base NP corpora of the original study. As expected, we find that lexicalization dramatically improves the performance of the unpruned treebank grammars; however, for the simple base noun phrase data set, the lexicalized grammar performs below the corresponding unlexicalized but pruned grammar, suggesting that lexicalization is not critical for recognizing very simple, relatively unambiguous constituents. Somewhat surprisingly, we also find that error-driven pruning improves the performance of the probabilistic, lexicalized base noun phrase grammars by up to 1.0% recall and 0.4% precision, and does so even using the original pruning strategy that fails to distinguish the effects of lexicalization. This result may have implications for many probabilistic grammar-based approaches to problems in natural language processing: error-driven pruning is a remarkably robust method for improving the performance of probabilistic and non-probabilistic grammars alike.

Shape Grammars: Five Questions:

Abstract: In the paper “Shape grammars: six types”, the issue of decidability in relation to shape grammars was introduced. Decidability concerns, first, the identification of different types of grammars, and, second, the answerability or solvability of questions about these types of grammars. The first of these two topics was explored in “Six types”. Six different types of shape grammars were defined in terms of different restrictions on rule format and rule ordering. The generative power, practicality, pedagogical value, and other characteristics of each type of shape grammars were discussed. In this paper, the second of the two topics in decidability is addressed. Five questions about the different types of shape grammars defined in “Six types” are posed. These questions are formulated for their practical value in design applications of shape grammars, as well as their theoretical interest. The answerability of each question is examined in detail for each type of shape grammar.

Preserving Semantic Dependencies in Synchronous Tree Adjoining Grammar

Abstract: Rambow, Wier and Vijay-Shanker (Rambow et al., 1995) point out the differences between TAG derivation structures and semantic or predicate-argument dependencies, and Joshi and Vijay-Shanker (Joshi and Vijay-Shanker, 1999) describe a monotonic compositional semantics based on attachment order that represents the desired dependencies of a derivation without underspecifying predicate-argument relationships at any stage. In this paper, we apply the Joshi and Vijay-Shanker conception of compositional semantics to the problem of preserving semantic dependencies in Synchronous TAG translation (Shieber and Schabes, 1990; Abeille et al., 1990). In particular, we describe an algorithm to obtain the semantic dependencies on a TAG parse forest and construct a target derivation forest with isomorphic or locally non-isomorphic dependencies in O (n7) time.

A hierarchy result for random forbidding context picture grammars

Abstract: We use random context picture grammars to generate pictures through successive refinement. The productions of such a grammar are context-free, but their application is regulated — "permitted" or "forbidden" — by context randomly distributed in the developing picture. Grammars using this relatively weak context often succeed where context-free grammars fail, e.g. in generating the Sierpinski carpets. On the other hand it proved possible to develop iteration theorems for three subclasses of these grammars, namely a pumping–shrinking, a pumping and a shrinking lemma for context-free, random permitting and random forbidding context picture grammars, respectively. Finding necessary conditions is problematic in the case of most models of context-free grammars with context-sensing ability, since they consider a variable and its context as a finite connected array. We have already shown that context-free picture grammars are strictly weaker than both random permitting and random forbidding context picture grammars, also that random permitting context is strictly weaker than random context. We now show that grammars which use forbidding context only are strictly weaker than random context picture grammars.

Parsing with an extended domain of locality

Abstract: One of the claimed benefits of Tree Adjoining Grammars is that they have an extended domain of locality (EDOL). We consider how this can be exploited to limit the need for feature structure unification during parsing. We compare two wide-coverage lexicalized grammars of English, LEXSYS and XTAG, finding that the two grammars exploit EDOL in different ways.

Evaluating two methods for Treebank grammar compaction

Abstract: Treebanks, such as the Penn Treebank, provide a basis for the automatic creation of broad coverage grammars. In the simplest case, rules can simply be ‘read off’ the parse-annotations of the corpus, producing either a simple or probabilistic context-free grammar. Such grammars, however, can be very large, presenting problems for the subsequent computational costs of parsing under the grammar. In this paper, we explore ways by which a treebank grammar can be reduced in size or ‘compacted’, which involve the use of two kinds of technique: (i) thresholding of rules by their number of occurrences; and (ii) a method of rule-parsing, which has both probabilistic and non-probabilistic variants. Our results show that by a combined use of these two techniques, a probabilistic context-free grammar can be reduced in size by 62% without any loss in parsing performance, and by 71% to give a gain in recall, but some loss in precision.

Parallel Composition of Graph Grammars.

Abstract: The specification of complex systems is usually done by the ‘divide and conquer’ idea: the system is divided into smaller, less complex components that are developed separately and then merged in some way to form the specification of the whole system. The main aim of this paper is to provide an approach to the parallel composition of graph grammars, formalizing the intuitive idea of ‘divide and conquer’ described above. This parallel composition of graph grammars provides a suitable formalism for the specification of concurrent systems based on the specifications of their components. ‘Dividing’ is formalized by special graph grammar morphisms, called specialization morphisms. These morphisms also describe structural and behavioural compatibilities between graph grammars. As a main result, we characterize the parallel composition as the pullback in the category of graph grammars.

Separating words with small grammars

Abstract: We study the following problem: given two words w and x, with jwj; jxj n, what is the size of the smallest context-free grammar G which generates exactly one of fw; xg? If jwj 6 = jxj, then we prove there exists a G separating w from x of size O(log log n), and this bound is best possible. If jwj = jxj, then we get an upper bound on the size of G of O(log n), and a lower bound of (logn loglog n).

Logical aspects of computational linguistics : second international conference, LACL '97, Nancy, France, September 22-24, 1997 : selected papers

Abstract: Invited papers.- Type Grammar Revisited.- Optimal Parameters.- Selected papers.- Strong Equivalence of Generalized Ajdukiewicz and Lambek Grammars.- Linguistic, Philosophical, and Pragmatic Aspects of Type-Directed Natural Language Parsing.- Derivational and Representational Views of Minimalist Transformational Grammar.- The MSO Logic-Automaton Connection in Linguistics.- The Logic of Tune A Proof-Theoretic Analysis of Intonation.- A Linear Logic Treatment of Phrase Structure Grammars For Unbounded Dependencies.- Underspecification in Type-Logical Grammars.- On Fibring Feature Logics with Concatenation Logics.- An Operational Model for Parsing De.nite Clause Grammars with In.nite Terms.- Mathematical Vernacular and Conceptual Well-Formedness in Mathematical Language.