Showing papers in "Computational Linguistics in 1994"

An algorithm for pronominal anaphora resolution

Abstract: This paper presents an algorithm for identifying the noun phrase antecedents of third person pronouns and lexical anaphors (reflexives and reciprocals). The algorithm applies to the syntactic representations generated by McCord's Slot Grammar parser and relies on salience measures derived from syntactic structure and a simple dynamic model of attentional state. Like the parser, the algorithm is implemented in Prolog. The authors have tested it extensively on computer manual texts and conducted a blind test on manual text containing 360 pronoun occurrences. The algorithm successfully identifies the antecedent of the pronoun for 86% of these pronoun occurrences. The relative contributions of the algorithm's components to its overall success rate in this blind test are examined. Experiments were conducted with an enhancement of the algorithm that contributes statistically modelled information concerning semantic and real-world relations to the algorithm's decision procedure. Interestingly, this enhancement only marginally improves the algorithm's performance (by 2%). The algorithm is compared with other approaches to anaphora resolution that have been proposed in the literature. In particular, the search procedure of Hobbs' algorithm was implemented in the Slot Grammar framework and applied to the sentences in teh blind test set. The authors' algorithm achieves a higher rate of success (4%) than Hobbs' algorithm. The relation of the algorithm to the centering approach is discussed, as well as to models of anaphora resolution that invoke a variety of informational factors in ranking antecedent candidates.

Regular models of phonological rule systems

Abstract: This paper presents a set of mathematical and computational tools for manipulating and reasoning about regular languages and regular relations and argues that they provide a solid basis for computational phonology It shows in detail how this framework applies to ordered sets of context-sensitive rewriting rules and also to grammars in Koskenniemi's two-level formalism This analysis provides a common representation of phonological constraints that supports efficient generation and recognition by a single simple interpreter

Tagging English text with a probabilistic model

Abstract: In this paper we present some experiments on the use of a probabilistic model to tag English text, i.e. to assign to each word the correct tag (part of speech) in the context of the sentence. The main novelty of these experiments is the use of untagged text in the training of the model. We have used a simple triclass Markov model and are looking for the best way to estimate the parameters of this model, depending on the kind and amount of training data provided. Two approaches in particular are compared and combined:using text that has been tagged by hand and computing relative frequency counts,using text without tags and training the model as a hidden Markov process, according to a Maximum Likelihood principle.Experminents show that the best training is obtained by using as much tagged text as possible. They also show that Maximum Likelihood training, the procedure that is routinely used to estimate hidden Markov models parameters from training data, will not necessarily improve the tagging accuracy. In fact, it will generally degrade this accuracy, except when only a limited amount of hand-tagged text is available.

Word sense disambiguation using a second language monolingual corpus

Abstract: This paper presents a new approach for resolving lexical ambiguities in one language using statistical data from a monolingual corpus of another language. This approach exploits the differences between mappings of words to senses in different languages. The paper concentrates on the problem of target word selection in machine translation, for which the approach is directly applicable. The presented algorithm identifies syntactic relations between words, using a source language parser, and maps the alternative interpretations of these relations to the target language, using a bilingual lexicon. The preferred senses are then selected according to statistics on lexical relations in the target language. The selection is based on a statistical model and on a constraint propagation algorithm, which simultaneously handles all ambiguities in the sentence. The method was evaluated using three sets of Hebrew and German examples and was found to be very useful for disambiguation. The paper includes a detailed comparative analysis of statistical sense disambiguation methods.

Tracking point of view in narrative

Abstract: Third-person fictional narrative text is composed not only of passages that objectively narrate events, but also of passages that present characters' thoughts, perceptions, and inner states. Such passages take a character's psychological point of view. A language understander must determine the current psychological point of view in order to distinguish the beliefs of the characters from the facts of the story, to correctly attribute beliefs and other attitudes to their sources, and to understand the discourse relations among sentences. Tracking the psychological point of view is not a trivial problem, because many sentences are not explicitly marked for point of view, and whether the point of view of a sentence is objective or that of a character (and if the latter, which character it is) often depends on the context in which the sentence appears. Tracking the psychological point of view is the problem addressed in this work. The approach is to seek, by extensive examinations of naturally occurring narrative, regularities in the ways that authors manipulate point of view, and to develop an algorithm that tracks point of view on the basis of the regularities found. This paper presents this algorithm, gives demonstrations of an implemented system, and describes the results of some preliminary empirical studies, which lend support to the algorithm.

Machine translation divergences: a formal description and proposed solution

Abstract: There are many cases in which the natural translation of one language into another results in a very different form than that of the original. The existence of translation divergences (i.e., crosslinguistic distinctions) makes the straightforward transfer from source structures into target structures impractical. Many existing translation systems have mechanisms for handling divergent structures but do not provide a general procedure that takes advantage of takes advantage of the systematic relation between lexical-semantic structure and syntactic structure. This paper demonstrates that a systematic solution to the divergence problem can be derived from the formalization of two types of information: (1) the linguistically grounded classes upon which lexical-semantic divergences are based; and (2) the techniques by which lexical-semantic divergences are resolved. This formalization is advantageous in that it facilitates the design and implementation of the system, allows one to make an evaluation of the status of the system, and provides a basis for proving certain important properties about the system.

Japanese discourse and the process of centering

Abstract: This paper has three aims: (1) to generalize a compulational account of the discourse process called CENTERING, (2) to apply this account to discourse processing in Japanese so that it can be used in computational systems for machine translation or language understanding, and (3) to provide some insights on the effect of syntactic factors in Japanese on discourse interpretation. We argue that while discourse interpretation is an inferential process, syntactic cues constrain this process; we demonstrate this argument with respect to the interpretation of ZEROS, unexpressed arguments of the verb, in Japanese. The syntactic cues in Japanese discourse that we investigate are the morphological markers for grammatical TOPIC, the postposition wa, as well as those for grammatical functions such as SUBJECT, ga, OBJECT, o and OBJECT2, ni. In addition, we investigate the role of speaker's EMPATHY, which is the viewpoint from which an event is described. This is syntactically indicated through the use of verbal compounding, i.e. the auxiliary use of verbs such as kureta, kita. Our results are based on a survey of native speakers of their interpretation of short discourses, consisting of minimal pairs, varied by one of the above factors. We demonstrate that these syntactic cues do indeed affect the interpretation of ZEROS, but that having previously been the TOPIC and being realized as a ZERO also contributes to the salience of a discourse entity. We propose a discourse rule of ZERO TOPIC ASSIGNMENT, and show that CENTERING provides constraints on when a ZERO can be interpreted as the ZERO TOPIC.

A syntactic analysis method of long Japanese sentences based on the detection of conjunctive structures

Abstract: This paper presents a syntactic analysis method that first detects conjunctive structures in a sentence by checking parallelism of two series of words and then analyzes the dependency structure of the sentence with the help of the information about the conjunctive structures. Analysis of long sentences is one of the most difficult problems in natural language processing. The main reason for this difficulty is the structural ambiguity that is common for conjunctive structures that appear in long sentences. Human beings can recognize conjunctive structures because of a certain, but sometimes subtle, similarity that exists between conjuncts. Therefore, we have developed an algorithm for calculating a similarity measure between two arbitrary series of words from the left and the right of a conjunction and selecting the two most similar series of words that can reasonably be considered as composing a conjunctive structure. This is realized using a dynamic programming technique. A long sentence can be reduced into a shorter form by recognizing conjunctive structures. Consequently, the total dependency structure of a sentence can be obtained by relatively simple head-dependent rules. A serious problem concerning conjunctive structures, besides the ambiguity of their scopes, is the ellipsis of some of their components. Through our dependency analysis process, we can find the ellipses and recover the omitted components. We report the results of analyzing 150 Japanese sentences to illustrate the effectiveness of this method.

An alternative conception of tree-adjoining derivation

Abstract: The precise formulation of derivation for tree-adjoining grammars has important ramifications for a wide variety of uses of the formalism, from syntactic analysis to semantic interpretation and statistical language modeling We argue that the definition of tree-adjoining derivation must be reformulated in order to manifest the proper linguistic dependencies in derivations The particular proposal is both precisely characterizable through a definition of TAG derivations as equivalence classes of ordered derivation trees, and computationally operational, by virtue of a compilation to linear indexed grammars together with an efficient algorithm for recognition and parsing according to the compiled grammar

One-level phonology: autosegmental representations and rules as finite automata

Abstract: When phonological rules are regarded as declarative descriptions, it is possible to construct a model of phonology in which rules and representations are no longer distinguished and such procedural devices as rule-ordering are absent In this paper we present a finite-state model of phonology in which automata are the descriptions and tapes (or strings) are the objects being described This provides the formal semantics for an autosegmental phonology without structure-changing rules Logical operations on the phonological domain---such as conjunction, disjunction, and negation---make sense since the phonological domain consists of descriptions rather than objects These operations as applied to automata are the straightforward operations of intersection, union, and complement If the arrow in a rewrite rule is viewed as logical implication, then a phonological rule can also be represented as an automaton, albeit a less restrictive automaton than would be required for a lexical representaton The model is then compared with the transducer models for autosegmental phonology of Kay (1987), Kornai (1991), and Wiebe (1992) We conclude that the declarative approach to phonology presents an attractive way of extending finite-state techniques to autosegmental phonology while remaining within the confines of regular grammar

A hierarchical stochastic model for automatic prediction of prosodic boundary location

Abstract: Prosodic phrase structure provides important information for the understanding and naturalness of synthetic speech, and a good model of prosodic phrases has applications in both speech synthesis and speech understanding. This work describes a statistical model of an embedded hierarchy of prosodic phrase structure, motivated by results in linguistic theory. Each level of the hierarchy is modeled as a sequence of subunits at the next level, with the lowest level of the hierarchy representing factors such as syntactic branching and prosodic constituent length using a binary tree classification. A maximum likelihood solution for parameter estimation is presented, allowing automatic training of different speaking styles. For predicting prosodic phrase breaks from text, a dynamic programming algorithm is given for finding the maximum probability prosodic parse. Experimental results on a corpus of radio news demonstrate a high rate of success for predicting major and minor phrase boundaries from text without syntactic information (81% correct prediction with 4% false prediction).

The acquisition of stress: a data-oriented approach

Abstract: A data-oriented (empiricist) alternative to the currently pervasive (nativist) Principles and Parameters approach to the acquisition of stress assignment is investigated. A similarity-based algorithm, viz. an augmented version of Instance-Based Learning is used to learn the system of main stress assignment in Dutch. In this nontrivial task a comprehensive lexicon of Dutch monomorphemes is used instead of the idealized and highly simplified description of the empirical data used in previous approaches.It is demonstrated that a similarity-based learning method is effective in learning the complex stress system of Dutch. The task is accomplished without the a priori knowledge assumed to pre-exist in the learner in a Principles and Parameters framework.A comparison of the system's behavior with a consensus linguistic analysis (in the framework of Metrical Phonology) shows that ease of learning correlates with decreasing degrees of markedness of metrcal phenomena. It is also shown that the learning algorithm captures subregularities within the stress system of Dutch that cannot be described without going beyond some of the theoretical assumptions of metrical phonology.

A hierarchical stochastic model for automatic prediction of prosodic boundary location

Abstract: Prosodic phrase structure provides important information for the understanding and naturalness of synthetic speech, and a good model of prosodic phrases has applications in both speech synthesis an

Training and scaling preference functions for disambiguation

Abstract: We present an automatic method for weighting the contributions of preference functions used in disambiguation. Initial scaling factors are derived as the solution to a least squares minimization problem, and improvements are then made by hill climbing. The method is applied to disambiguating sentences in the Air Travel Information System corpus, and the performance of the resulting scaling factors is compared with hand-tuned factors. We then focus on one class of preference function, those based on semantic lexical collocations. Experimental results are presented showing that such functions vary considerably in selecting correct analyses. In particular, we define a function that performs significantly better than ones based on mutual information and likelihood ratios of lexical associations.

Phonological analysis in typed feature systems

Abstract: Research on constraint-based grammar frameworks has focused on syntax and semantics largely to the exclusion of phonology. Likewise, current developments in phonology have generally ignored the technical and linguistic innovations available in these frameworks. In this paper we suggest some strategies for reuniting phonology and the rest of grammar in the context of a uniform constraint formalism. We explain why this is a desirable goal, and we present some conservative extensions to current practice in computational linguistics and in nonlinear phonology that we believe are necessary and sufficient for achieving this goal.We begin by exploring the application of typed feature logic to phonology and propose a system of prosodic types. Next, taking HPSG as an exemplar of the grammar frameworks we have in mind, we show how the phonology attribute can be enriched so that it can encode multi-tiered, hierarchical phonological representations. Finally, we exemplify the approach in some detail for the nonconcatenative morphology of Sierra Miwok and for schwa alternation in French. The approach taken in this paper lends itself particularly well to capturing phonological generalizations in terms of high-level prosodic constraints.

RAFT/RAPR and centering: a comparison and discussion of problems related to processing complex sentences

Abstract: preferences underlying the frameworks share much in common, the approaches very often make the same predictions. RAFT/RAPR's focusing and pronoun resolution algorithms reflect the following underlying preferences: We prefer for the CF to be something that is co-referential with an element mentioned earlier in the discourse. We prefer for the CF to be a pronoun rather than a full definite description. We use preferences for resolving pronouns and computing the CF that involve the grammatical role of the pronoun. We prefer for the CF to be the same as the last CF and the SF to be the same as the last SF. Centering shares the first two of these in common with us. But, centering prefers the local focus to stay the same and for the subject to be the local focus (which is stronger than preference 4). As noted earlier, we do not apply preference 2 without regard for the grammatical roles of the NPs and which focusing data structures they co-specify. Furthermore, centering invokes the second preference during pronoun resolution, while we invoke it only after resolving the pronouns. In addition, centering requires that the focus (Cb) realizes an element in the immediately preceding sentence, as opposed to merely preferring for the focus to be something in the immediately preceding sentence but allowing it instead to be co-referential with an element further back in the preceding text (as RAFT/RAPR does). In sum, the order in which these preferences are applied differs between the two frameworks. Furthermore, as a result of (some) differences between the frameworks with respect to these preferences, resolvhlg a pronoun to have the same grammatical role as its antecedent is less important in centering than it is in RAFT/RAPR (Section 4.3). RAFT/RAPR presents possible referents for pronouns one possibility at a time, and if pragmatic, semantic, and general knowledge inferencing rejects a referent, RAFT/RAPR proposes an alternative. Centering, on the other hand, (in addition to requiring this same kind of inferencing) sometimes suggests multiple possibilities for the co-specifications of pronouns in a sentence (Walker 1989). Is pragmatic inferencing applied at this point to pick among the possible antecedents? We assume this to be the case, otherwise it is not clear how centering will choose among the multiple potential referents. Yet, assuming this is the case, centering seems to involve more complex inferencing than our approach involves; inferencing for centering must pick which antecedent is better. RAFT/RAPR only asks inferencing to confirm a co-specification. 19 19 We do not check for ambiguity of pronouns in the same way that Sidner did. However, even if we were to incorporate similar checks for ambiguity, such checks would involve using inferencing to confirm two possible co-specifications for a pronoun, while centering might require the use of inferencing to confirm more than two. Thus, the inferencing required by RAFT/RAPR would still be more limited than that required by centering. 315

Tree-adjoining grammar parsing and boolean matrix multiplication

Abstract: The computational problem of parsing a sentence in a tree-adjoining language is investigated. An interesting relation is studied between this problem and the well-known computational problem of Boolean matrix multiplication: it is shown that any algorithm for the solution of the former problem can easily be converted into an algorithm for the solution of the latter problem. This result bears on at least two important computational issues. First, we realize that a straightforward method that improves the known upper bound for tree-adjoining grammar parsing is hard to find. Second, we understand which features of the tree-adjoining grammar parsing problem are responsible for the claimed difficulty.

The acquisition of stress

Abstract: A data-oriented (empiricist) alternative to the currently pervasive (nativist) Principles and Parameters approach to the acquisition of stress assignment is investigated. A similarity-based algorit...

The reconstruction engine: a computer implementation of the comparative method

Abstract: We describe the implementation of a computer program, the Reconstruction Engine (RE), which models the comparative method for establishing genetic affiliation among a group of languages. The program is a research tool designed to aid the linguist in evaluating specific hypotheses, by calculating the consequences of a set of postulated sound changes (proposed by the linguist) on complete lexicons of several languages. It divides the lexicons into a phonologically regular part and a part that deviates from the sound laws. RE is bi-directional: given words in modern languages, it can propose cognate sets (with reconstructions); given reconstructions, it can project the modern forms that would result from regular changes. RE operates either interactively, allowing word-by-word evaluation of hypothesized sound changes and semantic shifts, or in a "batch" mode, processing entire multilingual lexicons en masse.We describe the algorithms implemented in RE, specifically the parsing and combinatorial techniques used to make projections upstream or downstream in the sense of time, the procedures for creating and consolidating cognate sets based on these projections, and the ad hoc techniques developed for handling the semantic component of the comparative method.Other programs and computational approaches to historical linguistics are briefly reviewed.Some results from a study of the Tamang languages of Nepal (a subgroup of the Tibeto-Burman family) are presented, and data from these languages are used throughout for exemplification of the operation of the program.Finally, we discuss features of RE that make it possible to handle the complex and sometimes imprecise representations of lexical items, and speculate on possible directions for future research.

One-level phonology

Abstract: When phonological rules are regarded as declarative descriptions, it is possible to construct a model of phonology in which rules and representations are no longer distinguished and such procedural...

Computing with features as formulae

Abstract: This paper extends the approach to feature structures developed in Johnson (1991a), which uses Schonfinkel-Bernays' formulae to express feature structure constraints. These are shown to be a disjunctive generalization of Datalog clauses, as used in database theory. This paper provides a fixed-point characterization of the minimal models of these formulae that serves as the theoretical foundation of a forward-chaining algorithm for determining their satisfiability. This algorithm, which generalizes the standard attribute-value unification algorithm, is also recognizable as a nondeterministic variant of the semi-naive bottom-up algorithm for evaluating Datalog programs, further strengthening the connection between the theory of feature structures and databases.

Machine translation divergences

Abstract: There are many cases in which the natural translation of one language into another results in a very different form than that of the original. The existence of translation divergences (i.e., crossl...

Parsing and empty nodes

Abstract: This paper describes a method for ensuring the termination of parsers using grammars that freely posit empty nodes. The basic idea is that each empty node must be associated with a lexical item appearing in the input string, called its sponsor. A lexical item, as well as labeling the node for the corresponding word, provides labels for a fixed number, possibly zero, of empty nodes. The number of nodes appearing in the parse tree is thus bounded before parsing begins. Termination follows trivially. The technique is applicable to any standard parsing algorithm.

Word sense disambiguation using a second language monolingual corpus

Abstract: This paper presents a new approach for resolving lexical ambiguities in one language using statistical data from a monolingual corpus of another language. This approach exploits the differences bet...

Introduction to computational phonology

Abstract: Despite being the oldest discipline in linguistics, phonology remains largely unexplored from a computational standpoint. While phonology gave us such innovations as the ‘distinctive feature’, now heavily used in computational linguistics, phonology itself is yet to reap the benefits of the formal and technological developments it gave rise to. Recently however, computational phonology has been rapidly gaining recognition as an independent area of inquiry within computational linguistics. The ACL Special Interest Group inComputational Phonology (SIGPHON)was formed in 1991and has served as a focus for ongoing work in the area. In June of that year I proposed that there be a special issue of Computational Linguistics dedicated to computational phonology, since there were many good-quality papers in circulation which had no obvious venue for publication. The resulting collection which you have before you is a representative sample of this work; some submissions not ready in time for this volume will appear in subsequent regular issues. Other work in this area is to be found in the Proceedings of the First Meeting of the ACL Special Interest Group in Computational Phonology, published by the ACL in 1994, and two edited collections (Bird 1991, Ellison & Scobbie 1993). The purposeof this short piece is to introduce computationalphonology and the special issue. I shall begin by presenting some background to the field, followed by a survey of the research themes currently under investigation. Next, an overview of the papers in this collection is given, concluding with an explanation of the one-page commentaries which follow each paper. So, what is phonology, and why should computational linguists care about it?

Tagging English text with a probabilistic model

Abstract: In this paper we present some experiments on the use of a probabilistic model to tag English text, i.e. to assign to each word the correct tag (part of speech) in the context of the sentence. The m...

Stroing logical form in a shared-packed forest

Abstract: To co~npactly represent sentences containing syntactic ambiguity until the necessary inform.ation has been processed t o refine the meaning, we have modified an all-path context-free grammar parser t o generate a shared-packed parse forest annotated with logical form. An annotated shared-packed forest cannot contain every representation of a highly ambiguous sentence without using an intractable amount of space. Hence, our progra,m stores procedure calls for creating all possible logical forms for a c.onstituent in the forest. The resulting forest contains the same number of nodes and is not much bigger than the original forest. Furthermore, the stored procedural information can be used by a program to construct representations for any of the constituents in. the forest for subsequent testing against a world model. After performing each test, the program can incrementally prune the forest of ambiguity.

Commentary on Kaplan and Kay

Abstract: Anyone with a fundamental interest in morphology and phonology, either from a scientific or a computational perspective, will want to study this long-awaited paper carefully. Kaplan and Kay (henceforth K&K) announce two goals: \"to provide the core of a mathematical framework for phonology\" and \"to establish a solid basis for computation in the domain of phonological and orthographic systems.\" They show how the algebra of regular relations, with their corresponding automata, can be used to compile systems of phonological rules in the style of SPE, including directionality, optionality, and ordering. They sketch mechanisms for incorporating a lexicon and for dealing with exceptional forms, thus providing a complete treatment in a unified framework. This accomplishment in itself will not compel the attention of many working phonologists, who have found good reasons to replace the SPE framework (see Kenstowicz [1994] for a survey of modern practice), and whose efforts since 1975 have been aimed mainly at finding representational primitives to explain typological generalizations, support accounts of learning, generalization and change, and provide one end of the mapping between symbols and speech. In this effort, there has been little emphasis on SPE's goal of giving phonological descriptions an algorithmically specified denotation. Perhaps this paper, despite its superficial lack of connection to contemporary work in phonology, will set in motion a discussion that will ultimately redress the balance. On the computational side, practitioners of practical NLP will be happy to make extensive use of the algebra of regular relations, since it provides a truly elegant engineering solution to a wide range of problems. However, although direct interpretation of some simple FSTs can be efficient (e.g. Feigenbaum et al. 1991), and although Koskenniemi has documented efficient implementation techniques for his two-level systems, the overall architecture presented in this paper is not practically usable as written, because of either the size of the resulting automata or the time needed for (unwisely implemented) nondeterminism, or both. A range of well-known techniques enable programs based on the algebraic combination of (unary) FSAs to make efficient use of both time and space. Although these methods do not apply to FSTs in general, we may presume that K&K have developed analogous techniques for the crucial range of cases. With the growing interest in this technology, we can expect that either K&K will publish their work or others will recapitulate it, so that the algebra of regular relations can take its proper and prominent place in the toolkit of computational linguistics.