Showing papers by "Taro Watanabe published in 2015"

Transition-based Neural Constituent Parsing

Abstract: Constituent parsing is typically modeled by a chart-based algorithm under probabilistic context-free grammars or by a transition-based algorithm with rich features. Previous models rely heavily on richer syntactic information through lexicalizing rules, splitting categories, or memorizing long histories. However enriched models incur numerous parameters and sparsity issues, and are insufficient for capturing various syntactic phenomena. We propose a neural network structure that explicitly models the unbounded history of actions performed on the stack and queue employed in transition-based parsing, in addition to the representations of partially parsed tree structure. Our transition-based neural constituent parsing achieves performance comparable to the state-of-the-art parsers, demonstrating F1 score of 90.68% for English and 84.33% for Chinese, without reranking, feature templates or additional data to train model parameters.

Leave-one-out Word Alignment without Garbage Collector Effects

Abstract: Expectation-maximization algorithms, such as those implemented in GIZA++ pervade the field of unsupervised word alignment. However, these algorithms have a problem of over-fitting, leading to “garbage collector effects,” where rare words tend to be erroneously aligned to untranslated words. This paper proposes a leave-one-out expectationmaximization algorithm for unsupervised word alignment to address this problem. The proposed method excludes information derived from the alignment of a sentence pair from the alignment models used to align it. This prevents erroneous alignments within a sentence pair from supporting themselves. Experimental results on Chinese-English and Japanese-English corpora show that the F1, precision and recall of alignment were consistently increased by 5.0% ‐ 17.2%, and BLEU scores of end-to-end translation were raised by 0.03 ‐ 1.30. The proposed method also outperformed l0-normalized GIZA++ and Kneser-Ney smoothed GIZA++.

Word alignment score computing apparatus, word alignment apparatus, and computer program

Abstract: [Object] An object is to provide an apparatus for attaining highly precise word alignment. [Solution] The apparatus includes: selecting means receiving a bilingual sentence pair and a word alignment for the bilingual sentence pair, for successively selecting words f j of a sentence in a first language in a prescribed order; and a recurrent neural network (RNN) 100 , computing, for all words of the sentence in the first language, a score 102 representing a probability that a word pair consisting of the word f j and a word e a _ {j} aligned with the word f j by a word alignment a j in a second language of the bilingual sentence pair is a correct word pair, and based on this score, for computing a score of the word alignment a j . When computing a score of word pair (f j , e a _ {j} ), RNN 100 computes a score 102 of the word pair (f j , e a _ {j} ) based on all word alignments a 1 j-1 selected by the selecting means prior to the word f j of the word pair (f j , e a _ {j} ), of the word alignments a j , by means of a recurrent connection 118.

Hierarchical Back-off Modeling of Hiero Grammar based on Non-parametric Bayesian Model

Abstract: In hierarchical phrase-based machine translation, a rule table is automatically learned by heuristically extracting synchronous rules from a parallel corpus. As a result, spuriously many rules are extracted which may be composed of various incorrect rules. The larger rule table incurs more run time for decoding and may result in lower translation quality. To resolve the problems, we propose a hierarchical back-off model for Hiero grammar, an instance of a synchronous context free grammar (SCFG), on the basis of the hierarchical Pitman-Yor process. The model can extract a compact rule and phrase table without resorting to any heuristics by hierarchically backing off to smaller phrases under SCFG. Inference is efficiently carried out using two-step synchronous parsing of Xiao et al., (2012) combined with slice sampling. In our experiments, the proposed model achieved higher or at least comparable translation quality against a previous Bayesian model on various language pairs; German/French/Spanish/JapaneseEnglish. When compared against heuristic models, our model achieved comparable translation quality on a full size GermanEnglish language pair in Europarl v7 corpus with significantly smaller grammar size; less than 10% of that for heuristic model.

Word alignment score calculation apparatus, word alignment device, and computer program

Abstract: PROBLEM TO BE SOLVED: To provide an apparatus for highly accurately performing word alignment.SOLUTION: The apparatus includes: selection means that receives a parallel translation pair and word alignments corresponding to the parallel translation pair, and sequentially selects a word fof a sentence in first language in prescribed order; and a recurrent type neural network (RNN) 100 that calculates, for all the words of the sentence in the first language, a score 102 representing correct possibility of a word pair consisting of a word ein second language as a parallel translation pair, associated with the word fby word alignment a, and the word f, and calculates a score of the word alignment aon the basis of the score. When calculating the score of a word pair (fand e), the RNN 100 calculates the score 102 of the word pair (fand e) by cycling connection 118 on the basis of all the word alignments aselected by the selection means before the word fof the word pair (fand e) in the word alignments a.

Modeling Translation

Abstract: Machine translation (MT) has been a challenging application of natural language processing technologies for many years. However, recent major improvements in translation accuracy have led to instances such as Web-based services that almost instantly translate any text into different languages or business-to-business services for high-quality translation of domain-specific documents. This article covers the foundation of recent MT systems and introduces translation as a mathematical process. It also focuses on how an MT system automatically learns to translate using samples of translated texts and how it renders output by combining acquired knowledge.

Word alignment score computation device, word alignment device, and computer program

Abstract: [Problem] To provide a device for performing word alignment with high precision. [Solution] This device comprises: a selection means for receiving a bilingual sentence pair and word alignments for the bilingual sentence pair, and selecting, in a prescribed order, the words (fj) of a first language in sequence; and a recurrent neural network (RNN) (100) for computing for each word in the sentence in the first language a score (102) that indicates the likelihood of the correctness of a word pair consisting of the respective word (fj) and a word (ea_{j}) that is in the second language of the bilingual sentence pair and is aligned with the word (fj) by a word alignment (aj), and computing the score of the word alignment (aj) on the basis of that score. When computing the score of a word pair (fj, ea_{j}), the RNN (100) computes the score (102) of the word pair (fj, ea_{j}) by means of a cyclic connection (118), on the basis of all (a1 j-1) of the word alignments (aj) for words that had been selected by the selection means prior to word (fj) of the word pair (fj, ea_{j}).