scispace - formally typeset

Evaluation of machine translation

About: Evaluation of machine translation is a(n) research topic. Over the lifetime, 1170 publication(s) have been published within this topic receiving 77233 citation(s). more


Open accessProceedings ArticleDOI: 10.3115/1073083.1073135
06 Jul 2002-
Abstract: Human evaluations of machine translation are extensive but expensive. Human evaluations can take months to finish and involve human labor that can not be reused. We propose a method of automatic machine translation evaluation that is quick, inexpensive, and language-independent, that correlates highly with human evaluation, and that has little marginal cost per run. We present this method as an automated understudy to skilled human judges which substitutes for them when there is need for quick or frequent evaluations. more

16,385 Citations

Open accessProceedings ArticleDOI: 10.18653/V1/D15-1166
17 Aug 2015-
Abstract: An attentional mechanism has lately been used to improve neural machine translation (NMT) by selectively focusing on parts of the source sentence during translation. However, there has been little work exploring useful architectures for attention-based NMT. This paper examines two simple and effective classes of attentional mechanism: a global approach which always attends to all source words and a local one that only looks at a subset of source words at a time. We demonstrate the effectiveness of both approaches on the WMT translation tasks between English and German in both directions. With local attention, we achieve a significant gain of 5.0 BLEU points over non-attentional systems that already incorporate known techniques such as dropout. Our ensemble model using different attention architectures yields a new state-of-the-art result in the WMT’15 English to German translation task with 25.9 BLEU points, an improvement of 1.0 BLEU points over the existing best system backed by NMT and an n-gram reranker. 1 more

6,374 Citations

Open accessProceedings ArticleDOI: 10.3115/1557769.1557821
Philipp Koehn1, Hieu Hoang1, Alexandra Birch1, Chris Callison-Burch1  +10 moreInstitutions (7)
25 Jun 2007-
Abstract: We describe an open-source toolkit for statistical machine translation whose novel contributions are (a) support for linguistically motivated factors, (b) confusion network decoding, and (c) efficient data formats for translation models and language models. In addition to the SMT decoder, the toolkit also includes a wide variety of tools for training, tuning and applying the system to many translation tasks. more

5,646 Citations

Open accessProceedings ArticleDOI: 10.3115/1073445.1073462
27 May 2003-
Abstract: We propose a new phrase-based translation model and decoding algorithm that enables us to evaluate and compare several, previously proposed phrase-based translation models. Within our framework, we carry out a large number of experiments to understand better and explain why phrase-based models out-perform word-based models. Our empirical results, which hold for all examined language pairs, suggest that the highest levels of performance can be obtained through relatively simple means: heuristic learning of phrase translations from word-based alignments and lexical weighting of phrase translations. Surprisingly, learning phrases longer than three words and learning phrases from high-accuracy word-level alignment models does not have a strong impact on performance. Learning only syntactically motivated phrases degrades the performance of our systems. more

Topics: Noun phrase (72%), Phrase (70%), Determiner phrase (66%) more

3,654 Citations

Open accessProceedings ArticleDOI: 10.3115/1075096.1075117
Franz Josef Och1Institutions (1)
07 Jul 2003-
Abstract: Often, the training procedure for statistical machine translation models is based on maximum likelihood or related criteria. A general problem of this approach is that there is only a loose relation to the final translation quality on unseen text. In this paper, we analyze various training criteria which directly optimize translation quality. These training criteria make use of recently proposed automatic evaluation metrics. We describe a new algorithm for efficient training an unsmoothed error count. We show that significantly better results can often be obtained if the final evaluation criterion is taken directly into account as part of the training procedure. more

3,195 Citations

No. of papers in the topic in previous years

Top Attributes

Show by:

Topic's top 5 most impactful authors

Hermann Ney

39 papers, 2.5K citations

Philipp Koehn

19 papers, 8.4K citations

Andy Way

18 papers, 332 citations

Marta R. Costa-jussà

13 papers, 548 citations

Rafael E. Banchs

13 papers, 268 citations

Network Information
Related Topics (5)
Machine translation

22.1K papers, 574.4K citations

90% related

3.7K papers, 126.9K citations

87% related
Language model

17.5K papers, 545K citations

87% related
Text corpus

4K papers, 107.8K citations

87% related

21.5K papers, 545.4K citations

85% related