Pushing the Limits of Translation Quality Estimation
André F. T. Martins,Marcin Junczys-Dowmunt,Fabio Natanael Kepler,Ramón Fernandez Astudillo,Chris Hokamp,Roman Grundkiewicz +5 more
Reads0
Chats0
TLDR
A new, carefully engineered, neural model is stacked into a rich feature-based word-level quality estimation system and the output of an automatic post-editing system is used as an extra feature, obtaining striking results on WMT16.Abstract:
Translation quality estimation is a task of growing importance in NLP, due to its potential to reduce post-editing human effort in disruptive ways. However, this potential is currently limited by the relatively low accuracy of existing systems. In this paper, we achieve remarkable improvements by exploiting synergies between the related tasks of word-level quality estimation and automatic post-editing. First, we stack a new, carefully engineered, neural model into a rich feature-based word-level quality estimation system. Then, we use the output of an automatic post-editing system as an extra feature, obtaining striking results on WMT16: a word-level F 1 MULT score of 57.47% (an absolute gain of +7.95% over the current state of the art), and a Pearson correlation score of 65.56% for sentence-level HTER prediction (an absolute gain of +13.36%).read more
Citations
More filters
Proceedings ArticleDOI
Findings of the 2017 Conference on Machine Translation (WMT17)
Ondřej Bojar,Rajen Chatterjee,Christian Federmann,Yvette Graham,Barry Haddow,Shujian Huang,Matthias Huck,Philipp Koehn,Qun Liu,Varvara Logacheva,Christof Monz,Matteo Negri,Matt Post,Raphael Rubino,Lucia Specia,Marco Turchi +15 more
TL;DR: The results of the WMT17 shared tasks, which included three machine translation (MT) tasks (news, biomedical, and multimodal), two evaluation tasks (metrics and run-time estimation of MT quality), an automatic post-editing task, a neural MT training task, and a bandit learning task are presented.
Proceedings ArticleDOI
OpenKiwi: An Open Source Framework for Quality Estimation
TL;DR: This work introduces OpenKiwi, a Pytorch-based open source framework for translation quality estimation that supports training and testing of word-level and sentence-level quality estimation systems, implementing the winning systems of the WMT 2015–18 quality estimation campaigns.
Proceedings ArticleDOI
Findings of the WMT 2019 Shared Tasks on Quality Estimation
TL;DR: The WMT19 shared task on Quality Estimation is reported, the task of predicting the quality of the output of machine translation systems given just the source text and the hypothesis translations, with a novel addition is evaluating sentence-level QE against human judgments.
Book
Quality Estimation for Machine Translation
TL;DR: This research presents a novel approach to text-to-text transformation that automates the very labor-intensive and therefore time-heavy and expensive process of converting text to text.
Journal ArticleDOI
Predictor-Estimator: Neural Quality Estimation Based on Target Word Prediction for Machine Translation
TL;DR: A novel neural network architecture for quality estimation task—called the predictor-estimator—that considers word prediction as an additional pre-task that is able to transfer useful knowledge to quality estimation tasks is proposed.
References
More filters
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Proceedings ArticleDOI
Learning Phrase Representations using RNN Encoder--Decoder for Statistical Machine Translation
Kyunghyun Cho,Bart van Merriënboer,Caglar Gulcehre,Dzmitry Bahdanau,Fethi Bougares,Holger Schwenk,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio +8 more
TL;DR: In this paper, the encoder and decoder of the RNN Encoder-Decoder model are jointly trained to maximize the conditional probability of a target sequence given a source sequence.
Proceedings Article
Rectified Linear Units Improve Restricted Boltzmann Machines
Vinod Nair,Geoffrey E. Hinton +1 more
TL;DR: Restricted Boltzmann machines were developed using binary stochastic hidden units that learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset.
Proceedings Article
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot,Yoshua Bengio +1 more
TL;DR: The objective here is to understand better why standard gradient descent from random initialization is doing so poorly with deep neural networks, to better understand these recent relative successes and help design better algorithms in the future.
Posted Content
ADADELTA: An Adaptive Learning Rate Method
TL;DR: A novel per-dimension learning rate method for gradient descent called ADADELTA that dynamically adapts over time using only first order information and has minimal computational overhead beyond vanilla stochastic gradient descent is presented.