scispace - formally typeset
Proceedings ArticleDOI: 10.1109/CONFLUENCE.2018.8442777

A Summary and Comparative Study of Different Metrics for Machine Translation Evaluation

01 Jan 2018-pp 55-60
Abstract: Assessment of the Translation done by machines has always been a major topic of interest for the Natural Language Researchers. Every day, a new translator is out there in the market claiming to be the best one. But to challenge such claims, one needs to have a powerful assessment tool that can judge the output of any translator on different parameters like fluency of the language, adequacy as well as the accuracy of the output generated. In this paper, we have discussed the major issues with the algorithms used for the evaluation of Machine translation. We have also presented a brief description and comparative study of different metrics used to evaluate the outcome of a machine translator over English to Hindi Language parallel sentences.

...read more

Topics: Machine translation (71%), Evaluation of machine translation (68%), Natural language (55%) ...read more
Citations
  More

Open accessProceedings Article
01 Mar 2019-
Abstract: Machine translation is a process of translating one natural language to another without much human interaction. Evaluation of any Machine Translation System (MTS) is the most important factor in a machine learning environment. There are many techniques existing to determine and optimize the quality of output in any MTS. Earlier methods are based on human judgments. Even though human evaluation methods are very much reliable, they suffer due to some disadvantages such as high cost, more time consuming and also poor re-usability. Hence, automatic methods have been proposed to reduce time and cost. In this survey, we have discussed different metrics under the automatic evaluation techniques in order to evaluate the output quality of MTS. It is believed that machine learning system developers at large would get befitted by this survey.

...read more

Topics: Machine translation (59%)

1 Citations


Open accessJournal ArticleDOI: 10.17762/ITII.V9I2.458
13 Apr 2021-
Abstract: In many existing solutions of image-to-image rendering problems, the only focus is to find the closest output of the Generative Adversarial Network (GAN). In this research article, authors propose a generative adversarial network, a solution to pixel-to-pixel rendering problems and reduced the loss function to the maximum under all interactions. For achieving the best result, we have considered the mean square loss function in the generator and binary cross for the discriminator. Our proposed model deals with not only images but also read sketches where the edges are not sharp too. We have used a facade dataset to test our proposed model.

...read more

1 Citations


Journal ArticleDOI: 10.2174/2213275912666190716100145
05 Nov 2020-
Abstract: Automatic Machine Translation (AMT) Evaluation Metrics have become popular in the Machine Translation Community in recent times. This is because of the popularity of Machine Translation engines and Machine Translation as a field itself. Translator is a very important tool to break barriers between communities especially in countries like India, where people speak 22 different languages and their many variations. With the onset of Machine Translation engines, there is a need for a system that evaluates how well these are performing. This is where machine translation evaluation enters. This paper discusses the importance of Automatic Machine Translation Evaluation and compares various Machine Translation Evaluation metrics by performing Statistical Analysis on various metrics and human evaluations to find out which metric has the highest correlation with human scores. The correlation between the Automatic and Human Evaluation Scores and the correlation between the five Automatic evaluation scores are examined at the sentence level. Moreover, a hypothesis is set up and p-values are calculated to find out how significant these correlations are. The results of the statistical analysis of the scores of various metrics and human scores are shown in the form of graphs to see the trend of the correlation between the scores of Automatic Machine Translation Evaluation metrics and human scores. Out of the five metrics considered for the study, METEOR shows the highest correlation with human scores as compared to the other metrics.

...read more

Topics: Machine translation (68%), Hindi (61%)

1 Citations


Open accessJournal ArticleDOI: 10.3390/S21041493
21 Feb 2021-Sensors
Abstract: In this paper, we introduce new concepts in the machine translation paradigm. We treat the corpus as a database of frequent word sets. A translation request triggers association rules joining phrases present in the source language, and phrases present in the target language. It has to be noted that a sequential scan of the corpus for such phrases will increase the response time in an unexpected manner. We introduce the pre-processing of the bilingual corpus through proposing a data structure called Corpus-Trie (CT) that renders a bilingual parallel corpus in a compact data structure representing frequent data items sets. We also present algorithms which utilize the CT to respond to translation requests and explore novel techniques in exhaustive experiments. Experiments were performed on specific language pairs, although the proposed method is not restricted to any specific language. Moreover, the proposed Corpus-Trie can be extended from bilingual corpora to accommodate multi-language corpora. Experiments indicated that the response time of a translation request is logarithmic to the count of unrepeated phrases in the original bilingual corpus (and thus, the Corpus-Trie size). In practical situations, 5–20% of the log of the number of the nodes have to be visited. The experimental results indicate that the BLEU score for the proposed CT system increases with the size of the number of phrases in the CT, for both English-Arabic and English-French translations. The proposed CT system was demonstrated to be better than both Omega-T and Apertium in quality of translation from a corpus size exceeding 1,600,000 phrases for English-Arabic translation, and 300,000 phrases for English-French translation.

...read more

Topics: Machine translation (64%)

1 Citations


Proceedings ArticleDOI: 10.1109/SCCC49216.2019.8966444
01 Nov 2019-
Abstract: This work presents calcDPC, an algorithm to analyze psychological and linguistic changes in texts translated from English to Brazilian Portuguese, and compare differences between automatic and human-translated texts. For this purpose, we use the Brazilian Portuguese version of a tool named LIWC, that distributes lexical words in categories with linguistic and psychological characteristics. By counting words in categories, this work seeks to evaluate the percentage of psycholinguistic changes in automatically translated texts and compare them with a reliable translation performed by a human expert. In this way, this study aims to contribute to the improvement of automatic translation tools. Experimental results indicate promising directions for further research.

...read more

Topics: Brazilian Portuguese (53%), Psycholinguistics (53%)
Performance
Metrics
No. of citations received by the Paper in previous years
YearCitations
20212
20201
20192