Bilingual corpora are critical resources for machine translation research and development since parallel corpora contain translation equivalences of various granularities. Manual annotation of word alignments is of significance to provide a gold-standard for developing and evaluating both example-based machine translation models and statistical machine translation models. This paper presents research on building an English-Vietnamese parallel corpus, which is constructed for building a Vietnamese-English machine translation system. We describe the specification of collecting data for the corpus, linguistic tagging, bilingual annotation, and the tools specially developed for the manual annotation. An English-Vietnamese bilingual corpus of over 800,000 sentence pairs and 10,000,000 English words as well as Vietnamese words has been collected and aligned at the sentence level, and over 45,000 sentence pairs of this corpus have been aligned at the word level.

Building an English-Vietnamese Bilingual Corpus for Machine Translation

Bilingual corpora play an important role as resources not only for machine translation research and development but also for studying tasks in comparative linguistics. Manual annotation of word alignments is of significance to provide a gold-standard for developing and evaluating machine translation models and comparative linguistics tasks. This paper presents research on building an English-Vietnamese parallel corpus, which is constructed for building a Vietnamese-English machine translation system. We describe the specification of collecting data for the corpus, linguistic tagging, bilingual annotation, and the tools specially developed for the manual

EVBCorpus - A Multi-Layer English-Vietnamese Bilingual Corpus for Studying Tasks in Comparative Linguistics

Parsing plays a significant role in many natural language processing NLP applications as their efficiency relies on having an effective parser. This paper presents Amharic sentence parser developed using base phrase chunker that groups syntactically correlated words at different levels. We use HMM to chunk base phrases where incorrectly chunked phrases are pruned with rules. The task of parsing is then performed by taking chunk results as inputs. Bottom-up approach with transformation algorithm is used to transform the chunker to the parser. Corpus from Amharic news outlets and books was collected for training and testing. The training and testing datasets were prepared using the 10-fold cross validation technique. Test results on the test data showed an average parsing accuracy of 93.75%.

Amharic Sentence Parsing Using Base Phrase Chunking

Segmentation of a text into non-overlapping syntactic units (chunks) has become an essential component of many applications of natural language processing. This paper presents Amharic base phrase chunker that groups syntactically correlated words at different levels using HMM. Rules are used to correct chunk phrases incorrectly chunked by the HMM. For the identification of the boundary of the phrases IOB2 chunk specification is selected and used in this work. To test the performance of the system, corpus was collected from Amharic news outlets and books. The training and testing datasets were prepared using the 10-fold cross validation technique. Test results on the corpus showed an average accuracy of 85.31 % before applying the rule for error correction and an average accuracy of 93.75 % after applying rules.

Hierarchical Amharic Base Phrase Chunking Using HMM with Error Pruning

Antonymy and synonymy are basic semantic relations between words. Automatically distinguishing between antonymy and synonymy is an important task in natural language processing. This task is hard because antonyms and synonyms tend to occur in highly similar contexts. Recent studies often focus on exploiting densevector representations of words to deal with this problem. In this paper, we present a study on antonymy-synonymy discrimination for the Vietnamese language. We propose a deep neural network model (DVASNet) that can utilize not only embedding representations of words but also co-occurrence contexts and specific patterns of Vietnamese word structure. Our experimental results showed that the proposed method achieved significant improvements in comparison with a number of state-of-the-art methods by 14% to 17% in terms of F1 score for both benchmark

/pdf/a-neural-network-model-for-efficient-antonymy-synonymy-ram6p3v0wj.pdf

A Neural Network Model for Efficient Antonymy-Synonymy Classification by Exploiting Co-occurrence Contexts and Word-Structure Patterns

Noun phrase chunking is an important and useful task in many natural language processing applications. It is studied well for English, however with Vietnamese it is still an open problem. This paper presents a Vietnamese Noun Phrase chunking approach based on Conditional random fields (CRFs) models. We also describe a method to build Vietnamese corpus from a set of hand annotated sentences. For evaluation, we perform several experiments using different feature settings. Outcome results on our corpus show a high performance with the average of recall and precision 82.72% and 82.62% respectively.

/pdf/vietnamese-noun-phrase-chunking-based-on-conditional-random-5a7tz36mkk.pdf

Vietnamese Noun Phrase Chunking Based on Conditional Random Fields

The lexical entailment recognition problem aims to identify the is-a relation between words. The problem has recently been receiving research attention in the natural language processing field. In this study, we propose a novel method (VLER) for this problem on Vietnamese. For this purpose, we first exploit such lexical structure information of words as a feature, then combine this feature with vectors representation of words such as a unique feature for recognizing the relation. Moreover, we applied a number of methods based on word embedding and supervised learning, experimental results showed that our method achieves the best performance in the hypernymy detection task than other methods in terms of accuracy.

Enhancing Performance of Lexical Entailment Recognition for Vietnamese based on Exploiting Lexical Structure Features

This paper proposes a method based on linguistic word-formation rules and dictionaries for determining reduplicative words in Vietnamese. The key idea for identifying whether adjacent syllables in a text can form a reduplicative word based on its formation rules. For 2-syllable reduplicative words, this paper uses rules that describe the repeating and the opposing between pairs of initial consonants, rhymes and tones. Then the method is expanded to identify reduplicative words that have 3 or 4 syllables from 2-syllable ones for the Vietnamese word segmentation task. Experimental results showed that the F1-score was improved to 98.61% and that word segmentation errors were reduced significantly, 1.26%.

Nguyen Phuong Thai

Papers

Vietnamese Noun Phrase Chunking Based on Conditional Random Fields

Enhancing Performance of Lexical Entailment Recognition for Vietnamese based on Exploiting Lexical Structure Features

Identifying reduplicative words for Vietnamese word segmentation