Semantic relatedness between words is a core concept in natural language processing. While countless approaches have been proposed, measuring which one works best is still a challenging task. Thus, in this article, we give a comprehensive overview of the evaluation protocols and datasets for semantic relatedness covering both intrinsic and extrinsic approaches. One the intrinsic side, we give an overview of evaluation datasets covering more than 100 datasets in 20 different languages from a wide range of domains. To provide researchers with better guidance for selecting suitable dataset or even building new and better ones, we describe also the construction and annotation process of the datasets. We also shortly describe the evaluation metrics most frequently used for intrinsic evaluation. As for the extrinsic side, several applications involving semantic relatedness measures are detailed through recent research works and by explaining the benefit brought by the measures.

A survey of semantic relatedness evaluation datasets and procedures

Antonymy and synonymy are basic semantic relations between words. Automatically distinguishing between antonymy and synonymy is an important task in natural language processing. This task is hard because antonyms and synonyms tend to occur in highly similar contexts. Recent studies often focus on exploiting densevector representations of words to deal with this problem. In this paper, we present a study on antonymy-synonymy discrimination for the Vietnamese language. We propose a deep neural network model (DVASNet) that can utilize not only embedding representations of words but also co-occurrence contexts and specific patterns of Vietnamese word structure. Our experimental results showed that the proposed method achieved significant improvements in comparison with a number of state-of-the-art methods by 14% to 17% in terms of F1 score for both benchmark

/pdf/a-neural-network-model-for-efficient-antonymy-synonymy-ram6p3v0wj.pdf

A Neural Network Model for Efficient Antonymy-Synonymy Classification by Exploiting Co-occurrence Contexts and Word-Structure Patterns

The lexical entailment recognition problem aims to identify the is-a relation between words. The problem has recently been receiving research attention in the natural language processing field. In this study, we propose a novel method (VLER) for this problem on Vietnamese. For this purpose, we first exploit such lexical structure information of words as a feature, then combine this feature with vectors representation of words such as a unique feature for recognizing the relation. Moreover, we applied a number of methods based on word embedding and supervised learning, experimental results showed that our method achieves the best performance in the hypernymy detection task than other methods in terms of accuracy.

Enhancing Performance of Lexical Entailment Recognition for Vietnamese based on Exploiting Lexical Structure Features

The challenge of Lexical Entailment Recognition (LER) aims to identify the is-a relation between words. This problem has recently received attention from researchers in the field of natural language processing because of its application to varied downstream tasks. However, almost all prior studies have only focused on datasets that include single words; thus, how to handle compound words effectively is still a challenge. In this study, we propose a novel method called LERC (Lexical Entailment Recognition Combination) to solve this problem by combining embedding representations and subword semantic features. For this aim, firstly a specialized word embedding model for the LER tasks is trained. Secondly, subword semantic information of word pairs is exploited to compute another feature vector. This feature vector is combined with embedding vectors for supervised classification. We considered three LER tasks, including Lexical Entailment Detection, Lexical Entailment Directionality, and Lexical Entailment Determination. Experimental results conducted on several benchmark datasets in English and Vietnamese languages demonstrated that the subword semantic feature is useful for these tasks. Moreover, LERC outperformed several methods published recently. 

Combining Specialized Word Embeddings and Subword Semantic Features for Lexical Entailment Recognition

Antonymy and Synonymy are paradigmatic relations which are in the core problems of language. Distinguishing antonyms from synonyms is a key task to achieve high performance in natural language processing systems. Distinguishing between antonyms and synonyms is a hard problem because the co-occurrence distributions of the antonyms or synonyms tend to be highly similar. On the other hand, this issue has been thoroughly studied in English. However, it has not been effectively addressed for Vietnamese. Compared to English, Vietnamese has its own word-level characteristics that indicate the synonymous or antonymous relation. In this paper, we introduce a framework which exploits exhaustively special Vietnamese features to distinguish between antonyms from synonyms. We propose a deep neural network model (ViASNet) that can utilize not only lexico-syntactic information captured from the context of word pairs in a corpus but also its word-level features, and distribution features as well. The experimental results show that the proposed method is effective. Furthermore, our method achieved high performance in comparison to several the state of the art methods.

Antonyms-Synonyms Discrimination Based on Exploiting Rich Vietnamese Features.

Measuring word similarity is a core issue because it has many applications in natural language processing. Although many studies have been reported and the techniques have been developed for addressing this issue for English, however, the study dealing with the applications, analyses and evaluation word similarity techniques to Vietnamese still has not reported yet. Especially, there is still lack of the benchmark Vietnamese dataset for evaluating these techniques. In this paper, we report three main topics including: firstly, construct a benchmark dataset for evaluation of similar techniques to the Vietnamese language; secondly, experiment with some similarity techniques based on WordNet and word embeddings; and finally, propose an extension for Lesk algorithm in order to improving the efficiency of similar measuring with Vietnamese language.

Pham Van Lam

Papers

Construction of a word similarity dataset and evaluation of word similarity techniques for Vietnamese

Antonyms-Synonyms Discrimination Based on Exploiting Rich Vietnamese Features.