The embeddings of entities in a large knowledge base (e.g., Wikipedia) are highly beneficial for solving various natural language tasks that involve real world knowledge. In this paper, we present Wikipedia2Vec, a Python-based open-source tool for learning the embeddings of words and entities from Wikipedia. The proposed tool enables users to learn the embeddings efficiently by issuing a single command with a Wikipedia dump file as an argument. We also introduce a web-based demonstration of our tool that allows users to visualize and explore the learned embeddings. In our experiments, our tool achieved a state-of-the-art result on the KORE entity relatedness dataset, and competitive results on various standard benchmark datasets. Furthermore, our tool has been used as a key component in various recent studies. We publicize the source code, demonstration, and the pretrained embeddings for 12 languages at https://wikipedia2vec.github.io/.

Wikipedia2Vec: An Efficient Toolkit for Learning and Visualizing the Embeddings of Words and Entities from Wikipedia

Inferring users’ interests from their activities on social networks has been an emerging research topic in the recent years. Most existing approaches heavily rely on the explicit contributions (posts) of a user and overlook users’ implicit interests, i.e., those potential user interests that the user did not explicitly mention but might have interest in. Given a set of active topics present in a social network in a specified time interval, our goal is to build an interest profile for a user over these topics by considering both explicit and implicit interests of the user. The reason for this is that the interests of free-riders and cold start users who constitute a large majority of social network users, cannot be directly identified from their explicit contributions to the social network. Specifically, to infer users’ implicit interests, we propose a graph-based link prediction schema that operates over a representation model consisting of three types of information: user explicit contributions to topics, relationships between users, and the relatedness between topics. Through extensive experiments on different variants of our representation model and considering both homogeneous and heterogeneous link prediction, we investigate how topic relatedness and users’ homophily relation impact the quality of inferring users’ implicit interests. Comparison with state-of-the-art baselines on a real-world Twitter dataset demonstrates the effectiveness of our model in inferring users’ interests in terms of perplexity and in the context of retweet prediction application. Moreover, we further show that the impact of our work is especially meaningful when considered in case of free-riders and cold start users.

Mining user interests over active topics on social networks

We present a novel method based on interdependent representations of short texts for determining their degree of semantic similarity. The method represents each short text as two dense vectors: the former is built using the word-to-word similarity based on pre-trained word vectors, the latter is built using the word-to-word similarity based on external sources of knowledge. We also developed a preprocessing algorithm that chains coreferential named entities together and performs word segmentation to preserve the meaning of phrasal verbs and idioms. We evaluated the proposed method on three popular datasets, namely Microsoft Research Paraphrase Corpus, STS2015 and P4PIN, and obtained state-of-the-art results on all three without using prior knowledge of natural language, e.g., part-of-speech tags or parse tree, which indicates the interdependent representations of short text pairs are effective and efficient for semantic textual similarity tasks.

Learning short-text semantic similarity with word embeddings and external knowledge sources

Rolling bearing fault diagnosis is closely related to the safety of mechanical system. In real-world diagnosis, it is difficult to obtain abundant labeled data due to varying operation conditions, complex working environment and inevitable indirect measurement, which will affect the ability of diagnosing. To tackle this problem, a deep transfer maximum classifier discrepancy method is proposed under few labeled data, which utilizes fully deep learning and transfer learning. Firstly, a batch-normalized long-short term memory (BNLSTM) model which can learn the mapping relationship between two kinds of datasets is designed to generate some auxiliary samples. Then, a transfer maximum classifier discrepancy (TMCD) method, which considers the characteristics of each data type by an adversarial strategy, is applied to align probability distributions of auxiliary samples generated by BNLSTM and unlabeled data from target domain. Sufficient experimental results indicate the effectiveness of the proposed method under few labeled data.

A deep transfer maximum classifier discrepancy method for rolling bearing fault diagnosis under few labeled data

The advancements of search engines for traditional text documents have enabled the effective retrieval of massive textual information in a resource-efficient manner. However, such conventional search methodologies often suffer from poor retrieval accuracy especially when documents exhibit unique properties that behoove specialized and deeper semantic extraction. Recently, AlgorithmSeer, a search engine for algorithms has been proposed, that extracts pseudo-codes and shallow textual metadata from scientific publications and treats them as traditional documents so that the conventional search engine methodology could be applied. However, such a system fails to facilitate user search queries that seek to identify algorithm-specific information, such as the datasets on which algorithms operate, the performance of algorithms, and runtime complexity, etc. In this paper, a set of enhancements to the previously proposed algorithm search engine are presented. Specifically, we propose a set of methods to automatically identify and extract algorithmic pseudo-codes and the sentences that convey related algorithmic metadata using a set of machine-learning techniques. In an experiment with over 93,000 text lines, we introduce 60 novel features, comprising content-based, font style based and structure-based feature groups, to extract algorithmic pseudo-codes. Our proposed pseudo-code extraction method achieves 93.32% F1-score, outperforming the state-of-the-art techniques by 28%. Additionally, we propose a method to extract algorithmic-related sentences using deep neural networks and achieve an accuracy of 78.5%, outperforming a Rule-based model and a support vector machine model by 28% and 16%, respectively.

Deep Learning-based Extraction of Algorithmic Metadata in Full-Text Scholarly Documents

We propose a novel method for measuring semantic similarity between two sentences. The method exploits both syntactic and semantic features to assess the similarity. In our method, words in a sentence are weighted using their information content. The weights of words help differentiate their contribution towards the meaning of the sentence. The originality of this research is that we explore named entities and their coreference relations as important indicators for measuring the similarity. We conduct experiments and evaluate our proposed method on Microsoft Research Paraphrase Corpus. The experiment results show that named entities and their coreference relations improve significantly the performance of paraphrase identification and the proposed method is comparable with state-of-the-art methods for paraphrase identification.

A Multifaceted Approach to Sentence Similarity

In this paper, we present a hybrid approach to the paraphrase detection task. The approach takes advantage of both feature-engineering and neural-based methods. First, we represent words and entities in a given sentence by using their pre-trained vectors. Then, those pre-trained vectors are encoded by a bidirectional long-short term memory network. The output matrix is fed into an attention network to obtain an attention vector. The final representation of the sentence is inner product of the matrix and the attention vector. We conduct experiments on the Microsoft Research Paraphrase corpus, a popular dataset used for benchmarking paraphrase detection methods. The experimental results show that our approach achieves competitive results.

A Hybrid Approach to Paraphrase Detection

In this paper, we systematically investigate the methods of measuring semantic relatedness between concepts and categorize them into 6 types of methods: path based, information content, gloss based, vector based, corpus based and string based. Besides, we re-implement those types of methods for evaluating on the latest knowledge sources and made API, which is freely available for research purposes, called CM4J1.

Evaluating Semantic Relatedness Between Concepts

In this paper, we present a hybrid model for answer selection in question answering systems by representing multiple kinds of features, i.e., lexical-based, word-alignment, and word-embedding. The model employs convolutional neural network, multilayer perceptron, and support vector machines to train the classifiers. We evaluate our model on the two popular QA datasets, SemEval-2016 Task 3 and TREC QA. The experimental results show that our system outperforms the top-5 proposed systems in SemEval-2016 workshop, and also achieves the-state-of-art results on TREC QA dataset.

Phuc H. Duong

Papers

Learning short-text semantic similarity with word embeddings and external knowledge sources

A Multifaceted Approach to Sentence Similarity

A Hybrid Approach to Paraphrase Detection

Evaluating Semantic Relatedness Between Concepts

A Hybrid Approach to Answer Selection in Question Answering Systems.