IEEE Transactions on Pattern Analysis and Machine Intelligence

Short text classification has been widely explored in news tagging to provide more efficient search strategies and more effective search results for information retrieval. However, most existing studies, concentrating on long text classification, deliver unsatisfactory performance on short texts due to the sparsity issue and the insufficiency of labeled data. In this article, we propose a novel heterogeneous graph neural network-based method for semi-supervised short text classification, leveraging full advantage of limited labeled data and large unlabeled data through information propagation along the graph. Specifically, we first present a flexible heterogeneous information network (HIN) framework for modeling short texts, which can integrate any type of additional information and meanwhile capture their relations to address the semantic sparsity. Then, we propose Heterogeneous Graph Attention networks (HGAT) to embed the HIN for short text classification based on a dual-level attention mechanism, including node-level and type-level attentions. To efficiently classify new coming texts that do not previously exist in the HIN, we extend our model HGAT for inductive learning, avoiding re-training the model on the evolving HIN. Extensive experiments on single-/multi-label classification demonstrates that our proposed model HGAT significantly outperforms state-of-the-art methods across the benchmark datasets under both transductive and inductive learning.

HGAT: Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification

Short text classification has found rich and critical applications in news and tweet tagging to help users find relevant information. Due to lack of labeled training data in many practical use cases, there is a pressing need for studying semi-supervised short text classification. Most existing studies focus on long texts and achieve unsatisfactory performance on short texts due to the sparsity and limited labeled data. In this paper, we propose a novel heterogeneous graph neural network based method for semi-supervised short text classification, leveraging full advantage of few labeled data and large unlabeled data through information propagation along the graph. In particular, we first present a flexible HIN (heterogeneous information network) framework for modeling the short texts, which can integrate any type of additional information as well as capture their relations to address the semantic sparsity. Then, we propose Heterogeneous Graph ATtention networks (HGAT) to embed the HIN for short text classification based on a dual-level attention mechanism, including node-level and type-level attentions. The attention mechanism can learn the importance of different neighboring nodes as well as the importance of different node (information) types to a current node. Extensive experimental results have demonstrated that our proposed model outperforms state-of-the-art methods across six benchmark datasets significantly.

/pdf/heterogeneous-graph-attention-networks-for-semi-supervised-2xlrhbywt0.pdf

Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification

We analyze the dynamics of the behavior known as 'unfollow' in Twitter. We collected daily snapshots of the online relationships of 1.2 million Korean-speaking users for 51 days as well as all of their tweets. We found that Twitter users frequently unfollow. We then discover the major factors, including the reciprocity of the relationships, the duration of a relationship, the followees' informativeness, and the overlap of the relationships, which affect the decision to unfollow. We conduct interview with 22 Korean respondents to supplement the quantitative results. They unfollowed those who left many tweets within a short time, created tweets about uninteresting topics, or tweeted about the mundane details of their lives. To the best of our knowledge, this work is the first systematic study of the unfollow behavior in Twitter.

Fragile Online Relationship: A First Look at Unfollow Dynamics in Twitter

A Practical Approach to Constructing a Knowledge Graph for Cybersecurity

Imbalanced streaming data is widely existed in real world and has attracted much attention in recent years. Most studies focus on either imbalance data or streaming data; however, both imbalance data and streaming data are always accompanied in practice. In this paper, we propose a multi-window based ensemble learning (MWEL as short) method for the classification of imbalanced streaming data. Three types of windows are defined to store the current batch of instances, the latest minority instances and the ensemble classifier. The ensemble classifier consists of a set of latest sub-classifiers, and instances each sub-classifier trained on respectively. All sub-classifiers are weighted before predicting new arriving instance’s class labels and new sub-classifiers are trained if a precision is below a threshold. Extensive experiments on synthetic datasets and real world datasets demonstrate that the new approach can efficiently and efficiently classify imbalanced streaming data and outperform existing approaches.

Multi-window based ensemble learning for classification of imbalanced streaming data

Identifying influential nodes in complex networks has received increasing attention for its great theoretical and practical applications in many fields. Some classical methods, such as degree centrality, betweenness centrality, closeness centrality, and coreness centrality, were reported to have some limitations in detecting influential nodes. Recently, the famous h-index was introduced to the network world to evaluate the spreading ability of the nodes. However, this method always assigns too many nodes with the same value, which leads to a resolution limit problem in distinguishing the real influences of these nodes. In this paper, we propose a local h-index centrality (LH-index) method to identify and rank influential nodes in networks. The LH-index method simultaneously takes into account of h-index values of the node itself and its neighbors, which is based on the idea that a node connecting to more influential nodes will also be influential. Experimental analysis on stochastic Susceptible–Infected–Recovered (SIR) model and several networks demonstrates the effectivity of the LH-index method in identifying influential nodes in networks.

Leveraging local h-index to identify and rank influential spreaders in networks

Entity linking refers to the task of aligning mentions of entities in the text to their corresponding entries in a specific knowledge base, which is of great significance for many natural language process applications such as semantic text understanding and knowledge fusion. The pivotal of this problem is how to make effective use of contextual information to disambiguate mentions. Moreover, it has been observed that, in most cases, mention has similar or even identical strings to the entity it refers to. To prevent the model from linking mentions to entities with similar strings rather than the semantically similar ones, in this paper, we introduce the advanced language representation model called BERT (Bidirectional Encoder Representations from Transformers) and design a hard negative samples mining strategy to fine-tune it accordingly. Based on the learned features, we obtain the valid entity through computing the similarity between the textual clues of mentions and the entity candidates in the knowledge base. The proposed hard negative samples mining strategy benefits entity linking from the larger, more expressive pre-trained representations of BERT with limited training time and computing sources. To the best of our knowledge, we are the first to equip entity linking task with the powerful pre-trained general language model by deliberately tackling its potential shortcoming of learning literally, and the experiments on the standard benchmark datasets show that the proposed model yields state-of-the-art results.

/pdf/deep-entity-linking-via-eliminating-semantic-ambiguity-with-2ksbqjd99s.pdf

Deep Entity Linking via Eliminating Semantic Ambiguity With BERT

Anchor links connect information entities, such as entities of movies or products, across networks from different sources, and thus information in these networks can be transferred directly via anchor links. Therefore, anchor links have great value to many cross-network applications, such as cross-network social link prediction and cross-network recommendation. In this paper, we focus on studying the recommendation problem that can provide ratings of items or services. To address the problem, we propose a Cross-network Collaborative Matrix Factorization (CCMF) recommendation framework based on broad learning setting, which can effectively integrate multi-source information and alleviate the sparse information problem in each individual network. Based on item anchor links CCMF can fuse item similarity information and item latent information across networks from different sources. And different from most of the traditional works, CCMF can make multi-source recommendation tasks collaborate together via the information transfer based on the broad learning setting. During the transfer process, a novel cross-network similarity transfer method is applied to keep the consistency of item similarities between two different networks, and a domain adaptation matrix is used to overcome the domain difference problem. We conduct experiments to compare the proposed CCMF method with both classic and state-of-the-art recommendation techniques. The experimental results illustrate that CCMF outperforms other methods in different experimental circumstances, and has great advantages on dealing with different data sparse problems.

Broad Learning based Multi-Source Collaborative Recommendation

Short text classification is a difficult and challenging task in information retrieval systems since the text data is short, sparse and multidimensional In this paper, we represent short text with Wikipedia concepts for classification Short document text is mapped to Wikipedia concepts and the concepts are then used to represent document for text categorization Traditional methods for classification such as SVM can be used to perform text categorization on the Wikipedia concept document representation Experimental evaluation on real Google search snippets shows that our approach outperforms the traditional BOW method and gives good performance Although it's not better than the state-of-the-art classifier (see eg Phan et al WWW '08), our method can be easily implemented with low cost

Bin Zhou

Papers

Multi-window based ensemble learning for classification of imbalanced streaming data

Leveraging local h-index to identify and rank influential spreaders in networks

Deep Entity Linking via Eliminating Semantic Ambiguity With BERT

Broad Learning based Multi-Source Collaborative Recommendation

Short Text Classification Using Wikipedia Concept Based Document Representation