Abstract Class imbalance is often a problem in various real-world data sets, where one class (i.e. the minority class) contains a small number of data points and the other (i.e. the majority class) contains a large number of data points. It is notably difficult to develop an effective model using current data mining and machine learning algorithms without considering data preprocessing to balance the imbalanced data sets. Random undersampling and oversampling have been used in numerous studies to ensure that the different classes contain the same number of data points. A classifier ensemble (i.e. a structure containing several classifiers) can be trained on several different balanced data sets for later classification purposes. In this paper, we introduce two undersampling strategies in which a clustering technique is used during the data preprocessing step. Specifically, the number of clusters in the majority class is set to be equal to the number of data points in the minority class. The first strategy uses the cluster centers to represent the majority class, whereas the second strategy uses the nearest neighbors of the cluster centers. A further study was conducted to examine the effect on performance of the addition or deletion of 5 to 10 cluster centers in the majority class. The experimental results obtained using 44 small-scale and 2 large-scale data sets revealed that the clustering-based undersampling approach with the second strategy outperformed five state-of-the-art approaches. Specifically, this approach combined with a single multilayer perceptron classifier and C4.5 decision tree classifier ensembles delivered optimal performance over both small- and large-scale data sets.

分群式取樣法於類別不平衡問題之研究;Clustering-Based Under-sampling in Class Imbalanced Data

In recent years, the volume of textual data has rapidly increased, which has generated a valuable resource for extracting and analysing information. To retrieve useful knowledge within a reasonable time period, this information must be summarised. This paper reviews recent approaches for abstractive text summarisation using deep learning models. In addition, existing datasets for training and validating these approaches are reviewed, and their features and limitations are presented. The Gigaword dataset is commonly employed for single-sentence summary approaches, while the Cable News Network (CNN)/Daily Mail dataset is commonly employed for multisentence summary approaches. Furthermore, the measures that are utilised to evaluate the quality of summarisation are investigated, and Recall-Oriented Understudy for Gisting Evaluation 1 (ROUGE1), ROUGE2, and ROUGE-L are determined to be the most commonly applied metrics. The challenges that are encountered during the summarisation process and the solutions proposed in each approach are analysed. The analysis of the several approaches shows that recurrent neural networks with an attention mechanism and long short-term memory (LSTM) are the most prevalent techniques for abstractive text summarisation. The experimental results show that text summarisation with a pretrained encoder model achieved the highest values for ROUGE1, ROUGE2, and ROUGE-L (43.85, 20.34, and 39.9, respectively). Furthermore, it was determined that most abstractive text summarisation models faced challenges such as the unavailability of a golden token at testing time, out-of-vocabulary (OOV) words, summary sentence repetition, inaccurate sentences, and fake facts.

/pdf/deep-learning-based-abstractive-text-summarization-3z8j6m5u4i.pdf

Deep Learning Based Abstractive Text Summarization: Approaches, Datasets, Evaluation Measures, and Challenges

Being one of the most widely used social media tools, Twitter is seen as an important source of information for acquiring people’s attitudes, emotions, views and feedbacks. Within this context, Twitter sentiment analysis techniques were developed to decide whether textual tweets express a positive or negative opinion. In contrast to lower classification performance of traditional algorithms, deep learning models, including Convolution Neural Network (CNN) and Bidirectional Long Short-Term Memory (Bi-LSTM), have achieved a significant result in sentiment analysis. Although CNN can extract high-level local features efficiently by using convolutional layer and max-pooling layer, it cannot effectively learn sequence of correlations. On the other hand, Bi-LSTM uses two LSTM directions to improve the contexts available to deep learning algorithms, but Bi-LSTM cannot extract local features in a parallel way. Therefore, applying a single CNN or single Bi-LSTM for sentiment analysis cannot achieve the optimal classification result. An integrating structure of CNN and Bi-LSTM model is proposed in this study. ConvBiLSTM is implemented; a word embedding model which converts tweets into numerical values, CNN layer receives feature embedding as input and produces smaller dimension of features, and the Bi-LSTM model takes the input from the CNN layer and produces classification result. Word2Vec and GloVe were distinctly applied to observe the impact of the word embedding result on the proposed model. ConvBiLSTM was applied with retrieved Tweets and SST-2 datasets. ConvBiLSTM model with Word2Vec on retrieved Tweets dataset outperformed the other models with 91.13% accuracy.

/pdf/a-convbilstm-deep-learning-model-based-approach-for-twitter-3htf45t20q.pdf

A ConvBiLSTM Deep Learning Model-Based Approach for Twitter Sentiment Classification

Sentiment analysis of tweets using refined neutrosophic sets

Short text similarity plays an important role in natural language processing (NLP). It has been applied in many fields. Due to the lack of sufficient context in the short text, it is difficult to measure the similarity. The use of semantics similarity to calculate textual similarity has attracted the attention of academia and industry and achieved better results. In this survey, we have conducted a comprehensive and systematic analysis of semantic similarity. We first propose three categories of semantic similarity: corpus‐based, knowledge‐based, and deep learning (DL)‐based. We analyze the pros and cons of representative and novel algorithms in each category. Our analysis also includes the applications of these similarity measurement methods in other areas of NLP. We then evaluate state‐of‐the‐art DL methods on four common datasets, which proved that DL‐based can better solve the challenges of the short text similarity, such as sparsity and complexity. Especially, bidirectional encoder representations from transformer model can fully employ scarce information of short texts and semantic information and obtain higher accuracy and F1 value. We finally put forward some future directions.

A survey on the techniques, applications, and performance of short text semantic similarity

As a core task of natural language processing and information retrieval, automatic text summarization is widely applied in many fields. There are two existing methods for text summarization task at present: abstractive and extractive. On this basis we propose a novel hybrid model of extractive-abstractive to combine BERT (Bidirectional Encoder Representations from Transformers) word embedding with reinforcement learning. Firstly, we convert the human-written abstractive summaries to the ground truth labels. Secondly, we use BERT word embedding as text representation and pre-train two sub-models respectively. Finally, the extraction network and the abstraction network are bridged by reinforcement learning. To verify the performance of the model, we compare it with the current popular automatic text summary model on the CNN/Daily Mail dataset, and use the ROUGE (Recall-Oriented Understudy for Gisting Evaluation) metrics as the evaluation method. Extensive experimental results show that the accuracy of the model is improved obviously.

https://www.mdpi.com/2076-3417/9/21/4701/pdf

A text abstraction summary model based on BERT word embedding and reinforcement learning

Previous work has recognized the importance of using the attention mechanism to obtain the interaction between aspect words and contexts for sentiment analysis. However, for the most attention mechanisms, it is unrigorous to use the average vector of the aspect words to calculate the context attention. Besides, the feature extraction ability of the model is also essential for effective analysis, the combination of CNN and LSTM can enhance the feature extraction ability and semantic expression ability of the model, which is also a popular research trend. This paper introduces an aspect level neural network for sentiment analysis named Feature Enhanced Attention CNN-BiLSTM (FEA-NN). Our method is to extract a higher-level phrase representation sequence from the embedding layer by using CNN, which provides effective support for subsequent coding tasks. In order to improve the quality of context encoding and preserve semantic information, we use BiLSTM to capture both local features of phrases as well as global and temporal sentence semantics. Besides, we add an attention mechanism to model interaction relationships between aspect words and sentences to focus on those keywords of targets to learn more effective context representation. We evaluate the proposed model on three datasets: Restaurant, Laptop, and Twitter. Extensive experiments show that the effectivess of FEA-NN.

/pdf/aspect-based-sentiment-analysis-with-feature-enhanced-3y1rxhb8ay.pdf

Aspect Based Sentiment Analysis With Feature Enhanced Attention CNN-BiLSTM

Text representation learning is an important but challenging issue for various natural language processing tasks. Recently, deep learning-based representation models have achieved great success for sentiment classification. However, these existing models focus on more semantic information rather than sentiment linguistic knowledge, which provides rich sentiment information and plays a key role in sentiment analysis. In this paper, we propose a lexicon-enhanced attention network (LAN) based on text representation to improve the performance of sentiment classification. Specifically, we first propose a lexicon-enhanced attention mechanism by combining the sentiment lexicon with an attention mechanism to incorporate sentiment linguistic knowledge into deep learning methods. Second, we introduce a multi-head attention mechanism in the deep neural network to interactively capture the contextual information from different representation subspaces at different positions. Furthermore, we stack a LAN model to build a hierarchical sentiment classification model for large-scale text. Extensive experiments are conducted to evaluate the effectiveness of the proposed models on four popular real-world sentiment classification datasets at both the sentence level and the document level. The experimental results demonstrate that our proposed models can achieve comparable or better performance than the state-of-the-art methods.

https://www.mdpi.com/2076-3417/9/18/3717/pdf

Lexicon-Enhanced Attention Network Based on Text Representation for Sentiment Classification

Domain adaptation tasks have raised much attention in recent years, especially, the task of cross-domain sentiment classification, and remarkable success has been achieved on specific domains with large amounts of labeled data. However, annotating enough data in each domain is still expensive and time-consuming, which will produce difficulty in the application of domain adaptation. In this paper, we proposed a Capsule network method with Identifying Transferable Knowledge (CITK) as common knowledge for cross-domain sentiment classification. CITK model uses capsule network to encode the intrinsic spatial part-whole relationship constituting domain invariant knowledge, which bridges the knowledge gap between the source and target domains. In addition, we use Bidirectional Encoder Representations from Transformers (BERT) to convert sentences to equal length, which is called pre-training, in order to obtain more complete semantic embedded representation, so that Significant Consistent Polarity (SCP) words can be more accurate. Extensive experiments are conducted to evaluate the effectiveness of the proposed CITK model on a real world data set of four domains. Experimental results demonstrate that CITK can significantly outperform the state-of-the-art methods for the cross-domain sentiment classification task.

/pdf/capsule-network-with-identifying-transferable-knowledge-for-1f9p4hd04f.pdf

Capsule Network With Identifying Transferable Knowledge for Cross-Domain Sentiment Classification

Imbalanced classification is one of the most important problems of machine learning and data mining, existing in many real datasets. In the past, many basic classifiers such as SVM, KNN, and so on have been used for imbalanced datasets in which the number of one sample is larger than that of another, but the classification effect is not ideal. Some data preprocessing methods have been proposed to reduce the imbalance ratio of data sets and combine with the basic classifiers to get better performance. In order to improve the whole classification accuracy, we propose a novel classifier ensemble framework based on K-means and resampling technique (EKR). First, we divide the data samples in the majority class into several sub-clusters using K-means, k-value is determined by Average Silhouette Coefficient, and then adjust the number of data samples of each sub-cluster to be the same as that of the minority classes through resampling technology, after that each adjusted sub-cluster and the minority class are combined into several balanced subsets, the base classifier is trained on each balanced subset separately, and finally integrated into a strong ensemble classifier. In this paper, the extensive experimental results on 16 imbalanced datasets demonstrate the effectiveness and feasibility of the proposed algorithm in terms of multiple evaluation criteria, and EKR can achieve better performance when compared with several classical imbalanced classification algorithms using different data preprocessing methods.

Hongxia Yin

Papers

A text abstraction summary model based on BERT word embedding and reinforcement learning

Aspect Based Sentiment Analysis With Feature Enhanced Attention CNN-BiLSTM

Lexicon-Enhanced Attention Network Based on Text Representation for Sentiment Classification

Capsule Network With Identifying Transferable Knowledge for Cross-Domain Sentiment Classification

A Novel Ensemble Framework Based on K-Means and Resampling for Imbalanced Data