https://rgu-repository.worktribe.com/preview/512831/VUTTIPITTAYAMONGKOL%202020%20Neighbourhood%20based.pdf

Neighbourhood-based undersampling approach for handling imbalanced and overlapped data

In this paper, we propose a new methodology for learning evolving fuzzy systems (EFS) from data streams in terms of on-line regression/system identification problems. It comes with enhanced dynamic complexity reduction steps, acting on model components and on the input structure and by employing generalized fuzzy rules in arbitrarily rotated position. It is thus termed as Gen-Smart-EFS (GS-EFS), short for generalized smart evolving fuzzy systems. Equipped with a new projection concept for high-dimensional kernels onto one-dimensional fuzzy sets, our approach is able to provide equivalent conventional TS fuzzy systems with axis-parallel rules, thus maintaining interpretability when inferring new query samples. The on-line complexity reduction on rule level integrates a new merging concept based on a combined adjacency–homogeneity relation between two clusters (rules). On input structure level, complexity reduction is motivated by a combined statistical-geometric concept and acts in a smooth and soft manner by incrementally adapting feature weights: features may get smoothly out-weighted over time (\(\rightarrow\)soft on-line dimension reduction) but also may become reactivated at a later stage. Out-weighted features will contribute little to the rule evolution criterion, which prevents the generation of unnecessary rules and reduces over-fitting due to curse of dimensionality. The criterion relies on a newly developed re-scaled Mahalanobis distance measure for assuring monotonicity between feature weights and distance values. Gen-Smart-EFS will be evaluated based on high-dimensional real-world data (streaming) sets and compared with other well-known (evolving) fuzzy systems approaches. The results show improved accuracy with lower rule base complexity as well as smaller rule length when using Gen-Smart-EFS.

Generalized smart evolving fuzzy systems

Autonomous data stream clustering implementing split-and-merge concepts - Towards a plug-and-play approach

Class imbalance is an active research area in the machine learning community. However, existing and recent literature showed that class overlap had a higher negative impact on the performance of learning algorithms. This paper provides detailed critical discussion and objective evaluation of class overlap in the context of imbalanced data and its impact on classification accuracy. First, we present a thorough experimental comparison of class overlap and class imbalance. Unlike previous work, our experiment was carried out on the full scale of class overlap and an extreme range of class imbalance degrees. Second, we provide an in-depth critical technical review of existing approaches to handle imbalanced datasets. Existing solutions from selective literature are critically reviewed and categorised as class distribution-based and class overlap-based methods. Emerging techniques and the latest development in this area are also discussed in detail. Experimental results in this paper are consistent with existing literature and show clearly that the performance of the learning algorithm deteriorates across varying degrees of class overlap whereas class imbalance does not always have an effect. The review emphasises the need for further research towards handling class overlap in imbalanced datasets to effectively improve learning algorithms’ performance.

On the class overlap problem in imbalanced data classification.

Aiming at the problemsof too much iterative times in selecting initial centroids stochastically for K-Means algorithm,a method is proposed to optimize the initial centroids through cutting the set into k segmentations and select one point in each segmentation as initial centroids for iterative computing. A new valid function called clustering-index is defined as the sum of clustering-density and clustering-significance and can be used to search the optimization of k in the internal of [1,n(1/2) ]. The simulation experiment with IRIS data set shows that the proposed algorithm converges faster and the value k found is close to the actual value,which proves the validity of the algorithm.

An Improved K-Means Clustering Algorithm

The ability of a clustering algorithm to deal with overlapping clusters is a major indicator of its efficiency. However, the phenomenon of cluster overlapping is still not mathematically well characterized, especially in multivariate cases. In this paper, we are interested in the overlap phenomenon between Gaussian clusters, since the Gaussian mixture is a fundamental data distribution model suitable for many clustering algorithms. We introduce the novel concept of the ridge curve and establish a theory on the degree of overlap between two components. Based on this theory, we develop an algorithm for calculating the overlap rate. As an example, we use this algorithm to calculate the overlap rates between the classes in the IRIS data set and clear up some of the confusion as to the true number of classes in the data set. We investigate factors that affect the value of the overlap rate, and show how the theory can be used to generate "truthed data" as well as to measure the overlap rate between a given pair of clusters or components in a mixture. Finally, we show an example of application of the theory to evaluate the well known clustering algorithms.

Measuring the component overlapping in the Gaussian mixture model

Clustering high-dimensional data is a challenging task in data mining, and clustering high-dimensional categorical data is even more challenging because it is more difficult to measure the similarity between categorical objects. Most algorithms assume feature independence when computing similarity between data objects, or make use of computationally demanding techniques such as PCA for numerical data. Hierarchical clustering algorithms are often based on similarity measures computed on a common feature space, which is not effective when clustering highdimensional data. Subspace clustering algorithms discover feature subspaces for clusters, but are mostly partition-based; i.e. they do not produce a hierarchical structure of clusters. In this paper, we propose a hierarchical algorithm for clustering high-dimensional categorical data, based on a recently proposed information-theoretical concept named holo-entropy. The algorithm proposes new ways of exploring entropy, holo-entropy and attribute weighting in order to determine the feature subspace of a cluster and to merge clusters even though their feature subspaces differ. The algorithm is tested on UCI datasets, and compared with several state-of-the-art algorithms. Experimental results show that the proposed algorithm yields higher efficiency and accuracy than the competing algorithms and allows higher reproducibility.

Holo-Entropy Based Categorical Data Hierarchical Clustering

High dimensional data clustering is a difficult task in clustering analysis. Subspace clustering is an effective approach. The principle of subspace clustering is to maximize the retention of the original data information while searching for the minimal size of subspace for cluster representation. Based on information entropy and Holo-entropy, we propose an adaptive high dimensional weighted subspace clustering algorithm. The algorithm employs information entropy to extract the feature subspace, uses class compactness which binding Holo-entropy with weight in subspace for sub-clusters merging instead of the traditional similarity measurement method, and it selects the most compacted two sub-clusters to merge to achieve the maximum degree clustering effect. The algorithm is tested on nine UCI dataset, and compared with other algorithms. Our algorithm is better in both efficiency and accuracy than the other existing algorithms and has high reproducibility.

Haojun Sun

Papers

Measuring the component overlapping in the Gaussian mixture model

Holo-Entropy Based Categorical Data Hierarchical Clustering

A Hierarchical Clustering for Categorical Data Based on Holo-Entropy