Feature selection with test cost constraint

Imbalanced problem occurs when the size of the positive class is much smaller than that of the negative one. Positive class usually refers to the main interest of the classification task. Although conventional Support Vector Machine (SVM) results in relatively robust classification performance on imbalanced datasets, it treats all samples with the same importance leading to the decision surface biasing toward the negative class. To overcome this inherent drawback, Fuzzy SVM (FSVM) is proposed by applying fuzzy membership to training samples such that different samples provide different contributions to the classifier. However, how to evaluate an appropriate fuzzy membership is the main issue to FSVM. In this paper, we propose a novel fuzzy membership evaluation which determines the fuzzy membership based on the class certainty of samples. That is, the samples with higher class certainty are assigned to larger fuzzy memberships. As the entropy is utilized to measure the class certainty, the fuzzy membership evaluation is named as entropy-based fuzzy membership evaluation. Therefore, the Entropy-based FSVM (EFSVM) is proposed by using the entropy-based fuzzy membership. EFSVM can pay more attention to the samples with higher class certainty, i.e. enhancing the importance of samples with high class certainty. Meanwhile, EFSVM guarantees the importance of the positive class by assigning positive samples to relatively large fuzzy memberships. The contributions of this work are: (1) proposing a novel entropy-based fuzzy membership evaluation method which enhances the importance of certainty samples, (2) guaranteeing the importance of the positive samples to result in a more flexible decision surface. Experiments on imbalanced datasets validate that EFSV outperforms the compared algorithms.

Entropy-based fuzzy support vector machine for imbalanced datasets

Intuitionistic fuzzy multigranulation rough sets

Feature selection in large, incomplete decision systems is a challenging problem. To avoid exponential computation in exhaustive feature selection methods, many heuristic feature selection algorithms have been presented in rough set theory. However, these algorithms are still time-consuming to compute. It is therefore necessary to investigate effective and efficient heuristic algorithms. In this paper, rough entropy-based uncertainty measures are introduced to evaluate the roughness and accuracy of knowledge. Moreover, some of their properties are derived and the relationships among these measures are established. Furthermore, compared with several representative reducts, the proposed reduction method in incomplete decision systems can provide a mathematical quantitative measure of knowledge uncertainty. Then, a heuristic algorithm with low computational complexity is constructed to improve computational efficiency of feature selection in incomplete decision systems. Experimental results show that the proposed method is indeed efficient, and outperforms other available approaches for feature selection from incomplete and complete data sets.

Feature selection using rough entropy-based uncertainty measures in incomplete decision systems

Dynamic updating of the rough approximations is a critical factor for the success of the rough set theory since data is growing at an unprecedented rate in the information-explosion era. Though many updating schemes have been proposed to study such problem, few of them were carried out in a multigranulation environment. To fill such gap, the updating of the multigranulation rough approximations is firstly explored in this paper. Both naive and fast algorithms are presented for updating the multigranulation rough approximations with the increasing of the granular structures. Different from the naive algorithm, the fast algorithm is designed based on the monotonic property of the multigranulation rough approximations. Experiments on six microarray data sets show us that the fast algorithm can effectively reduce the computational time in comparison with the naive algorithm when facing high dimensional data sets. Moreover, it is also shown that fast algorithm is useful in decreasing the computational time of finding both traditional reduct and attribute clustering based reduct.

Updating multigranulation rough approximations with increasing of granular structures

Feature selection is viewed as an important preprocessing step for pattern recognition, machine learning and data mining Traditional hill-climbing search approaches to feature selection have difficulties to find optimal reducts And the current stochastic search strategies, such as GA, ACO and PSO, provide a more robust solution but at the expense of increased computational effort It is necessary to investigate fast and effective search algorithms Rough set theory provides a mathematical tool to discover data dependencies and reduce the number of features contained in a dataset by purely structural methods In this paper, we define a structure called power set tree (PS-tree), which is an order tree representing the power set, and each possible reduct is mapped to a node of the tree Then, we present a rough set approach to feature selection based on PS-tree Two kinds of pruning rules for PS-tree are given And two novel feature selection algorithms based on PS-tree are also given Experiment results demonstrate that our algorithms are effective and efficient

A rough set approach to feature selection based on power set tree

An entropy-based uncertainty measurement approach in neighborhood systems

Uncertainty measures can supply new points of view for analyzing data and help us to disclose the substantive characteristics of data sets. Accuracy and roughness proposed by Pawlak are mainly two important measures to deal with uncertainty information in rough set theory. However, these measures are constructed by set operations, which bring about poor efficiency. In this paper, we proposed a binary granule representation by a conversion from a set to a binary granule. Correspondingly, set operations are converted to binary granule computing. Besides, we investigate how to understand measures from rough set framework in the viewpoint of binary granule representation by introducing Hamming distance between granules. Moreover, a concept of granule swarm distance is defined for measuring uncertainty between two granule swarms, which provides a more comprehensible perspective for measures in rough set theory. Theoretical analysis shows that the binary granule representation is valuable to understand rough set measures.

A binary granule representation for uncertainty measures in rough set theory

In recent years, the deep neural network technology has developed rapidly and has been effective in processing and analyzing images, videos, sounds and many other aspects. However, traditional neural networks only learn the samples themselves and ignore the differences between samples. When the network depth is too deep, network degradation phenomenon may occur. To solve these drawbacks, we propose the fuzzy granular deep convolutional network with residual structures. Firstly, we define the concept, operation and correlation measures of fuzzy granules, and construct the granulation reference system by random sampling for fuzzy granulation. Furthermore, modules of granular neuron, granular activation function, granular convolution, and granular residual are defined, and the fuzzy granular deep convolutional network with residual structures is built. The loss functions and learning algorithm are designed for the granular neural network, and the fuzzy granular neural network is successfully trained. The fuzzy granular neural network owns the characteristics of multi-granularity, multi-angle and structured, which has good generalization performance. Finally, experiments are conducted on the Cifar100 and Tiny Imagenet datasets. The experimental results show the effectiveness of the residual granular neural network. We further found that the granular neural network could alleviate the degradation problem. Moreover granular neural network with fewer hidden layers can achieve the function of a neural network with more hidden layers. 

Fuzzy granular deep convolutional network with residual structures

Linear regression is the use of linear functions to model the relationship between a dependent variable and one or more independent variables. Linear regression models have been widely used in various fields such as finance, industry, and medicine. To address the problem that the traditional linear regression model is difficult to handle uncertain data, we propose a granule-based elastic network regression model. First we construct granules and granular vectors by granulation methods. Then, we define multiple granular operation rules so that the model can effectively handle uncertain data. Further, the granular norm and the granular vector norm are defined to design the granular loss function and construct the granular elastic network regression model. After that, we conduct the derivative of the granular loss function and design the granular elastic network gradient descent optimization algorithm. Finally, we performed experiments on the UCI datasets to verify the validity of the granular elasticity network. We found that the granular elasticity network has the advantage of good fit compared with the traditional linear regression model.

/pdf/granular-elastic-network-regression-with-stochastic-gradient-354pogoh.pdf

Keshou Wu

Papers

A rough set approach to feature selection based on power set tree

An entropy-based uncertainty measurement approach in neighborhood systems

A binary granule representation for uncertainty measures in rough set theory

Fuzzy granular deep convolutional network with residual structures

Granular Elastic Network Regression with Stochastic Gradient Descent