Feature selection using neighborhood entropy-based uncertainty measures for gene expression data classification

The concept of dependency in a neighborhood rough set model is an important evaluation function for the feature selection. This function considers only the classification information contained in the lower approximation of the decision while ignoring the upper approximation. In this paper, we construct a class of uncertainty measures: decision self-information for the feature selection. These measures take into account the uncertainty information in the lower and the upper approximations. The relationships between these measures and their properties are discussed in detail. It is proven that the fourth measure, called relative neighborhood self-information, is better for feature selection than the other measures, because not only does it consider both the lower and the upper approximations but also the change of its magnitude is largest with the variation of feature subsets. This helps to facilitate the selection of optimal feature subsets. Finally, a greedy algorithm for feature selection has been designed and a series of numerical experiments was carried out to verify the effectiveness of the proposed algorithm. The experimental results show that the proposed algorithm often chooses fewer features and improves the classification accuracy in most cases.

Feature Selection Based on Neighborhood Self-Information

For heterogeneous data sets containing numerical and symbolic feature values, feature selection based on fuzzy neighborhood multigranulation rough sets (FNMRS) is a very significant step to preprocess data and improve its classification performance. This article presents an FNMRS-based feature selection approach in neighborhood decision systems. First, some concepts of fuzzy neighborhood rough sets and neighborhood multigranulation rough sets are given, and then the FNMRS model is investigated to construct uncertainty measures. Second, the optimistic and pessimistic FNMRS models are built by using fuzzy neighborhood multigranulation lower and upper approximations from algebra view, and some fuzzy neighborhood entropy-based uncertainty measures are developed in information view. Inspired by both algebra and information views based on the FNMRS model, the fuzzy neighborhood pessimistic multigranulation entropy is proposed. Third, the Fisher score model is utilized to delete irrelevant features to decrease the complexity of high-dimensional data sets, and then, a forward feature selection algorithm is provided to promote the performance of heterogeneous data classification. Experimental results on 12 data sets show that the presented model is effective for selecting important features with the higher stability of classification in neighborhood decision systems.

Feature Selection Using Fuzzy Neighborhood Entropy-Based Uncertainty Measures for Fuzzy Neighborhood Multigranulation Rough Sets

Tumor classification is one of the most vital technologies for cancer diagnosis. Due to the high dimensionality, gene selection (finding a small, closely related gene set to accurately classify tumor) is an important step for improving gene expression data classification performance. Traditional rough set model as a classical attribute reduction method deals with discrete data only. As for the gene expression data containing real-value or noisy data, they are usually employed by a discrete preprocessing, which may result in poor classification accuracy. In this paper, a novel neighborhood rough sets and entropy measure-based gene selection with Fisher score for tumor classification is proposed, which has the ability of dealing with real-value data whilst maintaining the original gene classification information. First, the Fisher score method is employed to eliminate irrelevant genes to significantly reduce computation complexity. Next, some neighborhood entropy-based uncertainty measures are investigated for handling the uncertainty and noisy of gene expression data. Moreover, some of their properties are derived and the relationships among these measures are established. Finally, a joint neighborhood entropy-based gene selection algorithm with the Fisher score is presented to improve the classification performance of gene expression data. The experimental results under an instance and several public gene expression data sets prove that the proposed method is very effective for selecting the most relevant genes with high classification accuracy.

Joint neighborhood entropy-based gene selection method with fisher score for tumor classification

For incomplete data with mixed numerical and symbolic attributes, attribute reduction based on neighborhood multi-granulation rough sets (NMRS) is an important method to improve the classification performance. However, most classical attribute reduction methods can only handle finite sets as to produce more attributes and lower classification accuracy. This paper proposes a novel NMRS-based attribute reduction method using Lebesgue and entropy measures in incomplete neighborhood decision systems. First, some concepts of optimistic and pessimistic NMRS models in incomplete neighborhood decision systems are given, respectively. Then, a Lebesgue measure is combined with NMRS to study neighborhood tolerance class-based uncertainty measures. To analyze the uncertainty, noise and redundancy of incomplete neighborhood decision systems in detail, some neighborhood multi-granulation entropy-based uncertainty measures are developed by integrating Lebesgue and entropy measures. Inspired by both algebraic view with information view in NMRS, the pessimistic neighborhood multi-granulation dependency joint entropy is proposed. What is more, the corresponding properties are further deduced and the relationships among these measures are discussed, which can help to investigate the uncertainty of incomplete neighborhood decision systems. Finally, the Fisher linear discriminant method is used to eliminate irrelevant attributes to significantly reduce computational complexity for high-dimensional datasets, and a heuristic attribute reduction algorithm with complexity analysis is designed to improve classification performance of incomplete and mixed datasets. Experimental results under seven UCI datasets and eight gene expression datasets illustrate that the proposed method is effective to select most relevant attributes with higher classification accuracy, as compared with representative algorithms.

Jianzhong Zheng

Papers

Gene selection for tumor classification using neighborhood rough sets and entropy measures