Review of swarm intelligence-based feature selection methods

In this paper, for the first time, a feature selection procedure is modeled as a multi-criteria decision making (MCDM) process. This method is applied to a multi-label data and we have used the TOPSIS (Technique of Order Preference by Similarity to Ideal Solution) method as a famous MCDM algorithm to evaluate the features based on their relationship with multiple labels as different criteria. Our proposed method, Multi-label Feature Selection using Multi-Criteria Decision Making (MFS-MCDM) which treated the multi-label feature selection as an information fusion process, first obtains a decision matrix using the ridge regression algorithm and then calculates the weight of each column of this matrix based on the entropy of each label. Then, the TOPSIS approach is used to assign a score to each feature based on the weighted decision matrix. Finally, a rank vector for the features is generated as output which the user can select a desired number of features. The superiority of the proposed method is shown in experimental results comparing to other similar methods in terms of all evaluation metrics.

MFS-MCDM: Multi-label feature selection using multi-criteria decision making

Recently, multilabel classification has generated considerable research interest. However, the high dimensionality of multilabel data incurs high costs; moreover, in many real applications, a number of labels of training samples are randomly missed. Thus, multilabel classification can have great complexity and ambiguity, which means some feature selection methods exhibit poor robustness and yield low prediction accuracy. To solve these issues, this paper presents a novel feature selection method based on multilabel fuzzy neighborhood rough sets (MFNRS) and maximum relevance minimum redundancy (MRMR) that can be used on multilabel data with missing labels. First, to handle multilabel data with missing labels, a relation coefficient of samples, label complement matrix, and label-specific feature matrix are constructed and implemented in a linear regression model to recover missing labels. Second, the margin-based fuzzy neighborhood radius, fuzzy neighborhood similarity relationship, and fuzzy neighborhood information granule are developed. The MFNRS model is built based on multilabel neighborhood rough sets combined with fuzzy neighborhood rough sets. Based on algebra and information views, certain fuzzy neighborhood entropy-based uncertainty measures are proposed for MFNRS. The fuzzy neighborhood mutual information-based MRMR model with label correlation is improved to evaluate the performance of candidate features. Finally, a feature selection algorithm is designed to improve the performance for multilabel data with missing labels. Experiments on twenty datasets verify that our method is effective not only for recovering missing labels but also for selecting significant features with better classification performance.

Feature Selection with Missing Labels Using Multilabel Fuzzy Neighborhood Rough Sets and Maximum Relevance Minimum Redundancy

The existing three-way decision (TWD)-making methods cannot effectively handle incomplete multiattribute decision-making (MADM) problems in real life; it is necessary to explore an effective three-way MADM model in incomplete fuzzy decision systems (IFDSs). First, we consider the preference of decision makers for each alternative and introduce the concept of predecisions, thus an IFDS is obtained. Then, the weighted conditional probabilities are calculated with the aid of the defined similarity relation. Subsequently, we introduce the notion of relative utility functions and then present an approach to determine the relative utility function values. Afterward, we construct a TWD model in IFDSs and apply it to the modeling of incomplete MADM (IMADM) problems. Our study not only enriches TWD and MADM theories but also provides a new perspective for realistic IMADM problems. At last, the results of comparative and experimental analyses demonstrate the validity, stability, and superiority of our proposed model. 

A Novel Three-Way Decision Model Based on Utility Theory in Incomplete Fuzzy Decision Systems

Gene selection for microarray data classification via multi-objective graph theoretic-based method

Feature selection with rough sets aims to delete redundant conditional features from static data by considering single type features. However, traditional feature selection methods generally ignore real-world scenarios: hybrid conditional feature set including missing, categorical and numerical ones coexists in the data, and the object set may change dynamically in one by one over time. To deal with dynamic hybrid data with mixed-type features, we propose a neighborhood entropy-based incremental feature selection framework by neighborhood rough set model. In this paper, the dynamics of an object set involves the change of a single object and multiple objects. Therefore, two incremental feature selection algorithms are developed for hybrid data with the dynamic change of a single object and multiple objects, respectively. At first, an incremental manner is utilized to compute the neighborhood entropy as feature criterion. On this basis, the incremental computations of feature significance are used to select candidate features in a descending order. Meanwhile, a deletion strategy is employed to filter out redundant features from the selection results. Finally, experimental results on different real-life data sets demonstrate the proposed incremental algorithms can outperform the non-incremental algorithm for feature selection in speed within comparable classification accuracy. Especially for multiple objects adding into and deleting from the hybrid data, the incremental algorithm is more efficient to select a subset of features than the algorithm for handling the dynamic change of a single object.

Incremental feature selection for dynamic hybrid data using neighborhood rough set

Attribute reduction in incomplete ordered information systems with fuzzy decision

Label distribution feature selection for multi-label classification with rough set

Feature selection used for dimensionality reduction of the feature space plays an important role in multi-label learning where high-dimensional data are involved. Although most existing multi-label feature selection approaches can deal with the problem of label ambiguity which mainly focuses on the assumption of uniform distribution with logical labels, it cannot be applied to many practical applications where the significance of related label for every instance tends to be different. To deal with this issue, in this study, label distribution learning covered with a certain real number of labels is introduced to design a model for the labeling-significance. Nevertheless, multi-label feature selection is limited to handling only labels consisting of logical relations. In order to solve this problem, combining the random variable distribution with granular computing, we first propose a label enhancement algorithm to transform logical labels in multi-label data into label distribution with more supervised information, which can mine the hidden label significance from every instance. On this basis, to remove some redundant or irrelevant features in multi-label data, a label distribution feature selection algorithm using mutual information and label enhancement is developed. Finally, the experimental results show that the performance of the proposed method is superior to the other state-of-the-art approaches when dealing with multi-label data.

Wenbin Qian

Papers

Incremental feature selection for dynamic hybrid data using neighborhood rough set

Attribute reduction in incomplete ordered information systems with fuzzy decision

Label distribution feature selection for multi-label classification with rough set

Mutual information-based label distribution feature selection for multi-label learning

Feature selection based on label distribution and fuzzy mutual information