Manifold regularized discriminative feature selection for multi-label learning

Nowadays, with emerge the multi-label datasets, the multi-label learning processes attracted interest and increasingly applied to different fields. In such learning processes, unlike single-label learning, instances have more than one class label simultaneously. Also, multi-label learning suffers from the curse of dimensionality, and thus, the feature selection becomes a difficult task. In this paper, we propose a novel multi-label relevance–redundancy feature selection method based on Ant colony optimization (ACO) for the first time, called MLACO. By introducing two unsupervised and supervised heuristic functions, MLACO tries to search in the features space to find the most promising features with the lowest redundancy (unsupervised) and highest relevancy with class labels (supervised) through several iterations. For speeding up the convergence of the algorithm, the normalized cosine similarity between features and class labels have been used as the initial pheromone of each ant. The proposed method does not take into account any learning algorithm, and it can be classified as a filter-based method. We compare the performance of the MLACO against five well-known and state-of-the-art feature selection methods using ML-KNN classifier. The experimental results on several frequently used datasets show the superiority of the MLACO in different multi-label evaluation measures criteria and runtime.

MLACO: A multi-label feature selection algorithm based on ant colony optimization

Improving multi-label classification with missing labels by learning label-specific features

Multi-label learning generalizes traditional learning by allowing an instance to belong to multiple labels simultaneously. This causes multi-label data to be characterized by its large label space dimensionality and the dependencies among labels. These challenges have been addressed by feature selection techniques which improve the final model accuracy. However, the large number of features along with a large number of labels call for new approaches to manage data effectively and efficiently in distributed computing environments. This paper proposes a distributed model to compute a score that measures the quality of each feature with respect to multiple labels on Apache Spark. We propose two different approaches that study how to aggregate the mutual information of multiple labels: Euclidean Norm Maximization (ENM) and Geometric Mean Maximization (GMM). The former selects the features with the largest L 2 -norm whereas the latter selects the features with the largest geometric mean. Experiments compare 9 distributed multi-label feature selection methods on 12 datasets and 12 metrics. Results validated through statistical analysis indicate that ENM is able to outperform the reference methods by maximizing the relevance while minimizing the redundancy of the selected features in constant selection time.

Distributed multi-label feature selection using individual mutual information measures

In this paper, for the first time, a feature selection procedure is modeled as a multi-criteria decision making (MCDM) process. This method is applied to a multi-label data and we have used the TOPSIS (Technique of Order Preference by Similarity to Ideal Solution) method as a famous MCDM algorithm to evaluate the features based on their relationship with multiple labels as different criteria. Our proposed method, Multi-label Feature Selection using Multi-Criteria Decision Making (MFS-MCDM) which treated the multi-label feature selection as an information fusion process, first obtains a decision matrix using the ridge regression algorithm and then calculates the weight of each column of this matrix based on the entropy of each label. Then, the TOPSIS approach is used to assign a score to each feature based on the weighted decision matrix. Finally, a rank vector for the features is generated as output which the user can select a desired number of features. The superiority of the proposed method is shown in experimental results comparing to other similar methods in terms of all evaluation metrics.

MFS-MCDM: Multi-label feature selection using multi-criteria decision making

In multi-label learning, different labels may have their own inherent characteristics for distinguishing each other, in the meanwhile, exploiting the correlations among labels is another practical yet challenging task to improve the performance. In this work, we present a new method for the joint learning of label-specific features and label correlations. The key is the design of an optimization framework to learn the weight assignment scheme of features, and the correlations among labels are taken into account by constructing additional features at the same time. Through iteratively optimizing the two sets of unknown variables, which are referred to feature weights and label correlations-based features, label-specific features of each label are available to achieve multi-label classification. Comprehensive experiments on various multi-label data sets including two collected traditional Chinese medicine data sets reveal the advantages of our proposed algorithm.

Multi-label learning with label-specific features by resolving label correlations

Mutual information based multi-label feature selection via constrained convex optimization

The goal of TCM state identification is to identify the patient's syndromes and locations and natures of diseases according to symptoms. Generally, symptoms of a patient are associated with several syndromes and multiple locations and natures of diseases; hence, the TCM state identification is a typical multi‐label problem. In this paper, a new method is proposed to predict syndromes and locations and natures of diseases according to the diagnostic information of TCM. In detail, the correlation between features and the correlation between class labels are combined into a new uniform feature space. After that, the MDMR algorithm is used to select the most discriminatory features from the new uniform feature space, which is helpful to reduce the data dimensionality. Lastly, a KNN‐like algorithm is modified to calculate the label similarity of test data, and the finite set of labels of test data is predicted by ML‐KNN. In this paper, the test data is collected by Fujian University of Traditional Chinese Medicine according to the theory of TCM and medical ethics. The experiments show that the performance of the proposed method is superior to some other popular methods and is helpful in the identification of health state in TCM.

Multi‐label feature selection with application to TCM state identification

In recent years, multilabel learning has been applied to a lot of application areas and is yet a challenging task. In multilabel learning, an instance often belongs to multiple class labels simultaneously. The labels usually have correlations with others, and mining label correlations is helpful to enhance the multilabel classification performance. Aiming at increasing the accuracy of prediction, Label embedding (LE) is an important technique, and conducive to extracting label information for multilabel learning. In this paper, we present a novel multilabel learning approach via exploiting label correlations, which can be naturally extended to tackle feature selection problem. First, to obtain the discriminative features shared by all labels, the proposed algorithm learns a latent space by employing deep canonical correlation analysis. Then we exploit label correlations by enforcing predictions on similar labels to be similar, thereby improving the prediction performance. Results on several multiple datasets illustrate that the proposed algorithm has the advantages on multilabel classification and feature selection.

Liang Dai

Papers

Multi-label learning with label-specific features by resolving label correlations

Mutual information based multi-label feature selection via constrained convex optimization

Multi‐label feature selection with application to TCM state identification

Joint multilabel classification and feature selection based on deep canonical correlation analysis

Toward embedding-based multi-label feature selection with label and feature collaboration