In multi-label classification, the strategy of label-specific features has been shown to be effective to learn from multi-label examples by accounting for the distinct discriminative properties of each class label. However, most existing approaches exploit the semantic relations among labels as immutable prior knowledge, which may not be appropriate to constrain the learning process of label-specific features. In this paper, we propose to learn label semantics and label-specific features in a collaborative way. Accordingly, a deep neural network (DNN) based approach named CLIF, i.e. Collaborative Learning of label semantIcs and deep label-specific Features for multi-label classification, is proposed. By integrating a graph autoencoder for encoding semantic relations in the label space and a tailored feature-disentangling module for extracting label-specific features, CLIF is able to employ the learned label semantics to guide mining label-specific features and propagate label-specific discriminative properties to the learning process of the label semantics. In such a way, the learning of label semantics and label-specific features interact and facilitate with each other so that label semantics can provide more accurate guidance to label-specific feature learning. Comprehensive experiments on 14 benchmark data sets show that our approach outperforms other well-established multi-label classification algorithms. 

Collaborative Learning of Label Semantics and Deep Label-Specific Features for Multi-Label Classification

In multi-dimensional classification (MDC), a number of class variables are assumed in the output space with each of them specifying the class membership w.r.t. one heterogeneous class space. One major challenge in learning from MDC examples lies in the heterogeneity of class spaces, where the modeling outputs from different class spaces are not directly comparable. To tackle this problem, we propose a new strategy named decomposed label encoding which enables modeling alignment for MDC in an encoded label space derived from one-vs-one (OvO) decomposition. Specifically, the original MDC output space is transformed into a ternary encoded label space by conducting OvO decomposition w.r.t. each class space. Then, the manifold structure in the feature space is exploited to enrich the labeling information in the encoded label space. Finally, the predictive model is induced by fitting the metric-aligned modeling outputs with enriched labeling information. Extensive experiments over twenty benchmark data sets clearly show the superiority of the proposed MDC strategy against state-of-the-art approaches.

Multi-Dimensional Classification via Decomposed Label Encoding

Automated label prediction tools can help developers manage and categorize issues on GitHub. However, different open-source projects use various forms of labels with the same meaning. Previous label prediction methods mainly solve the problem of the synonymous labels by manual preprocessing rules, but these preprocessing rules can only identify synonyms with the same prefix or suffix. These factors inspire us to propose a method to identify these synonymous labels automatically and recommend personalized labels for different open-source projects. In this paper, we propose a Personalizing Label Prediction framework for Issues named PLPI. PLPI identifies labels with similar meanings by representing labels as semantic vectors and applying clustering methods . PLPI can predict personalized labels from the existing labels in the open-source project. We conduct a comprehensive study to compare seven commonly adopted labeling models with our approach. The experimental results demonstrate the advantages of our approach. Finally, we show some representative examples and discuss the visualization results of synonyms clustering by dimension reduction. The experimental results show that our method PLPI can improve label prediction performance and provide personalized label recommendation results for different open-source projects. 

Personalizing label prediction for GitHub issues

Multi-dimensional classification (MDC) assumes heterogeneous class spaces for each example, where class variables from different class spaces characterize semantics of the example along different dimensions. The heterogeneity of class spaces leads to incomparability of the modeling outputs from different class spaces, which is the major difficulty in designing MDC approaches. In this article, we make a first attempt toward adapting maximum margin techniques for MDC problem and a novel approach named M3MDC is proposed. Specifically, M3MDC maximizes the margins between each pair of class labels with respect to individual class variable while modeling relationship across class variables (as well as class labels within individual class variable) via covariance regularization. The resulting formulation admits convex objective function with nonlinear constraints, which can be solved via alternating optimization with quadratic programming (QP) or closed-form solution in either alternating step. Comparative studies on the most comprehensive real-world MDC datasets to date are conducted and it is shown that M3MDC achieves highly competitive performance against state-of-the-art MDC approaches. 

Maximum Margin Multi-Dimensional Classification

As an important model in multi-label learning, ProSVM considers two problems simultaneously: one is to distinguish the relevant labels from the irrelevant labels of instances, and the other is to rank the relevant labels. Thus it produces better generalization ability in comparison with other multi-label models. However, ProSVM leads to high computational cost when the label space is enormous since it employs the label-pairs instead of the instances in the training process. To effectively deal with this defect, a safe screening rule is proposed to speed up the learning process, named PSSR. Different from the existing safe screening rule, our screening method is constructed based on solving the primal problem of ProSVM rather than the dual one. So it can address large-scale problems. To the best of our knowledge, it is the first safe screening rule for the primal problem in multi-label learning. Experimental results on eight benchmark datasets demonstrate the superiority of PSSR. Finally, our PSSR is applied to the clinical data of diabetic patients, and obtain better performance.

Safe instance screening for primal multi-label ProSVM

Multi-label classification aims to assign a set of proper labels for each instance, where distance metric learning can help improve the generalization ability of instance-based multi-label classification models. Existing multi-label metric learning techniques work by utilizing pairwise constraints to enforce that examples with similar label assignments should have close distance in the embedded feature space. In this paper, a novel distance metric learning approach for multi-label classification is proposed by modeling structural interactions between instance space and label space. On one hand, compositional distance metric is employed which adopts the representation of a weighted sum of rank-1 PSD matrices based on component bases. On the other hand, compositional weights are optimized by exploiting triplet similarity constraints derived from both instance and label spaces. Due to the compositional nature of employed distance metric, the resulting problem admits quadratic programming formulation with linear optimization complexity w.r.t. the number of training examples. We also derive the generalization bound for the proposed approach based on algorithmic robustness analysis of the compositional metric. Extensive experiments on sixteen benchmark data sets clearly validate the usefulness of compositional metric in yielding effective distance metric for multi-label classification.

/pdf/compositional-metric-learning-for-multi-label-classification-3pafszcjmk.pdf

Yan-Ping Sun

Papers

Compositional metric learning for multi-label classification