scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

A Simple Method of Solution For Multi-label Feature Selection

TL;DR: This work proposed an efficient modification of a recent multi-label feature selection algorithm available in literature, which decomposes the output label space into lower dimensions using simple matrix factorization method and employs feature selection process in the decoupled reduced space.
Abstract: Multi-label learning has been a topic of research interest in multimedia, text & speech recognitions, music, image processing, information retrieval etc. In Multi-label classification (MLC) each instance is associated with a set of multiple class labels. Like other machine learning algorithms, data preprocessing plays an key role in MLC. Feature selection is an important preprocessing step in MLC, due to high dimensionality of datasets and associated computational costs.Extracting the most informative features considerably reduces the computational loads of MLC. Most of the Multi-label feature selection algorithms available in literature involve conversions to multiple single labeled feature selection problems. We proposed an efficient modification of a recent multi-label feature selection algorithm [1] available in literature. Our algorithm consists of two steps: in the first step we decompose the output label space into lower dimensions using simple matrix factorization method; subsequently we employ feature selection process in the decoupled reduced space. Our simulations with real world datasets reveal the efficiency of proposed framework.
Citations
More filters
Journal ArticleDOI
11 Apr 2021
TL;DR: This article analyses such methods like LASSO, Boruta, Recursive Feature Elimination (RFE), Regularised Random Forest (RRF) and DALEX, and discusses from the obtained features and the selected features with respect to the method chosen for study.
Abstract: Feature selection has predominant importance in various kinds of applications. However, it is still considered as a cumbersome process to identify the vital features among the available set for the problem taken for study. The researchers proposed wide variety of techniques over the period of time which concentrate on its own. Some of the existing familiar methods include Particle Swarm Optimisation (PSO), Genetic Algorithm (GA) and Simulated Annealing (SA). While some of the methods are existing, the emerging methods provide promising results compared with them. This article analyses such methods like LASSO, Boruta, Recursive Feature Elimination (RFE), Regularised Random Forest (RRF) and DALEX. The dataset of variant sizes is considered to assess the importance of feature selection out of the available features. The results are also discussed from the obtained features and the selected features with respect to the method chosen for study.

4 citations

Journal ArticleDOI
TL;DR: Comparative studies over ten benchmark data sets validate the effectiveness of the proposed algorithm in multi-label classification and estimate the label correlations and infer model parameters simultaneously via the local geometric structure of the input to achieve mutual enhancement.
Abstract: —Exploiting label correlations is important to multi-label classification. Previous methods capture the high-order label correlations mainly by transforming the label matrix to a latent label space with low-rank matrix factorization. However, the label matrix is generally a full-rank or approximate full-rank matrix, making the low-rank factorization inappropriate. Besides, in the latent space, the label correlations will become implicit. To this end, we propose a simple yet effective method to depict the high-order label correlations explicitly, and at the same time maintain the high-rank of the label matrix. Moreover, we estimate the label correlations and infer model parameters simultaneously via the local geometric structure of the input to achieve mutual enhancement. Comparative studies over ten benchmark data sets validate the effectiveness of the proposed algorithm in multi-label classification. The exploited high-order label correlations are consistent with common sense empirically. Our code is publicly available at https://github.com/601175936/HOMI .
References
More filters
Journal ArticleDOI
21 Oct 1999-Nature
TL;DR: An algorithm for non-negative matrix factorization is demonstrated that is able to learn parts of faces and semantic features of text and is in contrast to other methods that learn holistic, not parts-based, representations.
Abstract: Is perception of the whole based on perception of its parts? There is psychological and physiological evidence for parts-based representations in the brain, and certain computational theories of object recognition rely on such representations. But little is known about how brains or computers might learn the parts of objects. Here we demonstrate an algorithm for non-negative matrix factorization that is able to learn parts of faces and semantic features of text. This is in contrast to other methods, such as principal components analysis and vector quantization, that learn holistic, not parts-based, representations. Non-negative matrix factorization is distinguished from the other methods by its use of non-negativity constraints. These constraints lead to a parts-based representation because they allow only additive, not subtractive, combinations. When non-negative matrix factorization is implemented as a neural network, parts-based representations emerge by virtue of two properties: the firing rates of neurons are never negative and synaptic strengths do not change sign.

11,500 citations


"A Simple Method of Solution For Mul..." refers methods in this paper

  • ...Simple Matrix Factorization Simple matrix factorization (SMF) [17] methods have been used in many field successfully and in collaborative filtering several such algorithm are used routinely....

    [...]

01 Jan 1999
TL;DR: In this article, non-negative matrix factorization is used to learn parts of faces and semantic features of text, which is in contrast to principal components analysis and vector quantization that learn holistic, not parts-based, representations.
Abstract: Is perception of the whole based on perception of its parts? There is psychological and physiological evidence for parts-based representations in the brain, and certain computational theories of object recognition rely on such representations. But little is known about how brains or computers might learn the parts of objects. Here we demonstrate an algorithm for non-negative matrix factorization that is able to learn parts of faces and semantic features of text. This is in contrast to other methods, such as principal components analysis and vector quantization, that learn holistic, not parts-based, representations. Non-negative matrix factorization is distinguished from the other methods by its use of non-negativity constraints. These constraints lead to a parts-based representation because they allow only additive, not subtractive, combinations. When non-negative matrix factorization is implemented as a neural network, parts-based representations emerge by virtue of two properties: the firing rates of neurons are never negative and synaptic strengths do not change sign.

9,604 citations

Journal ArticleDOI
TL;DR: Applications to two real-world multilabel learning problems, i.e., functional genomics and text categorization, show that the performance of BP-MLL is superior to that of some well-established multILabel learning algorithms.
Abstract: In multilabel learning, each instance in the training set is associated with a set of labels and the task is to output a label set whose size is unknown a priori for each unseen instance. In this paper, this problem is addressed in the way that a neural network algorithm named BP-MLL, i.e., backpropagation for multilabel learning, is proposed. It is derived from the popular backpropagation algorithm through employing a novel error function capturing the characteristics of multilabel learning, i.e., the labels belonging to an instance should be ranked higher than those not belonging to that instance. Applications to two real-world multilabel learning problems, i.e., functional genomics and text categorization, show that the performance of BP-MLL is superior to that of some well-established multilabel learning algorithms

1,075 citations


"A Simple Method of Solution For Mul..." refers background in this paper

  • ...INTRODUCTION MLC problems have been found to be useful in diverse fields like gene & protein function classification [15], music categorization [16], Text classification [12], identification of drug side effects [14] and semantic scene classification [6]....

    [...]

Journal ArticleDOI
TL;DR: Empirical evidence indicates that RAkEL manages to improve substantially over LP, especially in domains with large number of labels and exhibits competitive performance against other high-performing multilabel learning methods.
Abstract: A simple yet effective multilabel learning method, called label powerset (LP), considers each distinct combination of labels that exist in the training set as a different class value of a single-label classification task. The computational efficiency and predictive performance of LP is challenged by application domains with large number of labels and training examples. In these cases, the number of classes may become very large and at the same time many classes are associated with very few training examples. To deal with these problems, this paper proposes breaking the initial set of labels into a number of small random subsets, called labelsets and employing LP to train a corresponding classifier. The labelsets can be either disjoint or overlapping depending on which of two strategies is used to construct them. The proposed method is called RAkEL (RAndom k labELsets), where k is a parameter that specifies the size of the subsets. Empirical evidence indicates that RAkEL manages to improve substantially over LP, especially in domains with large number of labels and exhibits competitive performance against other high-performing multilabel learning methods.

795 citations


"A Simple Method of Solution For Mul..." refers methods in this paper

  • ...These methods include Binary Relevance [8], Label Powerset, Classifier Chains, Calibrated Label Ranking and Random k-label sets [9]....

    [...]

Proceedings Article
01 Jan 2008
TL;DR: In this paper, the automated detection of emotion in music is modeled as a multilabel classification task, where a piece of music may belong to more than one class.
Abstract: In this paper, the automated detection of emotion in music is modeled as a multilabel classification task, where a piece of music may belong to more than one class. Four algorithms are evaluated and compared in this task. Furthermore, the predictive power of several audio features is evaluated using a new multilabel feature selection method. Experiments are conducted on a set of 593 songs with 6 clusters of music emotions based on the Tellegen-Watson-Clark model. Results provide interesting insights into the quality of the discussed algorithms and features.

711 citations


"A Simple Method of Solution For Mul..." refers background in this paper

  • ...INTRODUCTION MLC problems have been found to be useful in diverse fields like gene & protein function classification [15], music categorization [16], Text classification [12], identification of drug side effects [14] and semantic scene classification [6]....

    [...]