Feature selection is an important task in data mining and machine learning to reduce the dimensionality of the data and increase the performance of an algorithm, such as a classification algorithm. However, feature selection is a challenging task due mainly to the large search space. A variety of methods have been applied to solve feature selection problems, where evolutionary computation (EC) techniques have recently gained much attention and shown some success. However, there are no comprehensive guidelines on the strengths and weaknesses of alternative approaches. This leads to a disjointed and fragmented field with ultimately lost opportunities for improving performance and successful applications. This paper presents a comprehensive survey of the state-of-the-art work on EC for feature selection, which identifies the contributions of these different algorithms. In addition, current issues and challenges are also discussed to identify promising areas for future research.

/pdf/a-survey-on-evolutionary-computation-approaches-to-feature-5c2qznm5vr.pdf

A Survey on Evolutionary Computation Approaches to Feature Selection

In machine learning, the data imbalance imposes challenges to perform data analytics in almost all areas of real-world research. The raw primary data often suffers from the skewed perspective of data distribution of one class over the other as in the case of computer vision, information security, marketing, and medical science. The goal of this article is to present a comparative analysis of the approaches from the reference of data pre-processing, algorithmic and hybrid paradigms for contemporary imbalance data analysis techniques, and their comparative study in lieu of different data distribution and their application areas.

A Systematic Review on Imbalanced Data Challenges in Machine Learning: Applications and Solutions

Because of their high dimensionality, dealing with large datasets can hinder the data mining process. Thus, the feature selection is a pre-process mandatory phase for reducing the dimensionality of datasets through using the most informative features and at the same time maximizing the classification accuracy. This paper proposes a new Grey Wolf Optimizer algorithm integrated with a Two-phase Mutation to solve the feature selection for classification problems based on the wrapper methods. The sigmoid function is used to transform the continuous search space to the binary one in order to match the binary nature of the feature selection problem. The two-phase mutation enhances the exploitation capability of the algorithm. The purpose of the first mutation phase is to reduce the number of selected features while preserving high classification accuracy. The purpose of the second mutation phase is to attempt to add more informative features that increase the classification accuracy. As the mutation phase can be time-consuming, the two-phase mutation can be done with a small probability. The wrapper methods can give high-quality solutions so we use one of the most famous wrapper methods which called k-Nearest Neighbor (k-NN) classifier. The Euclidean distance is computed to search for the k-NN. Each dataset is split into training and testing data using K-fold cross-validation to overcome the overfitting problem. Several comparisons with the most famous and modern algorithms such as flower algorithm, particle swarm optimization algorithm, multi-verse optimizer algorithm, whale optimization algorithm, and bat algorithm are done. The experiments are done using 35 datasets. Statistical analyses are made to prove the effectiveness of the proposed algorithm and its outperformance.

A new fusion of grey wolf optimizer algorithm with a two-phase mutation for feature selection

Evolving deep learning architectures for network intrusion detection using a double PSO metaheuristic

Since canonical machine learning algorithms assume that the dataset has equal number of samples in each class, binary classification became a very challenging task to discriminate the minority class samples efficiently in imbalanced datasets. For this reason, researchers have been paid attention and have proposed many methods to deal with this problem, which can be broadly categorized into data level and algorithm level. Besides, multi-class imbalanced learning is much harder than binary one and is still an open problem. Boosting algorithms are a class of ensemble learning methods in machine learning that improves the performance of separate base learners by combining them into a composite whole. This paper’s aim is to review the most significant published boosting techniques on multi-class imbalanced datasets. A thorough empirical comparison is conducted to analyze the performance of binary and multi-class boosting algorithms on various multi-class imbalanced datasets. In addition, based on the obtained results for performance evaluation metrics and a recently proposed criteria for comparing metrics, the selected metrics are compared to determine a suitable performance metric for multi-class imbalanced datasets. The experimental studies show that the CatBoost and LogitBoost algorithms are superior to other boosting algorithms on multi-class imbalanced conventional and big datasets, respectively. Furthermore, the MMCC is a better evaluation metric than the MAUC and G-mean in multi-class imbalanced data domains.

/pdf/boosting-methods-for-multi-class-imbalanced-data-2b67oa549f.pdf

Boosting methods for multi-class imbalanced data classification: an experimental review

term "data imbalance" in classification is a well established phenomenon in which data set contains unbalanced class distributions. Dataset is called unbalanced if it contains at least one class which is presented by very few examples. A range of solutions have been proposed for the problem of data imbalance including data sampling, cost evaluation of model, bagging, boosting, Genetic Programming (GP) based methods etc. This paper presents a survey of various methods introduced by researchers to handle data imbalance problem in order to improve classification performance and further the comparison between the methods on the basis of their advantages and disadvantages is done.

/pdf/a-survey-on-methods-for-solving-data-imbalance-problem-for-3ukr9u0qxd.pdf

A Survey on Methods for Solving Data Imbalance Problem for Classification

This paper presents a genetic programming (GP) based approach for designing classifiers with feature selection using a modified crossover operator [12]. The proposed GP methodology simultaneously selects a good subset of features and constructs a classifier using the selected features. For a c-class problem, it provides a classifier having c trees. To overcome the difficulties with standard crossover operator, we have used a crossover operator which discovers the best possible crossover site for a subtree and attains higher fitness values while processing fewer individuals. We have tested our method on several datasets having large number of features. We have compared the performance of our method with results available in the literature and found that the proposed method generates good results.

Construction of classifier with feature selection based on genetic programming

The concept of “bloat” in Genetic Programming is a well established phenomenon characterized by variable-length genomes gradually increasing in size during evolution. Bloat is basically a problem that occurs during crossover and mutation. In this paper we are proposing a special type of crossover operation named as Fitness, Elitism, Depth limit & Size (FEDS) crossover to reduce bloat in which we are using local elitism replacement in combination with depth limit and size of the trees to reduce bloat without a subsequent loss of performance. We are also using the point mutation technique together with the FEDS crossover in order to reduce the bloat. To demonstrate our approach we have designed a Multiclass Classifier using GP by taking few benchmark datasets. The results obtained show that by applying FEDS crossover together with point mutation reduces the problem of bloat substantially without compromising the performance.

Removing code bloating in crossover operation in Genetic Programming

The world wide web also known as WWW is the source for the largest shared information. To effectively organize, summarize and navigate through the information on the web in a fast and high quality manner, document clustering algorithms are needed. Various clustering algorithms are proposed by the researchers in which the K-means is widely used partitioning clustering algorithm which is easy for implementation, has fast convergence property in local area, and takes less time for execution. But major drawback of this method is its random choice of initial cluster centroids. To overcome this problem, an approach for document clustering using Particle Swarm Optimization (PSO) method is proposed in this paper. PSO method is applied before K-means for finding the optimal points in the search space and these points are used as initial cluster centroids for K-means algorithm to find final clusters of documents. Results of clustering algorithms are tested on four different document datasets. The outcome shows that the most efficient clustering results are generated than traditional K-means algorithm.

An approach for document clustering using PSO and K-means algorithm

This paper presents an approach for designing classifiers for a multiclass problem using Genetic Programming (GP). The proposed approach takes an integrated view of all classes when GP evolves. An individual of the population will be represented using multiple trees. The GP is trained with a set of N training samples in steps. A concept of unfitness of a tree is used in order to improve genetic evolution. Weak trees having poor performance are given more chance to participate in the genetic operations, and thus improve themselves. In this context, a new mutation operation called nondestructive directed point mutation is used, which reduces the destructive nature of mutation operation. The approach is being demonstrated by experimenting on some datasets.

Anuradha Purohit

Papers

A Survey on Methods for Solving Data Imbalance Problem for Classification

Construction of classifier with feature selection based on genetic programming

Removing code bloating in crossover operation in Genetic Programming

An approach for document clustering using PSO and K-means algorithm

A multiclass classifier using Genetic Programming