https://fada.birzeit.edu/bitstream/20.500.11889/5314/1/Hybrid%20Whale%20Optimization%20Algorithm%20with%20Simulated%20Annealing%20for%20FS.pdf

Hybrid Whale Optimization Algorithm with Simulated Annealing for Feature Selection

Searching for the (near) optimal subset of features is a challenging problem in the process of feature selection (FS). In the literature, Swarm Intelligence (SI) algorithms show superior performance in solving this problem. This motivated our attempts to test the performance of the newly proposed Salp Swarm Algorithm (SSA) in this area. As such, two new wrapper FS approaches that use SSA as the search strategy are proposed. In the first approach, eight transfer functions are employed to convert the continuous version of SSA to binary. In the second approach, the crossover operator is used in addition to the transfer functions to replace the average operator and enhance the exploratory behavior of the algorithm. The proposed approaches are benchmarked on 22 well-known UCI datasets and the results are compared with 5 FS methods: Binary Grey Wolf Optimizer (BGWO), Binary Gravitational Search Algorithms (BGSA), Binary Bat Algorithm (BBA), Binary Particle Swarm Optimization (BPSO), and Genetic Algorithm (GA). The paper also considers an extensive study of the parameter setting for the proposed technique. From the results, it is observed that the proposed approach significantly outperforms others on around 90% of the datasets.

/pdf/an-efficient-binary-salp-swarm-algorithm-with-crossover-9kyl7dt8j9.pdf

An efficient binary Salp Swarm Algorithm with crossover scheme for feature selection problems

Searching for the optimal subset of features is known as a challenging problem in feature selection process. To deal with the difficulties involved in this problem, a robust and reliable optimization algorithm is required. In this paper, Grasshopper Optimization Algorithm (GOA) is employed as a search strategy to design a wrapper-based feature selection method. The GOA is a recent population-based metaheuristic that mimics the swarming behaviors of grasshoppers. In this work, an efficient optimizer based on the simultaneous use of the GOA, selection operators, and Evolutionary Population Dynamics (EPD) is proposed in the form of four different strategies to mitigate the immature convergence and stagnation drawbacks of the conventional GOA. In the first two approaches, one of the top three agents and a randomly generated one are selected to reposition a solution from the worst half of the population. In the third and fourth approaches, to give a chance to the low fitness solutions in reforming the population, Roulette Wheel Selection (RWS) and Tournament Selection (TS) are utilized to select the guiding agent from the first half. The proposed GOA_EPD approaches are employed to tackle various feature selection tasks. The proposed approaches are benchmarked on 22 UCI datasets. The comprehensive results and various comparisons reveal that the EPD has a remarkable impact on the efficacy of the GOA and using the selection mechanism enhanced the capability of the proposed approach to outperform other optimizers and find the best solutions with improved convergence trends. Furthermore, the comparative experiments demonstrate the superiority of the proposed approaches when compared to other similar methods in the literature.

/pdf/evolutionary-population-dynamics-and-grasshopper-38q85rv6mk.pdf

Evolutionary Population Dynamics and Grasshopper Optimization approaches for feature selection problems

Feature Selection (FS) is a challenging machine learning-related task that aims at reducing the number of features by removing irrelevant, redundant and noisy data while maintaining an acceptable level of classification accuracy. FS can be considered as an optimisation problem. Due to the difficulty of this problem and having a large number of local solutions, stochastic optimisation algorithms are promising techniques to solve this problem. As a seminal attempt, binary variants of the recent Grasshopper Optimisation Algorithm (GOA) are proposed in this work and employed to select the optimal feature subset for classification purposes within a wrapper-based framework. Two mechanisms are employed to design a binary GOA, the first one is based on Sigmoid and V-shaped transfer functions, and will be indicated by BGOA-S and BGOA-V, respectively. While the second mechanism uses a novel technique that combines the best solution obtained so far. In addition, a mutation operator is employed to enhance the exploration phase in BGOA algorithm (BGOA-M). The proposed methods are evaluated using 25 standard UCI datasets and compared with 8 well-regarded metaheuristic wrapper-based approaches, and six well known filter-based (e.g., correlation FS) approaches. The comparative results show the superior performance of the BGOA and BGOA-M methods compared to other similar techniques in the literature.

Binary grasshopper optimisation algorithm approaches for feature selection problems

Differential Evolution: A review of more than two decades of research

We developed a hybrid method for feature selection of classification tasks.Our hybrid method combines Artificial Bee Colony with Differential Evolution.We performed experiments over fifteen datasets from the UCI Repository.Our method selects good features without reducing accuracy of classification.By selecting features with our method, we reduced time required for classification. "Dimensionality" is one of the major problems which affect the quality of learning process in most of the machine learning and data mining tasks. Having high dimensional datasets for training a classification model may lead to have "overfitting" of the learned model to the training data. Overfitting reduces generalization of the model, therefore causes poor classification accuracy for the new test instances. Another disadvantage of dimensionality of dataset is to have high CPU time requirement for learning and testing the model. Applying feature selection to the dataset before the learning process is essential to improve the performance of the classification task. In this study, a new hybrid method which combines artificial bee colony optimization technique with differential evolution algorithm is proposed for feature selection of classification tasks. The developed hybrid method is evaluated by using fifteen datasets from the UCI Repository which are commonly used in classification problems. To make a complete evaluation, the proposed hybrid feature selection method is compared with the artificial bee colony optimization, and differential evolution based feature selection methods, as well as with the three most popular feature selection techniques that are information gain, chi-square, and correlation feature selection. In addition to these, the performance of the proposed method is also compared with the studies in the literature which uses the same datasets. The experimental results of this study show that our developed hybrid method is able to select good features for classification tasks to improve run-time performance and accuracy of the classifier. The proposed hybrid method may also be applied to other search and optimization problems as its performance for feature selection is better than pure artificial bee colony optimization, and differential evolution.

A hybrid approach of differential evolution and artificial bee colony for feature selection

Privacy preserving data mining is a hot research field of data mining. The aim of privacy preserving data mining is to prevent the leakage of the sensitive information of individuals while performing data mining techniques. Classification task is one of the most studied fields in data mining hence in privacy preserving data mining as well. On the other hand, differential privacy is a powerful privacy guarantee that determines privacy leakage ratio by using ∊ parameter and enables researchers to mine data which includes sensitive information. Implementations of some well-known classification algorithms such as k-NN, Naive Bayes, ID3, etc. with differential privacy have been developed. Although the success of the rule-based classifiers using meta-heuristics such as Ant-Miner, BeeMiner etc. in data mining has been demonstrated, any implementation of these classification algorithms with differential privacy has not been proposed in the literature until now to our best knowledge. Artificial bee colony (ABC) is a nature inspired algorithm which imitates foraging behavior of bees, and some approaches using ABC to discover classification rules have been proposed recently and the success of ABC algorithm for the discovery of classification rules has been demonstrated. Motivated by this shortcoming in the literature, we propose to develop a rule-based classifier using ABC algorithm with input perturbation technique of differential privacy to perform privacy preserving classification. According to our experimental results, the proposed ABC-based classifier performs better than the well-known algorithms that are SVM, C4.5, Holte’s One Rule, PART, and RIPPER over non-private and differentially private versions of the datasets in terms of classification performance.

Privacy preserving rule-based classifier using modified artificial bee colony algorithm

Differentially private 1R classification algorithm using artificial bee colony and differential evolution

Privacy preserving data classification is an important research area in data mining field. The goal of a privacy preserving classification algorithm is to protect the sensitive information as much as possible, while providing satisfactory classification accuracy. Differential privacy is a strong privacy guarantee that enables privacy of sensitive data stored in a database by determining the ratio of sensitive information leakage with respect to an ɛ parameter. In this study, our aim is to investigate the classification performance of the state‐of‐the‐art classification algorithms such as C4.5, Naïve Bayes, One Rule, Bayesian Networks, PART, Ripper, K*, IBk, and Random tree for performing privacy preserving classification. To preserve privacy of the data to be classified, we applied input perturbation technique coming from differential privacy, and observed the relationship between the ɛ parameter values and accuracy of the classifiers. To our best knowledge, this article is the first study that analyzes the performances of the well‐known classification algorithms over differentially private data, and discovers which datasets are more suitable for privacy preserving classification when input perturbation is applied to provide data privacy. The classification algorithms are compared by using the differentially private versions of the well‐known datasets from the UCI repository. According to the experimental results, we observed that, as ɛ parameter value increases, better classification accuracies are achieved with lower privacy levels. When the classifiers are compared, Naïve Bayes classifier is the most successful method. The ɛ parameter should be greater than or equal to 2 (i.e., ɛ ≥2) to achieve cloud server is malicious and untrusted, sensitive data will satisfactory classification accuracies.

Privacy preserving classification over differentially private data

Linear discriminant analysis (LDA) is a notable classification algorithm thanks to its major success in many applications of the real-world. In spite of its successfulness for low-dimensional data, a dimension reduction is inevitable for its achievement with high-dimensional data, especially in which the number of features is more than the training sample size or close to the training sample size. Principal component analysis (PCA) plus LDA (PCA+LDA), a quite popular technique, is widely used for raising the classification performance of LDA over high-dimensional data. However, PCA ignores the label information in the data. On the other hand, the reduced dimensional data through PCA still includes indiscriminate (i.e., irrelevant) features. To cope with the dimensionality problem of LDA, a hybrid dimension reduction approach of supervised and unsupervised algorithms is proposed in this study. In the supervised part of the proposed hybrid dimension reduction method, called DBDERF+PCA, we propose to combine an ensemble classifier (i.e., random forest) with dichotomous binary differential evolution (DBDE), a recently proposed variant of binary differential evolution, by introducing a robust wrapper feature selection. On the other hand, unsupervised part of the proposed hybrid dimension reduction method utilizes PCA. The experimental results show that DBDERF+PCA outperforms PCA in terms of dimension reduction. Thereupon, this hybrid dimension reduction based LDA, called DBDERF+PCA+LDA, performs better than PCA+LDA and LDA in terms of run-time and classification performances.

Ezgi Zorarpaci

Papers

A hybrid approach of differential evolution and artificial bee colony for feature selection

Privacy preserving rule-based classifier using modified artificial bee colony algorithm

Differentially private 1R classification algorithm using artificial bee colony and differential evolution

Privacy preserving classification over differentially private data

A Hybrid Dimension Reduction Based Linear Discriminant Analysis for Classification of High-Dimensional Data