Harris hawks optimization: Algorithm and applications

Feature selection is an important task in data mining and machine learning to reduce the dimensionality of the data and increase the performance of an algorithm, such as a classification algorithm. However, feature selection is a challenging task due mainly to the large search space. A variety of methods have been applied to solve feature selection problems, where evolutionary computation (EC) techniques have recently gained much attention and shown some success. However, there are no comprehensive guidelines on the strengths and weaknesses of alternative approaches. This leads to a disjointed and fragmented field with ultimately lost opportunities for improving performance and successful applications. This paper presents a comprehensive survey of the state-of-the-art work on EC for feature selection, which identifies the contributions of these different algorithms. In addition, current issues and challenges are also discussed to identify promising areas for future research.

/pdf/a-survey-on-evolutionary-computation-approaches-to-feature-5c2qznm5vr.pdf

A Survey on Evolutionary Computation Approaches to Feature Selection

This article reviews the state-of-the-art in overlapping community detection algorithms, quality measures, and benchmarks. A thorough comparison of different algorithms (a total of fourteen) is provided. In addition to community-level evaluation, we propose a framework for evaluating algorithms' ability to detect overlapping nodes, which helps to assess overdetection and underdetection. After considering community-level detection performance measured by normalized mutual information, the Omega index, and node-level detection performance measured by F-score, we reached the following conclusions. For low overlapping density networks, SLPA, OSLOM, Game, and COPRA offer better performance than the other tested algorithms. For networks with high overlapping density and high overlapping diversity, both SLPA and Game provide relatively stable performance. However, test results also suggest that the detection in such networks is still not yet fully resolved. A common feature observed by various algorithms in real-world networks is the relatively small fraction of overlapping nodes (typically less than 30p), each of which belongs to only 2 or 3 communities.

Overlapping community detection in networks: The state-of-the-art and comparative study

${k}$ nearest neighbor (kNN) method is a popular classification method in data mining and statistics because of its simple implementation and significant classification performance. However, it is impractical for traditional kNN methods to assign a fixed ${k}$ value (even though set by experts) to all test samples. Previous solutions assign different $k$ values to different test samples by the cross validation method but are usually time-consuming. This paper proposes a kTree method to learn different optimal $k$ values for different test/new samples, by involving a training stage in the kNN classification. Specifically, in the training stage, kTree method first learns optimal $k$ values for all training samples by a new sparse reconstruction model, and then constructs a decision tree (namely, kTree) using training samples and the learned optimal $k$ values. In the test stage, the kTree fast outputs the optimal $k$ value for each test sample, and then, the kNN classification can be conducted using the learned optimal $k$ value and all training samples. As a result, the proposed kTree method has a similar running cost but higher classification accuracy, compared with traditional kNN methods, which assign a fixed ${k}$ value to all test samples. Moreover, the proposed kTree method needs less running cost but achieves similar classification accuracy, compared with the newly kNN methods, which assign different ${k}$ values to different test samples. This paper further proposes an improvement version of kTree method (namely, k*Tree method) to speed its test stage by extra storing the information of the training samples in the leaf nodes of kTree, such as the training samples located in the leaf nodes, their kNNs, and the nearest neighbor of these kNNs. We call the resulting decision tree as k*Tree, which enables to conduct kNN classification using a subset of the training samples in the leaf nodes rather than all training samples used in the newly kNN methods. This actually reduces running cost of test stage. Finally, the experimental results on 20 real data sets showed that our proposed methods (i.e., kTree and k*Tree) are much more efficient than the compared methods in terms of classification tasks.

Efficient kNN Classification With Different Numbers of Nearest Neighbors

A recent set of overused population-based methods have been published in recent years. Despite their popularity, most of them have uncertain, immature performance, partially done verifications, similar overused metaphors, similar immature exploration and exploitation components and operations, and an insecure tradeoff between exploration and exploitation trends in most of the new real-world cases. Therefore, all users need to extensively modify and adjust their operations based on main evolutionary methods to reach faster convergence, more stable balance, and high-quality results. To move the optimization community one step ahead toward more focus on performance rather than change of metaphor, a general-purpose population-based optimization technique called Hunger Games Search (HGS) is proposed in this research with a simple structure, special stability features and very competitive performance to realize the solutions of both constrained and unconstrained problems more effectively. The proposed HGS is designed according to the hunger-driven activities and behavioural choice of animals. This dynamic, fitness-wise search method follows a simple concept of “Hunger” as the most crucial homeostatic motivation and reason for behaviours, decisions, and actions in the life of all animals to make the process of optimization more understandable and consistent for new users and decision-makers. The Hunger Games Search incorporates the concept of hunger into the feature process; in other words, an adaptive weight based on the concept of hunger is designed and employed to simulate the effect of hunger on each search step. It follows the computationally logical rules (games) utilized by almost all animals and these rival activities and games are often adaptive evolutionary by securing higher chances of survival and food acquisition. This method's main feature is its dynamic nature, simple structure, and high performance in terms of convergence and acceptable quality of solutions, proving to be more efficient than the current optimization methods. The effectiveness of HGS was verified by comparing HGS with a comprehensive set of popular and advanced algorithms on 23 well-known optimization functions and the IEEE CEC 2014 benchmark test suite. Also, the HGS was applied to several engineering problems to demonstrate its applicability. The results validate the effectiveness of the proposed optimizer compared to popular essential optimizers, several advanced variants of the existing methods, and several CEC winners and powerful differential evolution (DE)-based methods abbreviated as LSHADE, SPS_L_SHADE_EIG, LSHADE_cnEpSi, SHADE, SADE, MPEDE, and JDE methods in handling many single-objective problems. We designed this open-source population-based method to be a standard tool for optimization in different areas of artificial intelligence and machine learning with several new exploratory and exploitative features, high performance, and high optimization capacity. The method is very flexible and scalable to be extended to fit more form of optimization cases in both structural aspects and application sides. This paper's source codes, supplementary files, Latex and office source files, sources of plots, a brief version and pseudocode, and an open-source software toolkit for solving optimization problems with Hunger Games Search and online web service for any question, feedback, suggestion, and idea on HGS algorithm will be available to the public at https://aliasgharheidari.com/HGS.html .

Hunger games search: Visions, conception, implementation, deep analysis, perspectives, and towards performance shifts

In this paper, a new support vector machines (SVM) parameter tuning scheme that uses the fruit fly optimization algorithm (FOA) is proposed. Termed as FOA-SVM, the scheme is successfully applied to medical diagnosis. In the proposed FOA-SVM, the FOA technique effectively and efficiently addresses the parameter set in SVM. Additionally, the effectiveness and efficiency of FOA-SVM is rigorously evaluated against four well-known medical datasets, including the Wisconsin breast cancer dataset, the Pima Indians diabetes dataset, the Parkinson dataset, and the thyroid disease dataset, in terms of classification accuracy, sensitivity, specificity, AUC (the area under the receiver operating characteristic (ROC) curve) criterion, and processing time. Four competitive counterparts are employed for comparison purposes, including the particle swarm optimization algorithm-based SVM (PSO-SVM), genetic algorithm-based SVM (GA-SVM), bacterial forging optimization-based SVM (BFO-SVM), and grid search technique-based SVM (Grid-SVM). The empirical results demonstrate that the proposed FOA-SVM method can obtain much more appropriate model parameters as well as significantly reduce the computational time, which generates a high classification accuracy. Promisingly, the proposed method can be regarded as a useful clinical tool for medical decision making.

Evolving support vector machines using fruit fly optimization for medical data classification

Breast cancer is becoming a leading cause of death among women in the whole world, meanwhile, it is confirmed that the early detection and accurate diagnosis of this disease can ensure a long survival of the patients. Expert systems and machine learning techniques are gaining popularity in this field because of the effective classification and high diagnostic capability. In this paper, a rough set (RS) based supporting vector machine classifier (RS_SVM) is proposed for breast cancer diagnosis. In the proposed method (RS_SVM), RS reduction algorithm is employed as a feature selection tool to remove the redundant features and further improve the diagnostic accuracy by SVM. The effectiveness of the RS_SVM is examined on Wisconsin Breast Cancer Dataset (WBCD) using classification accuracy, sensitivity, specificity, confusion matrix and receiver operating characteristic (ROC) curves. Experimental results demonstrate the proposed RS_SVM can not only achieve very high classification accuracy but also detect a combination of five informative features, which can give an important clue to the physicians for breast diagnosis.

A support vector machine classifier with rough set-based feature selection for breast cancer diagnosis

A feature selection method for online detection of cotton foreign fiber is presented.The proposed method efficiently finds the excellent subset in multi-character feature sets.The selected sets by proposed method efficiently reduce the time of online detection.The proposed method improves the performance of industrial equipment. Feature selection plays an important role in the machine-vision-based online detection of foreign fibers in cotton because of improvement detection accuracy and speed. Feature sets of foreign fibers in cotton belong to multi-character feature sets. That means the high-quality feature sets of foreign fibers in cotton consist of three classes of features which are respectively the color, texture and shape features. The multi-character feature sets naturally contain a space constraint which lead to the smaller feature space than the general feature set with the same number of features, however the existing algorithms do not consider the space characteristic of multi-character feature sets and treat the multi-character feature sets as the general feature sets. This paper proposed an improved ant colony optimization for feature selection, whose objective is to find the (near) optimal subsets in multi-character feature sets. In the proposed algorithm, group constraint is adopted to limit subset constructing process and probability transition for reducing the effect of invalid subsets and improve the convergence efficiency. As a result, the algorithm can effectively find the high-quality subsets in the feature space of multi-character feature sets. The proposed algorithm is tested in the datasets of foreign fibers in cotton and comparisons with other methods are also made. The experimental results show that the proposed algorithm can find the high-quality subsets with smaller size and high classification accuracy. This is very important to improve performance of online detection systems of foreign fibers in cotton.

Feature selection based on improved ant colony optimization for online detection of foreign fiber in cotton

Bankruptcy prediction is one of the most important issues in financial decision-making. Constructing effective corporate bankruptcy prediction models in time is essential to make companies or banks prevent bankruptcy. This study proposes a novel bankruptcy prediction model based on an adaptive fuzzy k-nearest neighbor (FKNN) method, where the neighborhood size k and the fuzzy strength parameter m are adaptively specified by the continuous particle swarm optimization (PSO) approach. In addition to performing the parameter optimization for FKNN, PSO is also utilized to choose the most discriminative subset of features for prediction. Adaptive control parameters including time-varying acceleration coefficients (TVAC) and time-varying inertia weight (TVIW) are employed to efficiently control the local and global search ability of PSO algorithm. Moreover, both the continuous and binary PSO are implemented in parallel on a multi-core platform. The proposed bankruptcy prediction model, named PTVPSO-FKNN, is compared with five other state-of-the-art classifiers on two real-life cases. The obtained results clearly confirm the superiority of the proposed model in terms of classification accuracy, Type I error, Type II error and area under the receiver operating characteristic curve (AUC) criterion. The proposed model also demonstrates its ability to identify the most discriminative financial ratios. Additionally, the proposed model has reduced a large amount of computational time owing to its parallel implementation. Promisingly, PTVPSO-FKNN might serve as a new candidate of powerful early warning systems for bankruptcy prediction with excellent performance.

A novel bankruptcy prediction model based on an adaptive fuzzy k -nearest neighbor method

In this paper, a novel hybrid method named the LFDA_SVM, which integrates a new feature extraction method and a classification algorithm, has been introduced for diagnosing hepatitis disease. The two integrated methods are the local fisher discriminant analysis (LFDA) and the supporting vector machine (SVM), respectively. In the proposed LFDA_SVM, the LFDA is employed as a feature extraction tool for dimensionality reduction in order to further improve the diagnostic accuracy of the standard SVM algorithm. The effectiveness of the LFDA_SVM has been rigorously evaluated against the hepatitis dataset, a benchmark dataset, from UCI Machine Learning Database in terms of classification accuracy, sensitivity and specificity respectively. In addition, the proposed LFDA_SVM has been compared with three existing methods including the SVM based on principle component analysis (PCA_SVM), the SVM based on fisher discriminant analysis (FDA_SVM) and the standard SVM in terms of their classification accuracy. Experimental results have demonstrated that the LFDA_SVM greatly outperforms other three methods. The best classification accuracy (96.77%) obtained by the LFDA_SVM is much higher than that of the compared ones. Promisingly, the proposed LFDA_SVM might serve as a new candidate of powerful methods for diagnosing hepatitis with excellent performance.

Bo Yang

Papers

Evolving support vector machines using fruit fly optimization for medical data classification

A support vector machine classifier with rough set-based feature selection for breast cancer diagnosis

Feature selection based on improved ant colony optimization for online detection of foreign fiber in cotton

A novel bankruptcy prediction model based on an adaptive fuzzy k -nearest neighbor method

A new hybrid method based on local fisher discriminant analysis and support vector machines for hepatitis disease diagnosis