Feature selection is an important task in data mining and machine learning to reduce the dimensionality of the data and increase the performance of an algorithm, such as a classification algorithm. However, feature selection is a challenging task due mainly to the large search space. A variety of methods have been applied to solve feature selection problems, where evolutionary computation (EC) techniques have recently gained much attention and shown some success. However, there are no comprehensive guidelines on the strengths and weaknesses of alternative approaches. This leads to a disjointed and fragmented field with ultimately lost opportunities for improving performance and successful applications. This paper presents a comprehensive survey of the state-of-the-art work on EC for feature selection, which identifies the contributions of these different algorithms. In addition, current issues and challenges are also discussed to identify promising areas for future research.

/pdf/a-survey-on-evolutionary-computation-approaches-to-feature-5c2qznm5vr.pdf

A Survey on Evolutionary Computation Approaches to Feature Selection

In this paper, an advanced novel heuristic algorithm is presented, the hybrid genetic algorithm with neural networks (HGA-NN), which is used to identify an optimum feature subset and to increase the classification accuracy and scalability in credit risk assessment. This algorithm is based on the following basic hypothesis: the high-dimensional input feature space can be preliminarily restricted to only the important features. In this preliminary restriction, fast algorithms for feature ranking and earlier experience are used. Additionally, enhancements are made in the creation of the initial population, as well as by introducing an incremental stage in the genetic algorithm. The performances of the proposed HGA-NN classifier are evaluated using a real-world credit dataset that is collected at a Croatian bank, and the findings are further validated on another real-world credit dataset that is selected in a UCI database. The classification accuracy is compared with that presented in the literature. Experimental results that were achieved using the proposed novel HGA-NN classifier are promising for feature selection and classification in retail credit risk assessment and indicate that the HGA-NN classifier is a promising addition to existing data mining techniques.

https://bib.irb.hr/datoteka/640660.Genetic_algorithm-based_heuristic_for_feature_selection_in_credit_risk.pdf

Genetic algorithm-based heuristic for feature selection in credit risk assessment

COVID-19 and stock market volatility: An industry level analysis.

We introduce a literature review considering articles on artificial neural networks in business published in last two decades.412 suitable articles are identified and classified according to defined methodology.We focus on date, area of interest, type of neural network, benchmark method, journal and citations.The most applied are multilayer feedforward networks with backpropagation learning performed by gradient descent algorithm.Majority (25.73%) of the examined articles were published in Expert Systems with Applications. In recent two decades, artificial neural networks have been extensively used in many business applications. Despite the growing number of research papers, only few studies have been presented focusing on the overview of published findings in this important and popular area. Moreover, the majority of these reviews were introduced more than 15 years ago. The aim of this work is to expand the range of earlier surveys and provide a systematic overview of neural network applications in business between 1994 and 2015. We have covered a total of 412 articles and classified them according to the year of publication, application area, type of neural network, learning algorithm, benchmark method, citations and journal. Our investigation revealed that most of the research has aimed at financial distress and bankruptcy problems, stock price forecasting, and decision support, with special attention to classification tasks. Besides conventional multilayer feedforward network with gradient descent backpropagation, various hybrid networks have been developed in order to improve the performance of standard models. Even though neural networks have been established as well-known method in business, there is enormous space for additional research in order to improve their functioning and increase our understanding of this influential area.

Artificial neural networks in business

Information gain directed genetic algorithm wrapper feature selection for credit rating

The databases of the banks around the world have accumulated large quantities of information about clients and their financial and payment history. These databases can be used for the credit risk assessment, but they are commonly high dimensional. Irrelevant features in a training dataset may produce less accurate results of classification analysis. Data preprocessing is required to prepare the data for classification to increase the predictive accuracy. Feature selection is a preprocessing technique commonly used on high dimensional data and its purposes include reducing dimensionality, removing irrelevant and redundant features, facilitating data understanding, reducing the amount of data needed for learning, improving predictive accuracy of algorithms, and increasing interpretability of models. In this paper we investigate the extent to which the total data, owned by a bank, can be a good basis for predicting the borrower's ability to repay the loan on time. We propose a feature selection technique for finding an optimum feature subset that enhances the classification accuracy of neural network classifiers. Experiments were conducted on the credit dataset collected at a Croatian bank to assess the accuracy of our technique. We found that the hybrid system with genetic algorithm is competitive and can be used as feature selection technique to discover the most significant features in determining risk of default.

Hybrid system with genetic algorithm and artificial neural networks and its application to retail credit risk assessment

Display Omitted We connect data characteristics with feature selection techniques performance.Comparative analysis was extensive and included 1280 analysis on 128 data sets.We propose rules for techniques selection based on data characteristics. While extensive research in data mining has been devoted to developing better feature selection techniques, none of this research has examined the intrinsic relationship between dataset characteristics and a feature selection techniques performance. Thus, our research examines experimentally how dataset characteristics affect both the accuracy and the time complexity of feature selection. To evaluate the performance of various feature selection techniques on datasets of different characteristics, extensive experiments with five feature selection techniques, three types of classification algorithms, seven types of dataset characterization methods and all possible combinations of dataset characteristics are conducted on 128 publicly available datasets. We apply the decision tree method to evaluate the interdependencies between dataset characteristics and performance. The results of the study reveal the intrinsic relationship between dataset characteristics and feature selection techniques performance. Additionally, our study contributes to research in data mining by providing a roadmap for future research on feature selection and a significantly wider framework for comparative analysis.

Effects of dataset characteristics on the performance of feature selection techniques

Imbalanced learning data often emerges during the process of the knowledge discovery in data and presents a significant challenge for data mining methods. In this paper we investigate the influence of class imbalanced data on: artificial intelligence methods, i.e. neural networks and support vector machine and on classical classification methods represented by RIPPER and Naive Bayes classifier. The research is conducted on classification problems and, in purpose of measuring the quality of classification, the accuracy and the area under ROC curve measures are used. For the reduction of the negative influence of imbalanced data, SMOTE oversampling technique is used. All experiments on 30 different data sets, obtained from KEEL (Knowledge Extraction based on Evolutionary Learning) repository, are conducted on original datasets, and repeated on balanced datasets generated using SMOTE technique. The results of the research indicate that imbalanced data have significant negative influence on AUC measure on neural network and support vector machine. The same methods are showing improvement of AUC measure when applied on balanced data, but at the same time, are showing the deterioration of results from aspect of the classification accuracy. RIPPER results are also similar, but the changes are of smaller magnitude, while results of Naive Bayes classifier show overall deterioration of results on balanced distributions.

https://www.bib.irb.hr/717193/download/717193.CECIIS_2014_Oreski_Oreski.pdf

An experimental comparison of classification algorithm performances for highly imbalanced datasets

In the present study we propose a new classification technique based on genetic algorithm and neural network, optimized for the cost-sensitive measure and applied to retail credit risk assessment. The relative cost of misclassification, which properly accounts for different misclassification costs of minority and majority classes, is used as the primary evaluation measure. The test of the new algorithm is performed on Croatian and German retail credit datasets for seven different cost ratios. An empirical comparison with others in the literature presented models demonstrates the potential of the new technique in terms of misclassification costs.

Stjepan Oreški

Papers

Genetic algorithm-based heuristic for feature selection in credit risk assessment

Hybrid system with genetic algorithm and artificial neural networks and its application to retail credit risk assessment

Effects of dataset characteristics on the performance of feature selection techniques

An experimental comparison of classification algorithm performances for highly imbalanced datasets

Cost-Sensitive Learning from Imbalanced Datasets for Retail Credit Risk Assessment