This paper presents a survey of evolutionary algorithms that are designed for decision-tree induction. In this context, most of the paper focuses on approaches that evolve decision trees as an alternate heuristics to the traditional top-down divide-and-conquer approach. Additionally, we present some alternative methods that make use of evolutionary algorithms to improve particular components of decision-tree classifiers. The paper's original contributions are the following. First, it provides an up-to-date overview that is fully focused on evolutionary algorithms and decision trees and does not concentrate on any specific evolutionary approach. Second, it provides a taxonomy, which addresses works that evolve decision trees and works that design decision-tree components by the use of evolutionary algorithms. Finally, a number of references are provided that describe applications of evolutionary algorithms for decision-tree induction in different domains. At the end of this paper, we address some important issues and open questions that can be the subject of future research.

/pdf/a-survey-of-evolutionary-algorithms-for-decision-tree-jo3sneclhc.pdf

A Survey of Evolutionary Algorithms for Decision-Tree Induction

Tree-based classifiers are important in pattern recognitio n and have been well studied. Although the problem of finding an optimal decision tree has r eceived attention, it is a hard optimization problem. Here we propose utilizing a genetic algorithm to improve on the finding of compact, near-optimal decision trees. We present a method to encode and decode a decision tree to and from a chromosome where genetic operators such as mutation and crossover can be applied. Theoretical properties of decisi on trees, encoded chromosomes, and fitness functions are presented.

A Genetic Algorithm for Constructing Compact Binary Decision Trees

DEcision-tree induction is one of the most employed methods to extract knowledge from data. There are several distinct strategies for inducing decision trees from data, each one presenting advantages and disadvantages according to its corresponding inductive bias. These strategies have been continuously improved by researchers over the last 40 years. This thesis, following recent breakthroughs in the automatic design of machine learning algorithms, proposes to automatically generate decision-tree induction algorithms. Our proposed approach, namely HEAD-DT, is based on the evolutionary algorithms paradigm, which improves solutions based on metaphors of biological processes. HEAD-DT works over several manually-designed decision-tree components and combines the most suitable components for the task at hand. It can operate according to two different frameworks: i) evolving algorithms tailored to one single data set (specific framework); and ii) evolving algorithms from multiple data sets (general framework). The specific framework aims at generating one decision-tree algorithm per data set, so the resulting algorithm does not need to generalise beyond its target data set. The general framework has a more ambitious goal, which is to generate a single decision-tree algorithm capable of being effectively applied to several data sets. The specific framework is tested over 20 UCI data sets, and results show that HEAD-DT’s specific algorithms outperform algorithms like CART and C4.5 with statistical significance. The general framework, in turn, is executed under two different scenarios: i) designing a domain-specific algorithm; and ii) designing a robust domain-free algorithm. The first scenario is tested over 35 microarray gene expression data sets, and results show that HEAD-DT’s algorithms consistently outperform C4.5 and CART in different experimental configurations. The second scenario is tested over 67 UCI data sets, and HEAD-DT’s algorithms were shown to be competitive with C4.5 and CART. Nevertheless, we show that HEAD-DT is prone to a special case of overfitting when it is executed under the second scenario of the general framework, and we point to possible alternatives for solving this problem. Finally, we perform an extensive experiment for evaluating the best single-objective fitness function for HEAD-DT, combining 5 classification performance measures with three aggregation schemes. We evaluate the 15 fitness functions in 67 UCI data sets, and the best of them are employed to generate algorithms tailored to balanced and imbalanced data. Results show that the automatically-designed algorithms outperform CART and C4.5 with statistical significance, indicating that HEAD-DT is also capable of generating custom algorithms for data with a particular kind of statistical profile.

/pdf/automatic-design-of-decision-tree-induction-algorithms-2fu2jlwfpl.pdf

Automatic Design of Decision-Tree Induction Algorithms

Mapping temperate forest tree species using dense Sentinel-2 time series

Devanagari script is being used in various languages, in south Asian subcontinent, such as Sanskrit, Rajasthani, Marathi and Nepali and it is also the script of Hindi, the mother tongue of majority of Indians. Recognition of handwritten characters of Devanagari alphabet set is an important area of research. The work done for the recognition of Devanagari handwritten script is negligible in literature despite it is being used by millions people in India and abroad and it has numerous applications. The feature extraction method(s) used to recognize hand-printed characters play an important role in ICR applications. There are many feature extraction methods available in literature. We have tested the recognition performance of about 5 feature extraction methods available in literature on Devanagari handwritten characters. A database of more than 25000 handwritten Devanagari characters is developed by collecting the samples from hundreds writers belonging to 43 Devanagari alphabets. The performance comparisons have been made using two classifiers MLP and SVM.

Performance Comparison of Features on Devanagari Hand-printed Dataset

This paper presents a method to recognize the various defect patterns of a cold mill strip using a binary decision tree constructed by genetic algorithm(GA). In this paper, GA was used to select a subset of the suitable features at each node in the binary decision tree. The feature subset with maximum fitness is chosen and the patterns are divided into two classes using a linear decision function. In this way, the classifier using the binary decision tree can be constructed automatically, and the final recognizer is implemented by a neural network trained by standard patterns at each node. Experimental results are given to demonstrate the usefulness of the proposed scheme.

/pdf/binary-decision-tree-using-genetic-algorithm-for-recognizing-4o7olai0z6.pdf

Binary Decision Tree Using Genetic Algorithm for Recognizing Defect Patterns of Cold Mill Strip

Off-line handwritten numeral recognition is a very difficult task. It is hard to achieve high recognition results using a single set of features and a single classifier, since handwritten numerals contain many pattern variations which mostly depend upon individual writing styles. In this paper, we propose a recognition system using hybrid features and a combined classifier. To improve recognition rate, we select mutually beneficial features such as directional features, crossing point features and mesh features, and create three new hybrid feature sets from them. These feature sets hold the local and global characteristics of input numeral images. We also implement a combined classifier from three neural network classifiers to achieve a high recognition rate, using fuzzy integral for multiple network fusion. In order to verify the performance of the proposed recognition system, experiments with the unconstrained handwritten numeral database of Concordia University, Canada were performed, producing a recognition rate of 97.85%.

/pdf/recognition-of-handwritten-numerals-using-a-combined-fmkv8x9y3s.pdf

Recognition of Handwritten Numerals Using a Combined Classifier with Hybrid Features

http://www.cenparmi.concordia.ca/publications/pdf/2004/DetectionofRidgesandRavinesusingFuzzyLogicOperations.pdf

Detection of ridges and ravines using fuzzy logic operations

This paper proposes a method to recognize the various defect patterns of a cold mill strip using a binary decision. In classifying complex patterns with high similarity like these defect patterns, the selection of an optimal feature set and an appropriate recognizer is a pre-requisite to a high recognition rate. In this paper GA and K-means algorithm were used to select a subset of the suitable features at each node in the binary decision tree. The feature subset with maximum fitness is chosen and the patterns are divided into two classes using a linear decision function. This process is repeated at each node until all the patterns are classified into individual classes. In this way, the classifier using the binary decision tree can be constructed automatically, and the final recognizer is implemented by a neural network trained by standard patterns at each node. Experimental results are given to demonstrate the usefulness of the proposed scheme.

/pdf/binary-decision-tree-using-k-means-and-genetic-algorithm-for-5cstdvc19k.pdf

Binary decision tree using K-means and genetic algorithm for recognizing defect patterns of cold mill strip

This paper proposes dynamic thresholding scheme for fingerprint identification. As a user authentication method by fingerprint recognition technology, verification method based on 1:1 matching was mainly used in the past, but identification method based on 1:N matching is generally used recently. The control of the value of FAR is very important in the application areas such as access control and time attendance systems. This paper proposes dynamic thresholding scheme which could properly control the value of FAR according to the field of applications and size of the fingerprints database.

Joong Jo Park

Papers

Binary Decision Tree Using Genetic Algorithm for Recognizing Defect Patterns of Cold Mill Strip

Recognition of Handwritten Numerals Using a Combined Classifier with Hybrid Features

Detection of ridges and ravines using fuzzy logic operations

Binary decision tree using K-means and genetic algorithm for recognizing defect patterns of cold mill strip

Dynamic Thresholding Scheme for Fingerprint Recognition System