This paper presents a modification of Quinlan's C4.5 algorithm for imbalanced data classification. While the C4.5 algorithm uses the difference in information entropy to determine the goodness of a split, the proposed method, which is named AUC4.5, examines the difference in the area under the ROC curve (AUC) of a split. It implies that our method attempts to maximize the AUC value of a trained decision tree in order to cope with class imbalance in data. An extensive experimental study was performed on 20 real datasets from the machine learning repository at the University of California at Irvine, Irvine. The proposed AUC4.5 algorithm showed better classification than both the standard and cost-sensitive C4.5 algorithms.

/pdf/auc4-5-auc-based-c4-5-decision-tree-algorithm-for-imbalanced-4kk6olqc87.pdf

AUC4.5: AUC-Based C4.5 Decision Tree Algorithm for Imbalanced Data Classification

This chapter presents an overview of spam email as a serious problem in our internet world and creates a spam filter that reduces the previous weaknesses and provides better identification accuracy with less complexity. Since J48 decision tree is a widely used classification technique due to its simple structure, higher classification accuracy, and lower time complexity, it is used as a spam mail classifier here. Now, with lower complexity, it becomes difficult to get higher accuracy in the case of large number of records. In order to overcome this problem, particle swarm optimization is used here to optimize the spam base dataset, thus optimizing the decision tree model as well as reducing the time complexity. Once the records have been standardized, the decision tree is again used to check the accuracy of the classification. The chapter presents a study on various spam-related issues, various filters used, related work, and potential spam-filtering scope.

Spam Mail Filtering Using Data Mining Approach: A Comparative Performance Analysis

The induction of decision trees is a widely-used approach to build classification models that guarantee high performance and expressiveness. Since a recursive-partitioning strategy guided for some splitting criterion is commonly used to induce these classifiers, overfitting, attribute selection bias, and instability to small training set changes are well-known problems in them. Other approaches, such as incremental induction, classifier ensembles, and the global search in the decision-tree-space, have been implemented to overcome these problems. In particular, metaheuristics such as simulated annealing, genetic algorithms, genetic programming, and ant colony optimization have been used to induce compact and accurate decision trees. This paper presents a state-of-the-art review of the use of single-solution-based metaheuristics and swarm and evolutionary computation algorithms to build decision trees as classification models. We outline the decision-tree-induction process components and detail the existing literature studies on metaheuristic-based approaches to building these classifiers. Several timelines showing the chronological order in which these approaches were introduced in the literature are included. A summary analysis of these studies is also conducted, focusing on their internal components and experimental studies. This work provides a useful reference point for future research in this field.

Induction of decision trees as classification models through metaheuristics

• The three types of metaheuristic and the hyperheuristic-based methods to build decision tree induction algorithms are included, not just evolutionary-algorithm-based approaches for decision tree induction. • Three types of implementation strategies are introduced, according to the place where a metaheuristict is used in the decision tree induction process. • The differences in the solution representation are highlighted, as they impact the metaheuristic-based implementation. • The principal components of each method are detailed, such as fitness measures and variation operators. • An analysis of the experimental studies performed in these methods is conducted. The induction of decision trees is a widely-used approach to build classification models that guarantee high performance and expressiveness. Since a recursive-partitioning strategy guided for some splitting criterion is commonly used to induce these classifiers, overfitting, attribute selection bias, and instability to small training set changes are well-known problems in them. Other approaches, such as incremental induction, classifier ensembles, and the global search in the decision-tree-space, have been implemented to overcome these problems. In particular, metaheuristics such as simulated annealing, genetic algorithms, genetic programming, and ant colony optimization have been used to induce compact and accurate decision trees. This paper presents a state-of-the-art review of the use of single-solution-based metaheuristics and swarm and evolutionary computation algorithms to build decision trees as classification models. We outline the decision-tree-induction process components and detail the existing literature studies on metaheuristic-based approaches to building these classifiers. Several timelines showing the chronological order in which these approaches were introduced in the literature are included. A summary analysis of these studies is also conducted, focusing on their internal components and experimental studies. This work provides a useful reference point for future research in this field. 

Despite the wide range of research on pedestrian safety, previous studies have failed to analyse the real-time data of pedestrian walking misbehaviour on the basis of either pedestrian behaviour distraction or movements during specific activities to realise pedestrian safety for positive (normal) or aggressive pedestrians. Practically, pedestrian walking behaviour should be recognised, and aggressive pedestrians should be differentiated from normal pedestrians. This type of pedestrian behaviour recognition can be converted into a classification problem, which is the main challenge for pedestrian safety systems. In addressing the classification challenge, three issues should be considered: identification of factors, collection of data and exchange of data in the contexts of wireless communication and network failure. Thus, this work proposes a novel approach to pedestrian walking behaviour classification in the aforementioned contexts. Three useful phases are proposed for the methodology of this study. In the first phase involving factor identification, several factors of the irregular walking behaviour of mobile phone users are established by constructing a questionnaire that can determine users’ options (attitudes/opinions) about mobile usage whilst walking on the street. In the second phase involving data collection, four different testing scenarios are developed to acquire the real-time data of pedestrian walking behaviour by using gyroscope sensors. In the third phase involving data exchange, the proposed approach is presented on the basis of two modules. The first module for pedestrian behaviour classification uses random forest and decision tree classifiers part of machine learning techniques via wireless communication when a server becomes available. The developed module is then trained and evaluated using five category sets to obtain the best classification of pedestrian walking behaviour. The second module is based on four standard vectors for classifying pedestrian walking behaviour when a server is unavailable. Fault-tolerant pedestrian walking behaviour is identified and is initiated when failures occur in a network. Two sets of real-time data are presented in this work. The first dataset is related to the questionnaire data from 262 sampled respondents, and the second dataset comprises data on 263 sampled participants with pedestrian walking signals. Experimental results confirm the efficacy of the proposed approach relative to previous ones.

/pdf/an-approach-to-pedestrian-walking-behaviour-classification-2amxcls1f6.pdf

An approach to pedestrian walking behaviour classification in wireless communication and network failure contexts

Decision tree as a classification tool is being used successfully in many areas such as medical diagnosis, customer churn prediction, signal detection and so on. The main advantage of decision tree classifiers is their capability to break down a complex structure into a collection of simpler structures, thus providing a solution that is easy to interpret. Since decision tree is a top-down algorithm using a divide and conquer induction process, there is a risk of reaching a local optimal solution. This paper proposes a procedure of optimally determining thresholds of the chosen variables for a decision tree using an adaptive particle swarm optimization (APSO). The proposed algorithm consists of two phases. First, we construct a decision tree and choose the relevant variables. Second, we find the optimum thresholds simultaneously using an APSO for those selected variables. To validate the proposed algorithm, several artificial and real datasets are used. We compare our results with the original CART results and show that the proposed algorithm is promising for improving prediction accuracy.

/pdf/optimization-of-decision-tree-for-classification-using-a-43mhck50k4.pdf

Optimization of Decision Tree for Classification Using a Particle Swarm

The main advantage of tree classifiers is to provide rules that are simple in form and are easily interpretable Since decision tree is a top-down algorithm using a divide and conquer induction process, there is a risk of reaching a local optimal solution This paper proposes a procedure of optimally determining the splitting variables and their thresholds for a decision tree using an adaptive particle swarm optimization The proposed method consists of three phases – tree construction, threshold optimization and rule simplification To validate the proposed algorithm, several artificial and real datasets are used We compare our results with the original CART results and show that the proposed method is promising for improving prediction accuracy

/pdf/improving-tree-based-classification-rules-using-a-particle-1tlasysrho.pdf

Yun-Ju Cho

Papers

Optimization of Decision Tree for Classification Using a Particle Swarm

Improving Tree-Based Classification Rules Using a Particle Swarm Optimization