Analysis of Various Decision Tree Algorithms for Classification in Data Mining
17 Apr 2017-International Journal of Computer Applications (Foundation of Computer Science (FCS), NY, USA)-Vol. 163, Iss: 8, pp 15-19
TL;DR: Various algorithms of the decision tree (ID3, C4.5, CART), their features, advantages, and disadvantages are discussed.
Abstract: Today the computer technology and computer network technology has developed so much and is still developing with pace.Thus, the amount of data in the information industry is getting higher day by day. This large amount of data can be helpful for analyzing and extracting useful knowledge from it. The hidden patterns of data are analyzed and then categorized into useful knowledge. This process is known as Data Mining. [4].Among the various data mining techniques, Decision Tree is also the popular one. Decision tree uses divide and conquer technique for the basic learning strategy. A decision tree is a flow chart-like structure in which each internal node represents a “test” on an attribute where each branch represents the outcome of the test and each leaf node represents a class label. This paper discusses various algorithms of the decision tree (ID3, C4.5, CART), their features, advantages, and disadvantages.
Citations
More filters
TL;DR: A comprehensive survey on Intrusion Detection System (IDS) for IoT is presented and various IDS placement strategies and IDS analysis strategies in IoT architecture are discussed, along with Machine Learning (ML) and Deep Learning techniques for detecting attacks in IoT networks.
Abstract: Internet of Things (IoT) is widely accepted technology in both industrial as well as academic field. The objective of IoT is to combine the physical environment with the cyber world and create one big intelligent network. This technology has been applied to various application domains such as developing smart home, smart cities, healthcare applications, wireless sensor networks, cloud environment, enterprise network, web applications, and smart grid technologies. These wide emerging applications in variety of domains raise many security issues such as protecting devices and network, attacks in IoT networks, and managing resource-constrained IoT networks. To address the scalability and resource-constrained security issues, many security solutions have been proposed for IoT such as web application firewalls and intrusion detection systems. In this paper, a comprehensive survey on Intrusion Detection System (IDS) for IoT is presented for years 2015–2019. We have discussed various IDS placement strategies and IDS analysis strategies in IoT architecture. The paper discusses various intrusions in IoT, along with Machine Learning (ML) and Deep Learning (DL) techniques for detecting attacks in IoT networks. The paper also discusses security issues and challenges in IoT.
107 citations
TL;DR: A novel optimized deep learning approach based on binary particle swarm optimization with decision tree (BPSO-DT) and convolutional neural network (CNN) to classify different types of cancer based on tumor RNA sequence (RNA-Seq) gene expression data is introduced.
Abstract: Cancer is one of the most feared and aggressive diseases in the world and is responsible for more than 9 million deaths universally. Staging cancer early increases the chances of recovery. One staging technique is RNA sequence analysis. Recent advances in the efficiency and accuracy of artificial intelligence techniques and optimization algorithms have facilitated the analysis of human genomics. This paper introduces a novel optimized deep learning approach based on binary particle swarm optimization with decision tree (BPSO-DT) and convolutional neural network (CNN) to classify different types of cancer based on tumor RNA sequence (RNA-Seq) gene expression data. The cancer types that will be investigated in this research are kidney renal clear cell carcinoma (KIRC), breast invasive carcinoma (BRCA), lung squamous cell carcinoma (LUSC), lung adenocarcinoma (LUAD) and uterine corpus endometrial carcinoma (UCEC). The proposed approach consists of three phases. The first phase is preprocessing, which at first optimize the high-dimensional RNA-seq to select only optimal features using BPSO-DT and then, converts the optimized RNA-Seq to 2D images. The second phase is augmentation, which increases the original dataset of 2086 samples to be 5 times larger. The selection of the augmentations techniques was based achieving the least impact on manipulating the features of the images. This phase helps to overcome the overfitting problem and trains the model to achieve better accuracy. The third phase is deep CNN architecture. In this phase, an architecture of two main convolutional layers for featured extraction and two fully connected layers is introduced to classify the 5 different types of cancer according to the availability of images on the dataset. The results and the performance metrics such as recall, precision and F1 score show that the proposed approach achieved an overall testing accuracy of 96.90%. The comparative results are introduced, and the proposed method outperforms those in related works in terms of testing accuracy for 5 classes of cancer. Moreover, the proposed approach is less complex and consume less memory.
62 citations
Cites background or methods from "Analysis of Various Decision Tree A..."
...The proposed deep learning architecture for tumor gene expression had achieved median testing accuracy with 96.90% in the 80% training and 20% testing strategy with BPSO-DT, which means that the using BPSO-DT algorithm and more data the architecture had, the better accuracy the architecture could achieve....
[...]
...According to the update mechanism in BPSO, each particle updates each position based on the particle’s velocity, which acts as probability threshold as shown in the VOLUME 8, 2020 22875 probabilistic equation (4). sig ( V ki ) = 1 1+ e− k i (3) X ki = { 1, if rand < sig(V ki ) 0, if otherwise (4) where 1 means this feature is chosen as an important feature for the next regeneration and 0 means this feature is not chosen as an important feature to the next regeneration, and rand is a random number ∈ [0, 1]....
[...]
...The steps of BPSO-DT are presented in Algorithm 1, where the input is the tumor gene expression dataset consisting of RNA-seq that needs optimization....
[...]
...Deep learning training is the third phase, which relies on the deep N. Algorithm 1 Extraction the Important Features of RNASeq Input : Tumor gene expression dataset Output: Gbest position 1 Initialize the position and velocity of each particle randomly 2 while iteration condition is not satisfied do 3 Evaluate the fitness of the particle swarm by DT according to equation 5 4 for each particle i do 5 if the fitness of xi is greater than the fitness of the Pbest i then 6 Pbest i = x i 7 end 8 if the fitness of any particle of the swarm is greater than Gbest then 9 Gbest = particle’s position 10 end 11 for each dimension D = 1, . . . ,N do 12 Update particles velocity and particles position according to equation 1,3, and 4 respectively 13 end 14 end 15 go to next generation until termination criterion is met 16 end 17 Output Gbest A. PRE-PROCESSING PHASE (BPSO-DT AND 2D IMAGE CREATION) In this phase, BPSO is applied to implement the feature selection, and the decision tree (DT) [8] is used as BPSO’s fitness function for a classification problem....
[...]
...The first phase is the pre-processing, which included the optimization process using binary particle swarm optimization with design trees (BPSO-DT) algorithm to select the best features of RNA-Seq then converted it to 2D images....
[...]
TL;DR: A review of explainable artificial intelligence (XAI) can be found in this article, where the authors analyze and review various XAI methods, which are grouped into (i) pre-modeling explainability, (ii) interpretable model, and (iii) post-model explainability.
Abstract: Thanks to the exponential growth in computing power and vast amounts of data, artificial intelligence (AI) has witnessed remarkable developments in recent years, enabling it to be ubiquitously adopted in our daily lives. Even though AI-powered systems have brought competitive advantages, the black-box nature makes them lack transparency and prevents them from explaining their decisions. This issue has motivated the introduction of explainable artificial intelligence (XAI), which promotes AI algorithms that can show their internal process and explain how they made decisions. The number of XAI research has increased significantly in recent years, but there lacks a unified and comprehensive review of the latest XAI progress. This review aims to bridge the gap by discovering the critical perspectives of the rapidly growing body of research associated with XAI. After offering the readers a solid XAI background, we analyze and review various XAI methods, which are grouped into (i) pre-modeling explainability, (ii) interpretable model, and (iii) post-modeling explainability. We also pay attention to the current methods that dedicate to interpret and analyze deep learning methods. In addition, we systematically discuss various XAI challenges, such as the trade-off between the performance and the explainability, evaluation methods, security, and policy. Finally, we show the standard approaches that are leveraged to deal with the mentioned challenges.
54 citations
24 Oct 2018
TL;DR: This research focuses on the development of a novel robot learning architecture that uniquely combines learning from demonstration (LfD) and reinforcement learning (RL) algorithms to effectively teach socially assistive robots personalized behaviors.
Abstract: Socially assistive robots can autonomously provide activity assistance to vulnerable populations, including those living with cognitive impairments. To provide effective assistance, these robots should be capable of displaying appropriate behaviors and personalizing them to a user's cognitive abilities. Our research focuses on the development of a novel robot learning architecture that uniquely combines learning from demonstration (LfD) and reinforcement learning (RL) algorithms to effectively teach socially assistive robots personalized behaviors. Caregivers can demonstrate a series of assistive behaviors for an activity to the robot, which it uses to learn general behaviors via LfD. This information is used to obtain initial assistive state-behavior pairings using a decision tree. Then, the robot uses an RL algorithm to obtain a policy for selecting the appropriate behavior personalized to the user's cognition level. Experiments were conducted with the socially assistive robot Casper to investigate the effectiveness of our proposed learning architecture. Results showed that Casper was able to learn personalized behaviors for the new assistive activity of tea-making, and that combining LfD and RL algorithms significantly reduces the time required for a robot to learn a new activity.
48 citations
Cites methods from "Analysis of Various Decision Tree A..."
...A CART decision tree was used as it can provide accurate results even with a small number of demonstrations [33] and can easily handle outliers due to different interpretations or variations across multiple demonstrators without overfitting [34]....
[...]
TL;DR: The Random Forest and J48 algorithms are used to obtain a sustainable and practicable model to detect various stages of CKD with comprehensive medical accuracy and it was revealed that J48 predicted CKD in all stages better than random forest with a 85.5% accuracy.
Abstract: Chronic Kidney Disease (CKD), i.e., gradual decrease in the renal function spanning over a duration of several months to years without any major symptoms, is a life-threatening disease. It progresses in six stages according to the severity level. It is categorized into various stages based on the Glomerular Filtration Rate (GFR), which in turn utilizes several attributes, like age, sex, race and Serum Creatinine. Among multiple available models for estimating GFR value, Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI), which is a linear model, has been found to be quite efficient because it allows detecting all CKD stages. Early detection and cure of CKD is extremely desirable as it can lead to the prevention of unwanted consequences. Machine learning methods are being extensively advocated for early detection of symptoms and diagnosis of several diseases recently. With the same motivation, the aim of this study is to predict the various stages of CKD using machine learning classification algorithms on the dataset obtained from the medical records of affected people. Specifically, we have used the Random Forest and J48 algorithms to obtain a sustainable and practicable model to detect various stages of CKD with comprehensive medical accuracy. Comparative analysis of the results revealed that J48 predicted CKD in all stages better than random forest with an accuracy of 85.5%. The study also showed that J48 shows improved performance over Random Forest. The study concluded that it may be used to build an automated system for the detection of severity of CKD.
36 citations
References
More filters
01 Jan 2006
TL;DR: There have been many data mining books published in recent years, including Predictive Data Mining by Weiss and Indurkhya [WI98], Data Mining Solutions: Methods and Tools for Solving Real-World Problems by Westphal and Blaxton [WB98], Mastering Data Mining: The Art and Science of Customer Relationship Management by Berry and Linofi [BL99].
Abstract: The book Knowledge Discovery in Databases, edited by Piatetsky-Shapiro and Frawley [PSF91], is an early collection of research papers on knowledge discovery from data. The book Advances in Knowledge Discovery and Data Mining, edited by Fayyad, Piatetsky-Shapiro, Smyth, and Uthurusamy [FPSSe96], is a collection of later research results on knowledge discovery and data mining. There have been many data mining books published in recent years, including Predictive Data Mining by Weiss and Indurkhya [WI98], Data Mining Solutions: Methods and Tools for Solving Real-World Problems by Westphal and Blaxton [WB98], Mastering Data Mining: The Art and Science of Customer Relationship Management by Berry and Linofi [BL99], Building Data Mining Applications for CRM by Berson, Smith, and Thearling [BST99], Data Mining: Practical Machine Learning Tools and Techniques by Witten and Frank [WF05], Principles of Data Mining (Adaptive Computation and Machine Learning) by Hand, Mannila, and Smyth [HMS01], The Elements of Statistical Learning by Hastie, Tibshirani, and Friedman [HTF01], Data Mining: Introductory and Advanced Topics by Dunham, and Data Mining: Multimedia, Soft Computing, and Bioinformatics by Mitra and Acharya [MA03]. There are also books containing collections of papers on particular aspects of knowledge discovery, such as Machine Learning and Data Mining: Methods and Applications edited by Michalski, Brakto, and Kubat [MBK98], and Relational Data Mining edited by Dzeroski and Lavrac [De01], as well as many tutorial notes on data mining in major database, data mining and machine learning conferences.
2,591 citations
"Analysis of Various Decision Tree A..." refers background in this paper
...Gain Ratio differs from information gain, which measures the information with respect to a classification that is acquired based on some partitioning [5]....
[...]
..., all the tuples that fall into a given partition would belong into the same class) [5]....
[...]
...Claude Shannon studied the value or “information content” of messages and gave information gain as a measure in his Information Theory [5]....
[...]
...A constraint is added to avoid such condition, whereby the information gain of test selected must be large- at least as great as the average gain over all tests examined [5]....
[...]
01 Jan 2009
TL;DR: Distributed Algorithms contains the most significant algorithms and impossibility results in the area, all in a simple automata-theoretic setting, and their complexity is analyzed according to precisely defined complexity measures.
436 citations
TL;DR: This study presents the classical algorithm that is ID3, then highlights of this study will discuss in more detail C4.5 this one is a natural extension of the ID3 algorithm and a comparison between these two algorithms and others algorithms such as C5.0 and CART.
Abstract: Data mining is the useful tool to discovering the knowledge from large data. Different methods & algorithms are available in data mining. Classification is most common method used for finding the mine rule from the large database. Decision tree method generally used for the Classification, because it is the simple hierarchical structure for the user understanding & decision making. Various data mining algorithms available for classification based on Artificial Neural Network, Nearest Neighbour Rule & Baysen classifiers but decision tree mining is simple one. ID3 and C4.5 algorithms have been introduced by J.R Quinlan which produce reasonable decision trees. The objective of this paper is to present these algorithms. At first we present the classical algorithm that is ID3, then highlights of this study we will discuss in more detail C4.5 this one is a natural extension of the ID3 algorithm. And we will make a comparison between these two algorithms and others algorithms such as C5.0 and CART.
287 citations
04 Jul 2013
TL;DR: The goal of this study is to provide a comprehensive review of different classification techniques in data mining, including decision tree induction, Bayesian networks, k-nearest neighbor classifier, and more.
Abstract: Data mining is a process of inferring knowledge from such huge data. Data Mining has three major components Clustering or Classification, Association Rules and Sequence Analysis. By simple definition, in classification/clustering analyze a set of data and generate a set of grouping rules which can be used to classify future data. Data mining is the process is to extract information from a data set and transform it into an understandable structure. It is the computational process of discovering patterns in large data sets involving methods at the intersection of artificial intelligence, machine learning, statistics, and database systems. The actual data mining task is the automatic or semi-automatic analysis of large quantities of data to extract previously unknown interesting patterns. Data mining involves six common classes of tasks. Anomaly detection, Association rule learning, Clustering, Classification, Regression, Summarization. Classification is a major technique in data mining and widely used in various fields. Classification is a data mining (machine learning) technique used to predict group membership for data instances. In this paper, we present the basic classification techniques. Several major kinds of classification method including decision tree induction, Bayesian networks, k-nearest neighbor classifier, the goal of this study is to provide a comprehensive review of different classification techniques in data mining.
160 citations
TL;DR: Student qualitative data has been taken from educational data mining and the performance analysis of the decision tree algorithm ID3, C4.5 and CART are compared and the experimental results indicate that student's performance is influenced by qualitative factors.
Abstract: Decision Tree is the most widely applied supervised classification technique. The learning and classification steps of decision tree induction are simple and fast and it can be applied to any domain. In this research student qualitative data has been taken from educational data mining and the performance analysis of the decision tree algorithm ID3, C4.5 and CART are compared. The comparison result shows that the Gini Index of CART influence information Gain Ratio of ID3 and C4.5. The classification accuracy of CART is higher when compared to ID3 and C4.5. However the difference in classification accuracy between the decision tree algorithms is not considerably higher. The experimental results of decision tree indicate that student's performance also influenced by qualitative factors.
63 citations
"Analysis of Various Decision Tree A..." refers background or methods in this paper
...These steps are simple and fast and thus Decision Tree can be applied to any domain [10]....
[...]
...To construct a decision tree, ID3 uses a top-down, greedy search through the given sets, where each attribute at every tree node is tested to select the attribute that is best for classification of a given set [10]....
[...]