FRID: fuzzy-rough interactive dichotomizers
25 Jul 2004-Vol. 3, pp 1337-1342
TL;DR: Simulation results confirm that the use of proposed FRID (fuzzy-rough interactive dichotomizers) strategy leads to smaller decision trees and as a result better generalization performance.
Abstract: We propose FRID (fuzzy-rough interactive dichotomizers); a methodology for the induction of decision trees using rough set based measures to capture cognitive uncertainties inherent to databases. These measures are: 1) fuzzy-roughness and 2) fuzzy-rough entropy. Developed FRID algorithms have been initially applied to various real-world benchmark datasets, and experimentally compared with the three fuzzy decision tree generation algorithms reported so far. Simulation results confirm that the use of proposed strategy leads to smaller decision trees and as a result better generalization performance.
Citations
More filters
Book•
29 Sep 2008
TL;DR: Computational Intelligence and Feature Selection provides a high level audience with both the background and fundamental ideas behind feature selection with an emphasis on those techniques based on rough and fuzzy sets, including their hybridizations.
Abstract: Computational Intelligence and Feature Selection provides a high level audience with both the background and fundamental ideas behind feature selection with an emphasis on those techniques based on rough and fuzzy sets, including their hybridizations It introduces set theory, fuzzy set theory, rough set theory, and fuzzy-rough set theory, and illustrates the power and efficacy of the feature selection described through the use of real-world applications and worked examples Program files implementing major algorithms covered, together with the necessary instructions and datasets, are available on the Web
321 citations
Cites methods from "FRID: fuzzy-rough interactive dicho..."
...Initial research is presented in [32] where a method for fuzzy decision tree construction is given that employs the fuzzy-rough ownership function from [304]....
[...]
TL;DR: This paper proposes an NN algorithm that uses the lower and upper approximations from fuzzy-rough set theory in order to classify test objects, or predict their decision value, and shows that it outperforms other NN approaches and is competitive with leading classification and prediction methods.
113 citations
Cites background from "FRID: fuzzy-rough interactive dicho..."
...Moreover we require I to be decreasing in its first, and increasing in its second component....
[...]
TL;DR: A new fuzzy-rough nearest neighbour (FRNN) classification algorithm, which uses the nearest neighbours to construct lower and upper approximations of decision classes, and classifies test instances based on their membership to these approximation.
Abstract: A new fuzzy-rough nearest neighbour (FRNN) classification algorithm is presented in this paper, as an alternative to Sarkar's fuzzy-rough ownership function (FRNN-O) approach. By contrast to the latter, our method uses the nearest neighbours to construct lower and upper approximations of decision classes, and classifies test instances based on their membership to these approximations. In the experimental analysis, we evaluate our approach with both classical fuzzy-rough approximations (based on an implicator and a t-norm), as well as with the recently introduced vaguely quantified rough sets. Preliminary results are very good, and in general FRNN outperforms FRNN-O, as well as the traditional fuzzy nearest neighbour (FNN) algorithm.
60 citations
Cites methods from "FRID: fuzzy-rough interactive dicho..."
...Initial research is presented in [ 2 ] where a method for fuzzy decision tree construction is given that employs the fuzzy-rough ownership function....
[...]
TL;DR: Experimental results show that the proposed algorithm to generate FRCT outperforms existing fuzzy decision tree generation techniques and rough decomposition tree induction algorithm.
Abstract: Using fuzzy-rough hybrids, we have proposed a measure to quantify the functional dependency of decision attribute(s) on condition attribute(s) within fuzzy data. We have shown that the proposed measure of dependency degree is a generalization of the measure proposed by Pawlak for crisp data. In this paper, this new measure of dependency degree has been encapsulated into the decision tree generation mechanism to produce fuzzy-rough classification trees (FRCT); efficient, top-down, multi-class decision tree structures geared to solving classification problems from feature-based learning examples. The developed FRCT generation algorithm has been applied to 16 real-world benchmark datasets. It is experimentally compared with the five fuzzy decision tree generation algorithms reported so far, and the rough decomposition tree algorithm. Comparison has been made in terms of number of rules, average training time, and classification accuracy. Experimental results show that the proposed algorithm to generate FRCT outperforms existing fuzzy decision tree generation techniques and rough decomposition tree induction algorithm.
47 citations
Cites methods from "FRID: fuzzy-rough interactive dicho..."
...The detailed discussion on the proposed measure is given by Rajen and Gopal [25]....
[...]
...Rough set theory, developed by Pawlak in the early 1980s, is a formal mathematical tool of learning from data R. B. Bhatt (&) M. Gopal Samsung India Electronics Pvt....
[...]
...The other two algorithms for the induction of fuzzy decision trees have been proposed by Rajen and Gopal [26], named as fuzzy-rough interactive dichotomizers ver....
[...]
...The other two algorithms for the induction of fuzzy decision trees have been proposed by Rajen and Gopal [26], named as fuzzy-rough interactive dichotomizers ver....
[...]
...Properties of the proposed measure at extreme values of cxj yð Þ; i.e., at 1 and 0 has been derived by Rajen and Gopal [25]....
[...]
02 Oct 2009
TL;DR: This paper proposes an approach to automated generation of feature pattern-based if-then rules, based on fuzzy-rough sets, and is experimentally evaluated against leading classifiers, including fuzzy and rough rule inducers, and shown to be effective.
Abstract: The automated generation of feature pattern-based if-then rules is essential to the success of many intelligent pattern classifiers, especially when their inference results are expected to be directly human-comprehensible. Fuzzy and rough set theory have been applied with much success to this area as well as to feature selection. Since both applications of rough set theory involve the processing of equivalence classes for their successful operation, it is natural to combine them into a single integrated method that generates concise, meaningful and accurate rules. This paper proposes such an approach, based on fuzzy-rough sets. The algorithm is experimentally evaluated against leading classifiers, including fuzzy and rough rule inducers, and shown to be effective.
42 citations
Cites background from "FRID: fuzzy-rough interactive dicho..."
...A partitioning of the universe of discourse by a reduct will always produce equivalence classes that are subsets of the decision concepts and will cover each concept fully....
[...]
...If no rule is matched, the partly matched rules are considered and the most probable decision is chosen....
[...]
References
More filters
TL;DR: In this paper, an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail, is described, and a reported shortcoming of the basic algorithm is discussed.
Abstract: The technology for building knowledge-based systems by inductive inference from examples has been demonstrated successfully in several practical applications. This paper summarizes an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail. Results from recent studies show ways in which the methodology can be modified to deal with information that is noisy and/or incomplete. A reported shortcoming of the basic algorithm is discussed and two means of overcoming it are compared. The paper concludes with illustrations of current research directions.
17,177 citations
Book•
31 Oct 1991
TL;DR: Theoretical Foundations.
Abstract: I. Theoretical Foundations.- 1. Knowledge.- 1.1. Introduction.- 1.2. Knowledge and Classification.- 1.3. Knowledge Base.- 1.4. Equivalence, Generalization and Specialization of Knowledge.- Summary.- Exercises.- References.- 2. Imprecise Categories, Approximations and Rough Sets.- 2.1. Introduction.- 2.2. Rough Sets.- 2.3. Approximations of Set.- 2.4. Properties of Approximations.- 2.5. Approximations and Membership Relation.- 2.6. Numerical Characterization of Imprecision.- 2.7. Topological Characterization of Imprecision.- 2.8. Approximation of Classifications.- 2.9. Rough Equality of Sets.- 2.10. Rough Inclusion of Sets.- Summary.- Exercises.- References.- 3. Reduction of Knowledge.- 3.1. Introduction.- 3.2. Reduct and Core of Knowledge.- 3.3. Relative Reduct and Relative Core of Knowledge.- 3.4. Reduction of Categories.- 3.5. Relative Reduct and Core of Categories.- Summary.- Exercises.- References.- 4. Dependencies in Knowledge Base.- 4.1. Introduction.- 4.2. Dependency of Knowledge.- 4.3. Partial Dependency of Knowledge.- Summary.- Exercises.- References.- 5. Knowledge Representation.- 5.1. Introduction.- 5.2. Examples.- 5.3. Formal Definition.- 5.4. Significance of Attributes.- 5.5. Discernibility Matrix.- Summary.- Exercises.- References.- 6. Decision Tables.- 6.1. Introduction.- 6.2. Formal Definition and Some Properties.- 6.3. Simplification of Decision Tables.- Summary.- Exercises.- References.- 7. Reasoning about Knowledge.- 7.1. Introduction.- 7.2. Language of Decision Logic.- 7.3. Semantics of Decision Logic Language.- 7.4. Deduction in Decision Logic.- 7.5. Normal Forms.- 7.6. Decision Rules and Decision Algorithms.- 7.7. Truth and Indiscernibility.- 7.8. Dependency of Attributes.- 7.9. Reduction of Consistent Algorithms.- 7.10. Reduction of Inconsistent Algorithms.- 7.11. Reduction of Decision Rules.- 7.12. Minimization of Decision Algorithms.- Summary.- Exercises.- References.- II. Applications.- 8. Decision Making.- 8.1. Introduction.- 8.2. Optician's Decisions Table.- 8.3. Simplification of Decision Table.- 8.4. Decision Algorithm.- 8.5. The Case of Incomplete Information.- Summary.- Exercises.- References.- 9. Data Analysis.- 9.1. Introduction.- 9.2. Decision Table as Protocol of Observations.- 9.3. Derivation of Control Algorithms from Observation.- 9.4. Another Approach.- 9.5. The Case of Inconsistent Data.- Summary.- Exercises.- References.- 10. Dissimilarity Analysis.- 10.1. Introduction.- 10.2. The Middle East Situation.- 10.3. Beauty Contest.- 10.4. Pattern Recognition.- 10.5. Buying a Car.- Summary.- Exercises.- References.- 11. Switching Circuits.- 11.1. Introduction.- 11.2. Minimization of Partially Defined Switching Functions.- 11.3. Multiple-Output Switching Functions.- Summary.- Exercises.- References.- 12. Machine Learning.- 12.1. Introduction.- 12.2. Learning From Examples.- 12.3. The Case of an Imperfect Teacher.- 12.4. Inductive Learning.- Summary.- Exercises.- References.
7,826 citations
"FRID: fuzzy-rough interactive dicho..." refers background in this paper
...To handle classification uncertainties, rough set theory basically focuses on ambiguity caused by limited discernibility of objects in the domain of discourse [ 6 ]....
[...]
Book•
01 Jan 1966
TL;DR: In this article, the authors presented a case of two means regression method for the family error rate, which was used to estimate the probability of a family having a nonzero family error.
Abstract: 1 Introduction.- 1 Case of two means.- 2 Error rates.- 2.1 Probability of a nonzero family error rate.- 2.2 Expected family error rate.- 2.3 Allocation of error.- 3 Basic techniques.- 3.1 Repeated normal statistics.- 3.2 Maximum modulus (Tukey).- 3.3 Bonferroni normal statistics.- 3.4 ?2 projections (Scheffe).- 3.5 Allocation.- 3.6 Multiple modulus tests (Duncan).- 3.7 Least significant difference test (Fisher).- 4 p-mean significance levels.- 5 Families.- 2 Normal Univariate Techniques.- 1 Studentized range (Tukey).- 1.1 Method.- 1.2 Applications.- 1.3 Comparison.- 1.4 Derivation.- 1.5 Distributions and tables.- 2 F projections (Scheffe)48.- 2.1 Method.- 2.2 Applications.- 2.3 Comparison.- 2.4 Derivation.- 2.5 Distributions and tables.- 3 Bonferroni t statistics.- 3.1 Method.- 3.2 Applications.- 3.3 Comparison.- 3.4 Derivation.- 3.5 Distributions and tables.- 4 Studentized maximum modulus.- 4.1 Method.- 4.2 Applications.- 4.3 Comparison.- 4.4 Derivation.- 4.5 Distributions and tables.- 5 Many-one t statistics76.- 5.1 Method.- 5.2 Applications.- 5.3 Comparison.- 5.4 Derivation.- 5.5 Distributions and tables.- 6 Multiple range tests (Duncan).- 6.1 Method.- 6.2 Applications.- 6.3 Comparison.- 6.4 Derivation.- 6.5 Distributions and tables.- 7 Least significant difference test (Fisher).- 7.1 Method.- 7.2 Applications.- 7.3 Comparison.- 7.4 Derivation.- 7.5 Distributions and tables.- 8 Other techniques.- 8.1 Tukey's gap-straggler-variance test.- 8.2 Shortcut methods.- 8.3 Multiple F tests.- 8.4 Two-sample confidence intervals of predetermined length.- 8.5 An improved Bonferroni inequality.- 9 Power.- 10 Robustness.- 3 Regression Techniques.- 1 Regression surface confidence bands.- 1.1 Method.- 1.2 Comparison.- 1.3 Derivation.- 2 Prediction.- 2.1 Method.- 2.2 Comparison.- 2.3 Derivation.- 3 Discrimination.- 3.1 Method.- 3.2 Comparison.- 3.3 Derivation.- 4 Other techniques.- 4.1 Linear confidence bands.- 4.2 Tolerance intervals.- 4.3 Unlimited discrimination intervals.- 4 Nonparametric Techniques.- 1 Many-one sign statistics (Steel).- 1.1 Method.- 1.2 Applications.- 1.3 Comparison.- 1.4 Derivation.- 1.5 Distributions and tables.- 2 k-sample sign statistics.- 2.1 Method.- 2.2 Applications.- 2.3 Comparison.- 2.4 Derivation.- 2.5 Distributions and tables.- 3 Many-one rank statistics (Steel).- 3.1 Method.- 3.2 Applications.- 3.3 Comparison.- 3.4 Derivation.- 3.5 Distributions and tables.- 4 k-sample rank statistics.- 4.1 Method.- 4.2 Applications.- 4.3 Comparison.- 4.4 Derivation.- 4.5 Distributions and tables.- 5 Signed-rank statistics.- 6 Kruskal-Wallis rank statistics (Nemenyi).- 6.1 Method.- 6.2 Applications.- 6.3 Comparison.- 6.4 Derivation.- 6.5 Distributions and tables.- 7 Friedman rank statistics (Nemenyi).- 7.1 Method.- 7.2 Applications.- 7.3 Comparison.- 7.4 Derivation.- 7.5 Distributions and tables.- 8 Other techniques.- 8.1 Permutation tests.- 8.2 Median tests (Nemenyi).- 8.3 Kolmogorov-Smirnov statistics.- 5 Multivariate Techniques.- 1 Single population covariance scalar unknown.- 1.1 Method.- 1.2 Applications.- 1.3 Comparison.- 1.4 Derivation.- 1.5 Distributions and tables.- 2 Single population covariance matrix unknown.- 2.1 Method.- 2.2 Applications.- 2.3 Comparison.- 2.4 Derivation.- 2.5 Distributions and tables.- 3 k populations covariance matrix unknown.- 3.1 Method.- 3.2 Applications.- 3.3 Comparison.- 3.4 Derivation.- 3.5 Distributions and tables.- 4 Other techniques.- 4.1 Variances known covariances unknown.- 4.2 Variance-covariance intervals.- 4.3 Two-sample confidence intervals of predetermined length.- 6 Miscellaneous Techniques.- 1 Outlier detection.- 2 Multinomial populations.- 2.1 Single population.- 2.2 Several populations.- 2.3 Cross-product ratios.- 2.4 Logistic response curves.- 3 Equality of variances.- 4 Periodogram analysis.- 5 Alternative approaches: selection, ranking, slippage.- A Strong Law For The Expected Error Rate.- B TABLES.- I Percentage points of the studentized range.- II Percentage points of the Bonferroni t statistic.- III Percentage points of the studentized maximum modulus.- IV Percentage points of the many-one t statistics.- V Percentage points of the Duncan multiple range test.- VI Percentage points of the many-one sign statistics.- VIII Percentage points of the many-one rank statistics.- IX Percentage points of the k-sample rank statistics.- Developments in Multiple Comparisons 1966-).- 3.5 Allocation.- 3.6 Multiple modulus tests (Duncan).- 3.7 Least significant difference test (Fisher).- 4 p-mean significance levels.- 5 Families.- 2 Normal Univariate Techniques.- 1 Studentized range (Tukey).- 1.1 Method.- 1.2 Applications.- 1.3 Comparison.- 1.4 Derivation.- 1.5 Distributions and tables.- 2 F projections (Scheffe)48.- 2.1 Method.- 2.2 Applications.- 2.3 Comparison.- 2.4 Derivation.- 2.5 Distributions and tables.- 3 Bonferroni t statistics.- 3.1 Method.- 3.2 Applications.- 3.3 Comparison.- 3.4 Derivation.- 3.5 Distributions and tables.- 4 Studentized maximum modulus.- 4.1 Method.- 4.2 Applications.- 4.3 Comparison.- 4.4 Derivation.- 4.5 Distributions and tables.- 5 Many-one t statistics76.- 5.1 Method.- 5.2 Applications.- 5.3 Comparison.- 5.4 Derivation.- 5.5 Distributions and tables.- 6 Multiple range tests (Duncan).- 6.1 Method.- 6.2 Applications.- 6.3 Comparison.- 6.4 Derivation.- 6.5 Distributions and tables.- 7 Least significant difference test (Fisher).- 7.1 Method.- 7.2 Applications.- 7.3 Comparison.- 7.4 Derivation.- 7.5 Distributions and tables.- 8 Other techniques.- 8.1 Tukey's gap-straggler-variance test.- 8.2 Shortcut methods.- 8.3 Multiple F tests.- 8.4 Two-sample confidence intervals of predetermined length.- 8.5 An improved Bonferroni inequality.- 9 Power.- 10 Robustness.- 3 Regression Techniques.- 1 Regression surface confidence bands.- 1.1 Method.- 1.2 Comparison.- 1.3 Derivation.- 2 Prediction.- 2.1 Method.- 2.2 Comparison.- 2.3 Derivation.- 3 Discrimination.- 3.1 Method.- 3.2 Comparison.- 3.3 Derivation.- 4 Other techniques.- 4.1 Linear confidence bands.- 4.2 Tolerance intervals.- 4.3 Unlimited discrimination intervals.- 4 Nonparametric Techniques.- 1 Many-one sign statistics (Steel).- 1.1 Method.- 1.2 Applications.- 1.3 Comparison.- 1.4 Derivation.- 1.5 Distributions and tables.- 2 k-sample sign statistics.- 2.1 Method.- 2.2 Applications.- 2.3 Comparison.- 2.4 Derivation.- 2.5 Distributions and tables.- 3 Many-one rank statistics (Steel).- 3.1 Method.- 3.2 Applications.- 3.3 Comparison.- 3.4 Derivation.- 3.5 Distributions and tables.- 4 k-sample rank statistics.- 4.1 Method.- 4.2 Applications.- 4.3 Comparison.- 4.4 Derivation.- 4.5 Distributions and tables.- 5 Signed-rank statistics.- 6 Kruskal-Wallis rank statistics (Nemenyi).- 6.1 Method.- 6.2 Applications.- 6.3 Comparison.- 6.4 Derivation.- 6.5 Distributions and tables.- 7 Friedman rank statistics (Nemenyi).- 7.1 Method.- 7.2 Applications.- 7.3 Comparison.- 7.4 Derivation.- 7.5 Distributions and tables.- 8 Other techniques.- 8.1 Permutation tests.- 8.2 Median tests (Nemenyi).- 8.3 Kolmogorov-Smirnov statistics.- 5 Multivariate Techniques.- 1 Single population covariance scalar unknown.- 1.1 Method.- 1.2 Applications.- 1.3 Comparison.- 1.4 Derivation.- 1.5 Distributions and tables.- 2 Single population covariance matrix unknown.- 2.1 Method.- 2.2 Applications.- 2.3 Comparison.- 2.4 Derivation.- 2.5 Distributions and tables.- 3 k populations covariance matrix unknown.- 3.1 Method.- 3.2 Applications.- 3.3 Comparison.- 3.4 Derivation.- 3.5 Distributions and tables.- 4 Other techniques.- 4.1 Variances known covariances unknown.- 4.2 Variance-covariance intervals.- 4.3 Two-sample confidence intervals of predetermined length.- 6 Miscellaneous Techniques.- 1 Outlier detection.- 2 Multinomial populations.- 2.1 Single population.- 2.2 Several populations.- 2.3 Cross-product ratios.- 2.4 Logistic response curves.- 3 Equality of variances.- 4 Periodogram analysis.- 5 Alternative approaches: selection, ranking, slippage.- A Strong Law For The Expected Error Rate.- B TABLES.- I Percentage points of the studentized range.- II Percentage points of the Bonferroni t statistic.- III Percentage points of the studentized maximum modulus.- IV Percentage points of the many-one t statistics.- V Percentage points of the Duncan multiple range test.- VI Percentage points of the many-one sign statistics.- VIII Percentage points of the many-one rank statistics.- IX Percentage points of the k-sample rank statistics.- Developments in Multiple Comparisons 1966-1976.- 1 Introduction.- 2 Papers of special interest.- 2.1 Probability inequalities.- 2.2 Methods for unbalanced ANOVA.- 2.3 Conditional confidence levels.- 2.4 Empirical Bayes approach.- 2.5 Confidence bands in regression.- 3 References.- 4 Bibliography 1966-1976.- 4.1 Survey articles.- 4.2 Probability inequalities.- 4.3 Tables.- 4.4 Normal multifactor methods.- 4.5 Regression.- 4.6 Categorical data.- 4.7 Nonparametric techniques.- 4.8 Multivariate methods.- 4.9 Miscellaneous.- 4.10 Pre-1966 articles missed in [6].- 4.11 Late additions.- 5 List of journals scanned.- Addendum New Table of the Studentized Maximum Modulus.- Table IIIA Percentage points of the studentized maximum modulus.- Author Index.
4,763 citations
TL;DR: It is argued that both notions of a rough set and a fuzzy set aim to different purposes, and it is more natural to try to combine the two models of uncertainty (vagueness and coarseness) rather than to have them compete on the same problems.
Abstract: The notion of a rough set introduced by Pawlak has often been compared to that of a fuzzy set, sometimes with a view to prove that one is more general, or, more useful than the other. In this paper we argue that both notions aim to different purposes. Seen this way, it is more natural to try to combine the two models of uncertainty (vagueness and coarseness) rather than to have them compete on the same problems. First, one may think of deriving the upper and lower approximations of a fuzzy set, when a reference scale is coarsened by means of an equivalence relation. We then come close to Caianiello's C-calculus. Shafer's concept of coarsened belief functions also belongs to the same line of thought. Another idea is to turn the equivalence relation into a fuzzy similarity relation, for the modeling of coarseness, as already proposed by Farinas del Cerro and Prade. Instead of using a similarity relation, we can start with fuzzy granules which make a fuzzy partition of the reference scale. The main contribut...
2,452 citations
"FRID: fuzzy-rough interactive dicho..." refers background in this paper
...Dubois and Prade were one of the first to investigate the problem of fuzzification of rough sets[ 7 ]....
[...]
TL;DR: Limitation analysis indicates, and numerical experiments confirm, that the Fukuyama-Sugeno index is sensitive to both high and low values of m and may be unreliable because of this, and calculations suggest that the best choice for m is probably in the interval [1.5, 2.5], whose mean and midpoint, m=2, have often been the preferred choice for many users of FCM.
Abstract: Many functionals have been proposed for validation of partitions of object data produced by the fuzzy c-means (FCM) clustering algorithm We examine the role a subtle but important parameter-the weighting exponent m of the FCM model-plays in determining the validity of FCM partitions The functionals considered are the partition coefficient and entropy indexes of Bezdek, the Xie-Beni (1991), and extended Xie-Beni indexes, and the Fukuyama-Sugeno index (1989) Limit analysis indicates, and numerical experiments confirm, that the Fukuyama-Sugeno index is sensitive to both high and low values of m and may be unreliable because of this Of the indexes tested, the Xie-Beni index provided the best response over a wide range of choices for the number of clusters, (2-10), and for m from 101-7 Finally, our calculations suggest that the best choice for m is probably in the interval [15, 25], whose mean and midpoint, m=2, have often been the preferred choice for many users of FCM >
1,724 citations