A Rough Hypercuboid Approach for Feature Selection in Approximation Spaces
TL;DR: A novel representation of degree of dependency of sample categories on features is proposed to measure the relevance, dependency, and significance of features in approximation spaces by introducing the concept of the hypercuboid equivalence partition matrix.
Abstract: The selection of relevant and significant features is an important problem particularly for data sets with large number of features. In this regard, a new feature selection algorithm is presented based on a rough hypercuboid approach. It selects a set of features from a data set by maximizing the relevance, dependency, and significance of the selected features. By introducing the concept of the hypercuboid equivalence partition matrix, a novel representation of degree of dependency of sample categories on features is proposed to measure the relevance, dependency, and significance of features in approximation spaces. The equivalence partition matrix also offers an efficient way to calculate many more quantitative measures to describe the inexactness of approximate classification. Several quantitative indices are introduced based on the rough hypercuboid approach for evaluating the performance of the proposed method. The superiority of the proposed method over other feature selection methods, in terms of computational complexity and classification accuracy, is established extensively on various real-life data sets of different sizes and dimensions.
Citations
More filters
TL;DR: A parameterized fuzzy relation is introduced to characterize the fuzzy information granules, using which the fuzzy lower and upper approximations of a decision are reconstructed and a new fuzzy rough set model is introduced.
Abstract: A fuzzy rough set is an important rough set model used for feature selection. It uses the fuzzy rough dependency as a criterion for feature selection. However, this model can merely maintain a maximal dependency function. It does not fit a given dataset well and cannot ideally describe the differences in sample classification. Therefore, in this study, we introduce a new model for handling this problem. First, we define the fuzzy decision of a sample using the concept of fuzzy neighborhood. Then, a parameterized fuzzy relation is introduced to characterize the fuzzy information granules, using which the fuzzy lower and upper approximations of a decision are reconstructed and a new fuzzy rough set model is introduced. This can guarantee that the membership degree of a sample to its own category reaches the maximal value. Furthermore, this approach can fit a given dataset and effectively prevents samples from being misclassified. Finally, we define the significance measure of a candidate attribute and design a greedy forward algorithm for feature selection. Twelve datasets selected from public data sources are used to compare the proposed algorithm with certain existing algorithms, and the experimental results show that the proposed reduction algorithm is more effective than classical fuzzy rough sets, especially for those datasets for which different categories exhibit a large degree of overlap.
181 citations
Cites methods from "A Rough Hypercuboid Approach for Fe..."
...The classical rough set model, introduced by Pawlak [35], has been successfully used as a feature selection tool [22], [24], [28], [39], [46]....
[...]
TL;DR: This paper proposes the concept of reduced maximal discernibility pairs, which directly adopts the perspective of the object pair in the framework of the fuzzy rough set model, and develops two attribute selection algorithms, named as reduced maximal perceptibility pairs selection and weighted reducedmaximal discernibility pair selection, based on the reduced maximal distinguishability pairs.
Abstract: Attribute reduction is one of the biggest challenges encountered in computational intelligence, data mining, pattern recognition, and machine learning. Effective in feature selection as the rough set theory is, it can only handle symbolic attributes. In order to overcome this drawback, the fuzzy rough set model is proposed, which is an extended model of rough sets and is able to deal with imprecision and uncertainty in both symbolic and numerical attributes. The existing attribute selection algorithms based on the fuzzy rough set model mainly take the angle of “attribute set,” which means they define the object function representing the predictive ability for an attribute subset with regard to the domain of discourse, rather than following the view of an “object pair.” Algorithms from the viewpoint of the object pair can ignore the object pairs that are already discerned by the selected attribute subsets and, thus, need only to deal with part of object pairs instead of the whole object pairs from the discourse, which makes such algorithms more efficient in attribute selection. In this paper, we propose the concept of reduced maximal discernibility pairs, which directly adopts the perspective of the object pair in the framework of the fuzzy rough set model. Then, we develop two attribute selection algorithms, named as reduced maximal discernibility pairs selection and weighted reduced maximal discernibility pair selection, based on the reduced maximal discernibility pairs. Experiment results show that the proposed algorithms are effective and efficient in attribute selection.
153 citations
01 Jan 2015
TL;DR: A new method for constructing simpler discernibility matrix with covering based rough sets is provided, and some characterizations of attribute reduction provided by Tsang et al. are improved.
Abstract: A simpler approach to attribute reduction based on discernibility matrix is presented with covering based rough sets.Some important properties of attribute reduction with covering based rough sets are improved.The computational complexity of the improved reduction approach is relatively reduced.A new algorithm to attribute reduction in decision tables is presented in a different strategy of identifying objects. Attribute reduction is viewed as an important preprocessing step for pattern recognition and data mining. Most of researches are focused on attribute reduction by using rough sets. Recently, Tsang et al. discussed attribute reduction with covering rough sets in the paper (Tsang et al., 2008), where an approach based on discernibility matrix was presented to compute all attribute reducts. In this paper, we provide a new method for constructing simpler discernibility matrix with covering based rough sets, and improve some characterizations of attribute reduction provided by Tsang et al. It is proved that the improved discernibility matrix is equivalent to the old one, but the computational complexity of discernibility matrix is relatively reduced. Then we further study attribute reduction in decision tables based on a different strategy of identifying objects. Finally, the proposed reduction method is compared with some existing feature selection methods by numerical experiments and the experimental results show that the proposed reduction method is efficient and effective.
80 citations
TL;DR: This paper proposes two quick feature selection algorithms based on the neighbor inconsistent pair, which can reduce the time consumed in finding a reduct and shows that the proposed algorithms are feasible and efficient.
Abstract: Rough set theory, as one of the most useful soft computing methods dealing with vague and uncertain information, has been successfully applied to many fields, and one of its main applications is to perform attribute reduction. Although many heuristic attribute reduction algorithms have been proposed within the framework of the rough set theory, these methods are still computationally time consuming. In order to overcome this deficit, we propose, in this paper, two quick feature selection algorithms based on the neighbor inconsistent pair, which can reduce the time consumed in finding a reduct. At first, we propose several concepts regarding simplified decision table( $U^{^{\prime }}$ ) and neighbor inconsistent pairs. Based on neighbor inconsistent pairs, we constructed two new attribute significance measures. Furthermore, we put forward two new attribute reduction algorithms based on quick neighbor inconsistent pairs. The key characteristic of the presented algorithms is that they only need to calculate $U^{^{\prime }}$ /R once under the process of selecting the best attribute from attribute sets: $C-R$ , while most existing algorithms need to calculate partition of $U^{^{\prime }}$ for $|C-R|$ times. In addition, the proposed algorithms need only to deal with the equivalent classes in $U^{^{\prime }}$ /R that contain at least one neighbor inconsistent pair, while most existing algorithms need to consider all objects in $U^{^{\prime }}$ . The experimental results show that the proposed algorithms are feasible and efficient.
69 citations
TL;DR: The extended variable precision rough set model (VPRS) based on the -tolerance relation in terms of Bhattacharyya distance is presented, and the proposed algorithms outperform the static algorithms and related incremental algorithms while inserting into or removing from attributes in PSvIS.
Abstract: Set-valued information systems are important type of data tables in many real applications, where the attribute values are described by sets to characterize uncertain and incomplete information. However, in some real situations, set-values may be depicted by probability distributions, which results in that the traditional tolerance relation based on intersection operation could not reasonably describe the indiscernibility relation of objects. To address this issue, we introduce the concept of probabilistic set-valued information systems (PSvIS), and present the extended variable precision rough set model (VPRS) based on the -tolerance relation in terms of Bhattacharyya distance. Considering the features of information systems will evolve over time in a dynamic data environment, it will lead to the change of information granulation and approximation structures. A matrix representation of rough approximation is presented based on two matrix operators and two vector functions in PSvIS. Then incremental mechanisms by the utilization of previously learned approximation results and region relation matrices for updating rough approximations are proposed, and the corresponding algorithms are developed and analyzed. Experimental results show that the proposed algorithms outperform the static algorithms and related incremental algorithms while inserting into or removing from attributes in PSvIS.
60 citations
References
More filters
Book•
01 Jan 1995TL;DR: Setting of the learning problem consistency of learning processes bounds on the rate of convergence ofLearning processes controlling the generalization ability of learning process constructing learning algorithms what is important in learning theory?
Abstract: Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing learning algorithms what is important in learning theory?.
40,147 citations
"A Rough Hypercuboid Approach for Fe..." refers background in this paper
...However, the proposed method provides highest classification accuracy of the SVM at 0:0 0:4 for Satimage data, and 0:2 0:4, ¼ 0:6 and 0.7 for Leukemia data....
[...]
...Hence, the proposed indices such as , ?, and can be used to act as the objective function of the feature selection algorithm in approximation spaces as they reflect good quantitative measures like existing SVM and C4.5....
[...]
...The method due to Chen and Wasikowski [35] for binary class data sets achieves 63.16, 81.48, 80.54, 68.75, and 85.29 percent accuracy using the SVM, and 52.63, 92.02, 80.53, 61.25, and 88.24 percent accuracy using the C4.5 on Breast Cancer, Ionosphere, Lung, DLBCLNIH, Prostate Cancer data, respectively....
[...]
...On the other hand, the MRMS criterion attains higher classification accuracy of the SVM and HR value only for Satimage data, while the MDMS criterion achieves higher classification accuracy of both SVM and C4.5 for Satimage data, and lower value of S index for Breast Cancer data....
[...]
...Subsequent discussions analyze the results with respect to various proposed quantitative indices such as , ?, and , and the classification accuracy of both SVM and C4.5....
[...]
TL;DR: In this paper, an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail, is described, and a reported shortcoming of the basic algorithm is discussed.
Abstract: The technology for building knowledge-based systems by inductive inference from examples has been demonstrated successfully in several practical applications. This paper summarizes an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail. Results from recent studies show ways in which the methodology can be modified to deal with information that is noisy and/or incomplete. A reported shortcoming of the basic algorithm is discussed and two means of overcoming it are compared. The paper concludes with illustrations of current research directions.
17,177 citations
14,948 citations
TL;DR: The contributions of this special issue cover a wide range of aspects of variable selection: providing a better definition of the objective function, feature construction, feature ranking, multivariate feature selection, efficient search methods, and feature validity assessment methods.
Abstract: Variable and feature selection have become the focus of much research in areas of application for which datasets with tens or hundreds of thousands of variables are available. These areas include text processing of internet documents, gene expression array analysis, and combinatorial chemistry. The objective of variable selection is three-fold: improving the prediction performance of the predictors, providing faster and more cost-effective predictors, and providing a better understanding of the underlying process that generated the data. The contributions of this special issue cover a wide range of aspects of such problems: providing a better definition of the objective function, feature construction, feature ranking, multivariate feature selection, efficient search methods, and feature validity assessment methods.
14,509 citations
"A Rough Hypercuboid Approach for Fe..." refers background in this paper
...It is to search a set of features that approximates Max-Dependency criterion with the mean value of all dependency values between an individual feature and the target class label [26], [27], [28]....
[...]