Neighborhood rough set based heterogeneous feature subset selection

http://www.yuhuaqian.net/Cms_Data/Contents/SXU_YHQ/Folders/JournalPapers/~contents/KD22X6U6AUY584JT/Positive%20approximation-an%20accelerator%20for%20attribute%20reduction%20in%20rough%20set%20theory.pdf

Positive approximation: An accelerator for attribute reduction in rough set theory

Attribute reduction in decision-theoretic rough set models

K nearest neighbor classifier (K-NN) is widely discussed and applied in pattern recognition and machine learning, however, as a similar lazy classifier using local information for recognizing a new test, neighborhood classifier, few literatures are reported on. In this paper, we introduce neighborhood rough set model as a uniform framework to understand and implement neighborhood classifiers. This algorithm integrates attribute reduction technique with classification learning. We study the influence of the three norms on attribute reduction and classification, and compare neighborhood classifier with KNN, CART and SVM. The experimental results show that neighborhood-based feature selection algorithm is able to delete most of the redundant and irrelevant features. The classification accuracies based on neighborhood classifier is superior to K-NN, CART in original feature spaces and reduced feature subspaces, and a little weaker than SVM.

Neighborhood classifiers

Granular computing, as a new and rapidly growing paradigm of information processing, has attracted many researchers and practitioners. Granular computing is an umbrella term to cover any theories, methodologies, techniques, and tools that make use of information granules in complex problem solving. The aim of this paper is to review foundations and schools of research and to elaborate on current developments in granular computing research. We first review some basic notions of granular computing. Classification and descriptions of various schools of research in granular computing are given. We also present and identify some research directions in granular computing.

Granular Computing: Perspectives and Challenges

Rough set theory has proven to be an efficient tool for modeling and reasoning with uncertainty information. By introducing probability into fuzzy approximation space, a theory about fuzzy probabilistic approximation spaces is proposed in this paper, which combines three types of uncertainty: probability, fuzziness, and roughness into a rough set model. We introduce Shannon's entropy to measure information quantity implied in a Pawlak's approximation space, and then present a novel representation of Shannon's entropy with a relation matrix. Based on the modified formulas, some generalizations of the entropy are proposed to calculate the information in a fuzzy approximation space and a fuzzy probabilistic approximation space, respectively. As a result, uniform representations of approximation spaces and their information measures are formed with this work

Fuzzy probabilistic approximation spaces and their information measures

Feature subset selection presents a common challenge for the applications where data with tens or hundreds of features are available. Existing feature selection algorithms are mainly designed for dealing with numerical or categorical attributes. However, data usually comes with a mixed format in real-world applications. In this paper, we generalize Pawlak's rough set model into @d neighborhood rough set model and k-nearest-neighbor rough set model, where the objects with numerical attributes are granulated with @d neighborhood relations or k-nearest-neighbor relations, while objects with categorical features are granulated with equivalence relations. Then the induced information granules are used to approximate the decision with lower and upper approximations. We compute the lower approximations of decision to measure the significance of attributes. Based on the proposed models, we give the definition of significance of mixed features and construct a greedy attribute reduction algorithm. We compare the proposed algorithm with others in terms of the number of selected features and classification performance. Experiments show the proposed technique is effective.

Mixed feature selection based on granulation and approximation

Short-term wind speed prediction plays an important role in large-scale wind power penetration. However, there is still a large gap between the requirement of prediction performance and current techniques. In this paper, we propose a pattern-based approach to short-term wind speed prediction. It is well accepted that wind varies in different patterns in different weather conditions. Thus, we should use different models to describe these patterns, whereas most current works conduct wind speed prediction with a single model. Based on this observation, we introduce generalized principal component analysis to automatically discover the patterns hidden in the historical data of wind speed. Then we train a predicting function for each pattern and combine their outputs for the final prediction. Experimental results show that the proposed approach performs better than the clustering-based approach, a single model, and persistence forecasting.

Pattern-Based Wind Speed Prediction Based on Generalized Principal Component Analysis

Ordinal classification plays an important role in various decision making tasks. However, little at- tention is paid to this type of learning tasks compared with general classification learning. Shannon information entropy and the derived measure of mutual information play a fundamental role in a number of learning algo- rithms including feature evaluation, selection and decision tree construction. These measures are not applicable to ordinal classification for they cannot characterize the consistency of monotonicity in ordinal classification. In this paper, we generalize Shannon’s entropy to crisp ordinal classification and fuzzy ordinal classification, and show the information measures of ranking mutual information and fuzzy ranking mutual information. We discuss the properties of these measures and show that the proposed ranking mutual information and fuzzy ranking mutual information are the indexes of consistency of monotonicity in ordinal classification. In addition, the proposed indexes are used to evaluate the monotonicity degree between features and decision in the context of ordinal classification.

Jinfu Liu

Papers

Neighborhood rough set based heterogeneous feature subset selection

Fuzzy probabilistic approximation spaces and their information measures

Mixed feature selection based on granulation and approximation

Pattern-Based Wind Speed Prediction Based on Generalized Principal Component Analysis

Information entropy for ordinal classification