Despite significant successes achieved in knowledge discovery, traditional machine learning methods may fail to obtain satisfactory performances when dealing with complex data, such as imbalanced, high-dimensional, noisy data, etc. The reason behind is that it is difficult for these methods to capture multiple characteristics and underlying structure of data. In this context, it becomes an important topic in the data mining field that how to effectively construct an efficient knowledge discovery and mining model. Ensemble learning, as one research hot spot, aims to integrate data fusion, data modeling, and data mining into a unified framework. Specifically, ensemble learning firstly extracts a set of features with a variety of transformations. Based on these learned features, multiple learning algorithms are utilized to produce weak predictive results. Finally, ensemble learning fuses the informative knowledge from the above results obtained to achieve knowledge discovery and better predictive performance via voting schemes in an adaptive way. In this paper, we review the research progress of the mainstream approaches of ensemble learning and classify them based on different characteristics. In addition, we present challenges and possible research directions for each mainstream approach of ensemble learning, and we also give an extra introduction for the combination of ensemble learning with other machine learning hot spots such as deep learning, reinforcement learning, etc.

A survey on ensemble learning

Fuzziness based semi-supervised learning approach for intrusion detection system

http://www.yuhuaqian.net/Cms_Data/Contents/SXU_YHQ/Folders/JournalPapers/~contents/9QEJ2G57ZV5X99VP/Test-cost-sensitive%20attribute%20reduction.pdf

Test-cost-sensitive attribute reduction

We investigate essential relationships between generalization capabilities and fuzziness of fuzzy classifiers (viz., the classifiers whose outputs are vectors of membership grades of a pattern to the individual classes). The study makes a claim and offers sound evidence behind the observation that higher fuzziness of a fuzzy classifier may imply better generalization aspects of the classifier, especially for classification data exhibiting complex boundaries. This observation is not intuitive with a commonly accepted position in “traditional” pattern recognition. The relationship that obeys the conditional maximum entropy principle is experimentally confirmed. Furthermore, the relationship can be explained by the fact that samples located close to classification boundaries are more difficult to be correctly classified than the samples positioned far from the boundaries. This relationship is expected to provide some guidelines as to the improvement of generalization aspects of fuzzy classifiers.

A Study on Relationship Between Generalization Abilities and Fuzziness of Base Classifiers in Ensemble Learning

The concept of a single valued neutrosophic number (SVN-number) is of importance for quantifying an ill-known quantity and the ranking of SVN-numbers is a very difficult problem in multi-attribute decision making problems. The aim of this paper is to present a methodology for solving multi-attribute decision making problems with SVN-numbers. Therefore, we firstly defined the concepts of cut sets of SVN-numbers and then applied to single valued trapezoidal neutrosophic numbers (SVTN-numbers) and triangular neutrosophic numbers (SVTrN-numbers). Then, we proposed the values and ambiguities of the truth-membership function, indeterminacy-membership function and falsity-membership function for a SVN-numbers and studied some desired properties. Also, we developed a ranking method by using the concept of values and ambiguities, and applied to multi-attribute decision making problems in which the ratings of alternatives on attributes are expressed with SVTN-numbers.

A ranking method of single valued neutrosophic numbers and its applications to multi-attribute decision making problems

When fuzzy IF-THEN rules initially extracted from data have not a satisfying performance, we consider that the rules require refinement. Distinct from most existing rule-refinement approaches that are based on the further reduction of training error, this paper proposes a new rule-refinement scheme that is based on the maximization of fuzzy entropy on the training set. The new scheme, which is realized by solving a quadratic programming problem, is expected to have the advantages of improving the generalization capability of initial fuzzy IF-THEN rules and simultaneously overcoming the overfitting of refinement. Experimental results on a number of selected databases demonstrate the expected improvement of generalization capability and the prevention of overfitting by a comparison of both training and testing accuracy before and after the refinement.

Chun-Ru Dong

Papers

A Study on Relationship Between Generalization Abilities and Fuzziness of Base Classifiers in Ensemble Learning

Improving Generalization of Fuzzy IF--THEN Rules by Maximizing Fuzzy Entropy