Showing papers by "Xizhao Wang published in 2005"

The parameterization reduction of soft sets and its applications

Abstract: In this paper, we focus our discussion on the parameterization reduction of soft sets and its applications. First we point out that the results of soft set reductions offered in [1] are incorrect. We also observe that the algorithms used to first compute the reduct-soft-set and then to compute the choice value to select the optimal objects for the decision problems in [1] are not reasonable and we illustrate this with an example. Finally, we propose a reasonable definition of parameterization reduction of soft sets and compare it with the concept of attributes reduction in rough sets theory. By using this new definition of parameterization reduction, we improve the application of a soft set in a decision making problem found in [1].

Localized generalization error and its application to RBFNN training

Abstract: The generalization error bounds for the entire input space found by current error models using the number of effective parameters of a classifier and the number of training samples are usually very loose. But classifiers such as SVM, RBFNN and MLPNN, are really local learning machines used for many application problems, which consider unseen samples close to the training samples more important. In this paper, we propose a localized generalization error model which bounds above the generalization error within a neighborhood of the training samples using stochastic sensitivity measure (expectation of the squared output perturbations). It is then used to develop a model selection technique for a classifier with maximal coverage of unseen samples by specifying a generalization error threshold. Experiments by using eight real world datasets show that, in comparison with cross-validation, sequential learning, and two other ad-hoc methods, our technique consistently yields the best testing classification accuracy with fewer hidden neurons and less training time.

On the reduction of fuzzy rough sets

Abstract: Fuzzy rough set is a generalization of Pawlak rough set to deal with both fuzziness and vagueness. The existing approaches to fuzzy rough set pay more attention to the construction of approximation operators rather than the attributes reduction of the fuzzy rough set. This paper defines a fuzzy information system and investigates its reductions. Some fundamental properties of the system are discussed. An algorithm based on the discernibility matrix to compute all the attributes reductions for fuzzy information systems is developed as well.

An initial comparison on noise resisting between crisp and fuzzy decision trees

Abstract: Decision tree induction is an effective method to solve classification problem in machine learning domain. In general, there are two types of decision tree induction, i.e., crisp decision trees and fuzzy decision trees. Both decision tree inductions based on real-world data are unlikely to find the entirely accurate training set. This means noise existing in the training set. It should be noted that the noise can either cause attributes to become inadequate, or make the decision tree more complicated. It is necessary to further investigate decision trees where the influence of noise data is considered. Experimentally, the paper analyzes the effect of three types of noises, compares the tolerance capability of noise between fuzzy decision trees and crisp decision trees, discusses the modified degree of pruning methods in both fuzzy and crisp decision trees, and addresses the adjustable capability on noise by using different fuzzy reasoning operators in the fuzzy decision tree. Finally the empirical results show fuzzy decision tree is more robust than the crisp decision tree and the post-pruning crisp decision tree.

A parallel genetic algorithm for solving the inverse problem of support vector machines

Abstract: Support Vector Machines (SVMs) are learning machines that can perform binary classification (pattern recognition) and real valued function approximation (regression estimation) tasks. An inverse problem of SVMs is how to split a given dataset into two clusters such that the maximum margin between the two clusters is attained. Here the margin is defined according to the separating hyper-plane generated by support vectors. This paper investigates the inverse problem of SVMs by designing a parallel genetic algorithm. Experiments show that this algorithm can greatly decrease time complexity by the use of parallel processing. This study on the inverse problem of SVMs is motivated by designing a heuristic algorithm for generating decision trees with high generalization capability.

The infinite polynomial kernel for support vector machine

Abstract: This paper develops an infinite polynomial kernel k c for support vector machines. We also propose a mapping from an original data space into the high dimensional feature space on which the inner product is defined by the infinite polynomial kernel k c . Via this mapping, any two finite sets of data in the original space will become linearly separable in the feature space. Numerical experiments indicate that the proposed infinite polynomial kernel possesses some properties and performance better than the existing finite polynomial kernels.

An upper bound of input perturbation for RBFNNS sensitivity analysis

Abstract: According to the existing definition of sensitivity of radial basis function neural networks (RBFNNs), the sensitivity of each feature of RBFNNs can be numerically calculated. Usually features with small value of sensitivity are regarded as redundant ones that may be removed. By sorting the calculated sensitivity magnitudes, a sequence of features can be obtained. This sequence does depend generally on the input perturbation, which seriously affects the application of RBFNNs sensitivity to redundant feature removal. To overcome this defect, this paper gives an upper bound of input perturbation under which the sequence will be independent of the input perturbation. This upper bound keeps the feature sensitivity order unchanged and leads to a new sensitivity definition of RBFNNs. Simulation has been performed to verify the upper bound and the new sensitivity definition and the simulation result is consistent with the theoretical result.

Multilevel weighted fuzzy reasoning with interaction

Abstract: This paper presents an interactive weighted fuzzy reasoning algorithm for rule based systems based on interactive weighted fuzzy Petri nets (IWFPN). The fuzzy production rules in the knowledge base of a rule based system are modelled by IWFPN, where the weights appearing in the rules are represented by fuzzy numbers. In order to model and handle the interaction existing among attributes in a more flexible way, this paper also proposes a fuzzy number valued fuzzy integral (called extended Choquet integral) to draw the reasoning conclusion, where the fuzzy measure refers to an extended g-lambda measure, or a non-additive nonnegative fuzzy number valued set function specified by a domain expert. The proposed algorithm can allow the rule based systems to perform fuzzy reasoning in a more intelligent manner.

A second-order statistical detection approach with application to Internet anomaly detection

Abstract: Detecting multiple network attacks is essential to intrusion detection, network prevention, security defense and network traffic management. But in today's distributed computer networks, the various and frequent attacks make an effective detection difficult. This paper presents a covariance matrix based second-order statistical method to detect multiple known and unknown network anomalies. The detection method is initially based on the observations of the correlativity changes in typical flooding DoS attacks. It utilizes the difference of covariance matrices among observed samples in the detection. As case studies, extensive experiments are conducted to detect multiple DoS attacks - the prevalent Internet anomalies. The experimental results indicate that the proposed approach achieves high detection rates in detecting multiple known and unknown anomalies.

An approach to case-based maintenance : selecting representative cases

Abstract: Case-based maintenance is an important issue in Case-Based Reasoning (CBR) System. Generally speaking, the larger the case-base, the more accurate the solution. However, if the case base is too large, it may include many redundant cases and the case retrieve will not be effective. Moreover redundant cases will affect the solution accuracy. Therefore, removing redundant cases is a fundamental issue in maintaining CBR systems. In this paper, a new approach based on the Generalization Capability of cases to select the representative cases for Case-Based Maintenance is proposed. Using this method, most redundant cases can be deleted and the most representative cases can be identified and retained. The experiments show that the proposed method can greatly remove the redundant cases as well as preserve a satisfying degree of accuracy of solutions when it is used for classification tasks.

A covariance matrix based approach to internet anomaly detection

Abstract: Detecting multiple network attacks is essential to intrusion detection, network security defense and network traffic management. This paper presents a covariance matrix based detection approach to detecting multiple known and unknown network anomalies. It utilizes the difference of covariance matrices among observed samples in the detection. A threshold matrix is employed in the detection where each entry of the matrix evaluates the covariance changes of the corresponding features. As case studies, extensive experiments are conducted to detect multiple DoS attacks – the prevalent Internet anomalies. The experimental results indicate that the proposed approach achieves high detection rates in detecting multiple known and unknown anomalies.

A feature space analysis for anomaly detection

Abstract: Intrusion detection is an important part of assuring the reliability of computer systems. From the viewpoint of feature space partition of detectors, this paper investigates one of the limitations of two traditional anomaly detection technologies - NN-based anomaly detection and statistical detection approaches in detecting novel attacks. A high dimensional covariance matrix feature space and an on-line detection algorithm are proposed to detect various known and unknown attacks. An illustrative example of detecting various known and unknown probing attacks is provided.

Structured large margin learning

Abstract: This paper presents a new large margin learning approach, namely structured large margin machine (SLMM), which incorporates both merits of "structured" learning models and advantages of large margin learning schemes. The promising features of this model, such as enhanced generalization ability, scalability, extensibility, and noise tolerance, are demonstrated theoretically and empirically. SLMM is of theoretical importance because it is a generalization of learning models like SVM, MPM, LDA, and M4 etc. Moreover, it provides a novel insight into the study of learning methods and forms a foundation for conceiving other "structured" classifiers.

Using special structured fuzzy measure to represent interaction among IF-THEN rules

Abstract: When fuzzy IF-THEN rules are used to approximate reasoning, interaction exists among rules. Handling the interaction based on a non-integral can lead to an improvement of reasoning accuracy but the determination of non-linear integral usually needs to solve a linear programming problem with too many parameters when the rules are a little many. That is, the number of parameters increases exponentially with the number of rules. This paper proposes a new approach to denoting the interaction by a 2-additive fuzzy measure which replaces the general set function of the old non-linear integral approach. The number of parameters determined in the new approach is greatly less than the number of parameters in the old approach. Compared with the old approach, the new one has a little loss of accuracy but the new approach reduces the number of parameters from an exponential to polynomial quantity.

The infinite polynomial kernel for support vector machine

Abstract: This paper develops an infinite polynomial kernel kc for support vector machines. We also propose a mapping from an original data space into the high dimensional feature space on which the inner product is defined by the infinite polynomial kernel kc . Via this mapping, any two finite sets of data in the original space will become linearly separable in the feature space. Numerical experiments indicate that the proposed infinite polynomial kernel possesses some properties and performance better than the existing finite polynomial kernels.

A method to construct the mapping to the feature space for the dot product kernels

Abstract: Dot product kernels are a class of important kernel in the theory of support vector machine. This paper develops a method to construct the mapping that map the original data set into the high dimensional feature space, on which the inner product is defined by a dot product kernel. Our method can also be applied to the Gaussian kernels. Via this mapping, the structure of features in the feature space is easy to be observed, and the linear separability of data sets in the feature space is studied. We obtain that any two finite sets of data with empty overlap in the original space will become linearly separable in an infinite dimensional feature space, and a sufficient and necessary condition is also developed for two infinite sets of data in the original data space being linearly separable in the feature space, this condition can be applied to examine the existences and uniqueness of the hyperplane which can separate all the possible inputs correctly.

Finding reducts for ordinal decision tables

Abstract: Since its introduction, rough set theory has demonstrated its usefulness in many applications where imprecise and inconsistent information is involved. An important area of its application is in the induction of decision rules for decision problems. Recently, there are studies for applying rough set theory in decision related to ordering where items are ordered by assigning to them an ordinal class label such as excellent, good, fair, bad. In this paper we examine a particular situation of ordinal decision which has not been considered in previous studies. We introduce some new concepts in relation to reducts of such ordinal decision systems and proposed a way to find these reducts using a concept similar to discernibility matrix.

Reduction of attributes in ordinal decision systems

Abstract: Rough set theory has proven to be a very useful tool in dealing with many decision situations where imprecise and inconsistent information are involved. Recently, there are attempts to extent the use of rough set theory to ordinal decision making in which decisions are made on ordering of objects through assigning them to ordinal categories. Attribute reduction is one of the problems that is studied under such ordinal decision situations. In this paper we examine some of the proposed approaches to find ordinal reducts and present a new perspective and approach to the problem based on ordinal consistency.

A nonlinear integral defined on partition of set and its fundamental properties

Abstract: Nonlinear integrals play an important role in the information fusion. So far, many nonlinear integrals such as Sugeno integral, Choquet integral, pan-integral and Wang-integral have already been defined well and have been applied successfully to solve the problems of information fusion. All these existing nonlinear integrals of a function with respect to a set function are defined on a subset of a space. In many problems of information fusion such as decision tree generation in inductive learning, we often deal with the function defined on a partition of the space. Motivated by minimizing the classification information entropy of a partition while generating decision trees, this paper proposes a nonlinear integral of a function with respect to a non-negative set function on a partition. The basic properties of the proposed integral are discussed and the potential applications of the proposed integral to decision tree generation are outlined in this paper.

The representation of interaction among fuzzy rules

Abstract: In this paper, we apply the interaction model established by M. Grabisch (Fuzzy Sets and System, Vol. 92, pp. 167-189, 1997) to the reasoning of fuzzy production rules. When fuzzy production rules are used in approximate reasoning, interaction exists among rules that have the same consequent but different antecedent. In order to deal with the interaction among fuzzy rules properly, we employ a kind of representation coming from cooperative game theory, named interaction index (e.g. Shapley value) and discuss how to determine the interaction index among fuzzy rules via the fuzzy measure which is defined on the set of fuzzy rules. This paper also roughly discusses how to examine the interaction about fuzzy rules coming from experience.

Using 2-additive fuzzy measure to represent the interaction among if-then rules

Abstract: When fuzzy if-then rules are used to approximate reasoning, interaction exists among rules that have the same consequent Due to this interaction, the weighted average model frequently used in approximate reasoning may not work well in many real-world problems In order to handle this interaction, the paper "IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics, Volume: 34, No5, October 2004, pp 1 -9" proposed to use a non-additive nonnegative set function to replace the weights assigned to rules having the same consequent and to draw the reasoning conclusion based on an integral with respect to the non-additive nonnegative set function Handling interaction in fuzzy if-then rule reasoning in this way can lead to an improvement of reasoning accuracy In that paper, the authors proposed an approach to determining the set function when it was not given by the experts They need to solve a linear programming problem with too many parameters when the number of the rules is large Actually, it is not feasible to implement in the real world because the number of parameters increases exponentially with the number of rules This paper proposes a new approach to using the 2-additive fuzzy measure to replace the general set function for handling the interaction among if-then rules The number of parameters determined in the new approach is greatly less than the number of parameters in the old approach Compared with the old approach, the new one leads to an accuracy loss to some extent But the new approach reduces the number of parameters from an exponential to polynomial quantity It implies that the new approach is feasible and has more wide applications in the real world