Support vector machine (SVM) is considered to be one of the most powerful learning algorithms and is used for a wide range of real-world applications. The efficiency of SVM algorithm and its performance mainly depends on the kernel type and its parameters. Furthermore, the feature subset selection that is used to train the SVM model is another important factor that has a major influence on it classification accuracy. The feature subset selection is a very important step in machine learning, specially when dealing with high-dimensional data sets. Most of the previous researches handled these important factors separately. In this paper, we propose a hybrid approach based on the Grasshopper optimisation algorithm (GOA), which is a recent algorithm inspired by the biological behavior shown in swarms of grasshoppers. The goal of the proposed approach is to optimize the parameters of the SVM model, and locate the best features subset simultaneously. Eighteen low- and high-dimensional benchmark data sets are used to evaluate the accuracy of the proposed approach. For verification, the proposed approach is compared with seven well-regarded algorithms. Furthermore, the proposed approach is compared with grid search, which is the most popular technique for tuning SVM parameters. The experimental results show that the proposed approach outperforms all of the other techniques in most of the data sets in terms of classification accuracy, while minimizing the number of selected features.

/pdf/simultaneous-feature-selection-and-support-vector-machine-1dhqt91ln7.pdf

Simultaneous Feature Selection and Support Vector Machine Optimization Using the Grasshopper Optimization Algorithm

Least Squares Twin Support Vector Machine (LSTSVM) is a binary classifier and the extension of it to multiclass is still an ongoing research issue. In this paper, we extended the formulation of binary LSTSVM classifier to multi-class by using the concepts such as "One-versus-All", "One-versus-One", "All-versus-One" and Directed Acyclic Graph (DAG). This paper performs a comparative analysis of these multi-classifiers in terms of their advantages, disadvantages and computational complexity. The performance of all the four proposed classifiers has been validated on twelve benchmark datasets by using predictive accuracy and training-testing time. All the proposed multi-classifiers have shown better performance as compared to the typical multi-classifiers based on 'Support Vector Machine' and 'Twin Support Vector Machine'. Friedman's statistic and Nemenyi post hoc tests are also used to test significance of predictive accuracy differences between classifiers.

A comparison on multi-class classification methods based on least squares twin support vector machine

In this paper, we propose a new linear programming formulation of exact 1-norm twin support vector machine (TWSVM) for classification whose solution is obtained by solving a pair of dual exterior penalty problems as unconstrained minimization problems using Newton–Armijo algorithm. The idea of our formulation is to reformulate TWSVM as a strongly convex problem by incorporated regularization techniques and then derive an exact 1-norm linear programming formulation for TWSVM to improve robustness and sparsity. The solution of two modified unconstrained minimization problems reduces to solving just two systems of linear equations as opposed to solving two quadratic programming problems in TWSVM and TBSVM, which leads to extremely simple and fast algorithm. One significant advantage of our proposed method is the implementation of structural risk minimization principle. However, only empirical risk is considered in the primal problems of TWSVM due to its complex structure and thus may incur overfitting and suboptimal in some cases. Our approach has the advantage that a pair of matrix equation of order equals to the number of input examples is solved at each iteration of the algorithm. The algorithm converges from any starting point that can be easily implemented in MATLAB without using any optimization packages. Computational comparisons of our proposed method against original TWSVM, GEPSVM and SVM have been made on both synthetic and benchmark datasets. Experimental results show that our method is better or comparable in both computation time and classification accuracy.

Robust and Sparse Linear Programming Twin Support Vector Machines

Twin Support Vector Machine: A review from 2007 to 2014

This paper proposes a hybrid intelligent method for multi-fault detection of rotating machinery, in which three methods, i.e. including the redundant second generation wavelet package transform (RSGWPT), the kernel principal component analysis (KPCA) and the twin support vector machine (TWSVM), are combined. Firstly, RSGWPT is used to extract feature vectors from representative statistical characteristics in the decomposition frequency band, and then the KPCA in the feature space is performed to reduce the dimension of features and to extract the dominant features for the following classification. Finally, a novel support vector machine, called twin support vector machine is used to construct a multi-class classifier. Inputting superior features to this classifier, the condition of the monitored machine component can be determined. Experimental results demonstrate that the proposed hybrid method is effective for multi-fault detection of rotating machinery. The TWSVM is also indicated that has better classification performance and faster convergence speed than the normal SVM.

A hybrid intelligent multi-fault detection method for rotating machinery based on RSGWPT, KPCA and Twin SVM.

Twin support vector machine (TSVM) is a novel machine learning algorithm, which aims at finding two nonparallel planes for each class. In order to do so, one needs to resolve a pair of smaller-sized quadratic programming problems rather than a single large one. Classical TSVM is proposed for the binary classification problem. However, multi-class classification problem is often met in our real world. For this problem, a new multi-class classification algorithm, called Twin-KSVC, is proposed in this paper. It takes the advantages of both TSVM and K-SVCR (support vector classification-regression machine for k-class classification) and evaluates all the training points into a “1-versus-1-versus-rest” structure, so it generates ternary outputs { −1, 0, +1}. As all the samples are utilized in constructing the classification hyper-plane, our proposed algorithm yields higher classification accuracy in comparison with other two algorithms. Experimental results on eleven benchmark datasets demonstrate the feasibility and validity of our proposed algorithm.

Rui Guo

Papers

A Twin Multi-Class Classification Support Vector Machine