Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Data Mining - Concepts and Techniques.

A comprehensive survey on support vector machine classification: Applications, challenges and trends

We propose a novel nonparallel classifier, called nonparallel support vector machine (NPSVM), for binary classification. Our NPSVM that is fully different from the existing nonparallel classifiers, such as the generalized eigenvalue proximal support vector machine (GEPSVM) and the twin support vector machine (TWSVM), has several incomparable advantages: 1) two primal problems are constructed implementing the structural risk minimization principle; 2) the dual problems of these two primal problems have the same advantages as that of the standard SVMs, so that the kernel trick can be applied directly, while existing TWSVMs have to construct another two primal problems for nonlinear cases based on the approximate kernel-generated surfaces, furthermore, their nonlinear problems cannot degenerate to the linear case even the linear kernel is used; 3) the dual problems have the same elegant formulation with that of standard SVMs and can certainly be solved efficiently by sequential minimization optimization algorithm, while existing GEPSVM or TWSVMs are not suitable for large scale problems; 4) it has the inherent sparseness as standard SVMs; 5) existing TWSVMs are only the special cases of the NPSVM when the parameters of which are appropriately chosen. Experimental results on lots of datasets show the effectiveness of our method in both sparseness and classification accuracy, and therefore, confirm the above conclusion further. In some sense, our NPSVM is a new starting point of nonparallel classifiers.

/pdf/nonparallel-support-vector-machines-for-pattern-3aiuies44a.pdf

Nonparallel Support Vector Machines for Pattern Classification

Nonparallel hyperplane support vector machine for binary classification problems

Least squares recursive projection twin support vector machine for classification

This study proposes a new regressor—e-twin support vector regression (e-TSVR) based on TSVR. e-TSVR determines a pair of e-insensitive proximal functions by solving two related SVM-type problems. Different form only empirical risk minimization is implemented in TSVR, the structural risk minimization principle is implemented by introducing the regularization term in primal problems of our e-TSVR, yielding the dual problems to be stable positive definite quadratic programming problems, so can improve the performance of regression. In addition, the successive overrelaxation technique is used to solve the optimization problems to speed up the training procedure. Experimental results for both artificial and real datasets show that, compared with the popular e-SVR, LS-SVR and TSVR, our e-TSVR has remarkable improvement of generalization performance with short training time.

An ε-twin support vector machine for regression

Multiple recursive projection twin support vector machine (MPTSVM) is a recently proposed classifier and has been proved to be outstanding in pattern recognition However, MPTSVM is computationally expensive since it involves solving a series of quadratic programming problems To relieve the training burden, in this paper, we propose a novel multiple least squares recursive projection twin support vector machine (MLSPTSVM) based on least squares recursive projection twin support vector machine (LSPTSVM) for multi-class classification problem For a $$K(K>2)$$
 classes classification problem, MLSPTSVM aims at seeking K groups of projection axes, one for each class that separates it from all the other Due to solving a series of linear equations, our algorithm tends to relatively simple and fast Moreover, a recursive procure is introduced to generate multiple orthogonal projection axes for each class to enhance its performance Experimental results on several synthetic and UCI datasets, as well as on relatively large datasets demonstrate that our MLSPTSVM has comparable classification accuracy while takes significantly less computing time compared with MPTSVM, and also obtains better performance than several other SVM related methods being used for multi-class classification problem

Least squares recursive projection twin support vector machine for multi-class classification

The Best Separating Decision Tree Twin Support Vector Machine for Multi-Class Classification☆

In many cases, the output of a classifier should be a calibrated posterior probability to enable post-processing. However, twin support vector machines (TWSVM) do not provide such probabilities. In this paper, we propose a TWSVM probability model, called PTWSVM, to estimate the posterior probability. Note that our model is quite different from the SVM probability model because of the different mechanism of TWSVM and SVM. In fact, for TWSVM, we first define a new ranking continues output by comparing the distances between the sample and the two non-parallel hyperplanes, and then map this ranking continues output into probability by training the parameters of an additional sigmoid function. Our PTWSVM has been tested on both artificial datasets and several data-mining-style datasets, and the numerical experiments indicate that PTWSVM yields nice results.

Zhi-Min Yang

Papers

Least squares recursive projection twin support vector machine for classification

An ε-twin support vector machine for regression

Least squares recursive projection twin support vector machine for multi-class classification

The Best Separating Decision Tree Twin Support Vector Machine for Multi-Class Classification☆

Probabilistic outputs for twin support vector machines