Statistical learning theory

About: Statistical learning theory is a research topic. Over the lifetime, 1618 publications have been published within this topic receiving 158033 citations.

Qualitative Robustness of Bootstrap Approximations for Kernel Based Methods

Abstract: The finite sample distribution of many nonparametric methods from statistical learning theory is unknown because the distribution P from which the data were generated is unknown and because there often exist only asymptotical results on the behaviour of such methods.

Research on the Segmentation of MRI Image Based on Multi-Classification Support Vector Machine

Abstract: In head MRI image, the boundary of each encephalic tissue is highly complicated and irregular. It is a real challenge to traditional segmentation algorithms. As a new kind of machine learning, support vector machine (SVM) based on statistical learning theory (SLT) has high generalization ability, especially for dataset with small number of samples in high dimensional space. SVM was originally developed for two-class classification. It is extended to solve multi-class classification problem. In this paper, 57 dimensional feature vectors for MRI image are selected as input for SVM. The segmentation of MRI image based on the multi-classification SVM (MCSVM) is investigated. As our experiment demonstrates, the boundaries of 7 kinds of encephalic tissues are extracted successfully, and it can reach satisfactory generalization accuracy. Thus, SVM exhibits its great potential in image segmentation.

Support Vector Machines for Classification of Multi- and Hyperspectral Data

Abstract: As discussed in Chap. 5, support vector machines (SVMs) have originated from statistical learning theory for classification and regression problems. Unlike the popular neural network classifiers, SVMs do not minimize the empirical training error (Byun and Lee 2003). Instead, they aim to maximize the margin between two classes of interest by placing a linear separating hyperplane between them. While doing so, the upper bound on the generalization error is minimized. Thus, SVM based classifiers are expected to have more generalization capability than neural networks. Other advantages of SVMs are their ability to adapt their learning characteristic via a kernel function and to adequately classify data on a high-dimensional feature space with a limited number of training data sets thereby overcoming the Hughes Phenomenon. The theoretical background on SVMs has been presented in Chap. 5. This chapter builds on this theoretical knowledge to apply SVMs for classification of multi and hyperspectral remote sensing data.

Support vector machines based on the orthogonal projection kernel of father wavelet

Abstract: Recently the study on the theory of wavelets shows that the wavelets have not only the multi-resolution property both in frequency and time domain, but also the good approximation ability. SVMs based on the statistical learning theory are a kind of general and effective learning machines, and have described for us the nice application blueprint in machine learning domain. There exists a bottleneck problem, or the pre-selection of kernel parameter for SVMs. In this paper, the orthogonal projection kernels of father wavelet (OPFW kernels) are introduced into SVMs. In doing so SVMs based on the OPFW kernels can have good performance in both approximation and generalisation. Simultaneously the parameter pre-selection of wavelet kernels can be implemented by discrete wavelet transform. Experiments on regression estimation illustrate the approximation and generalisation ability of our method.

Measuring the shattering coefficient of decision tree models

Abstract: In spite of the relevance of Decision Trees (DTs), there is still a disconnection between their theoretical and practical results while selecting models to address specific learning tasks. A particular criterion is provided by the Shattering coefficient, a growth function formulated in the context of the Statistical Learning Theory (SLT), which measures the complexity of the algorithm bias as sample sizes increase. In attempt to establish the basis for a relative theoretical complexity analysis, this paper introduces a method to compute the Shattering coefficient of DT models using recurrence equations. Next, we assess the bias of models provided by DT algorithms while solving practical problems as well as their overall learning bounds in light of the SLT. As the main contribution, our results support other researchers to decide on the most adequate DT models to tackle specific supervised learning tasks.