Statistical learning theory

About: Statistical learning theory is a research topic. Over the lifetime, 1618 publications have been published within this topic receiving 158033 citations.

An Optimizing Algorithm of Super-parameters for SVM Based on Simulation Annealing

Abstract: Based on the statistical learning theory support vector machine focuses on the machine learning strategies under small samples and gets better generalization ability than that of those tools based on the experience risk minimization principle.Its classing or regression performance will be affected by relative super-parameters.An improved multi-object optimization algorithm based on simulated annealing is proposed and applied to super-parameters optimization of the support vector machine with a RBF kernel.Then selection of proper searching space,initial feasible solution,initial temperature and design of an optimal object function are discussed in detail.The validation on the some standard data sets is carried out and its feasibility and effectiveness are confirmed.

The improved method of least squares support vector machine modeling and its application

Abstract: Least squares support vector machines (LS-SVM) method is used for modeling, and its penalty factors and kernel parameters with different values will affect the accuracy of the soft sensor model. This paper presents a particle swarm optimization (PSO) algorithm with mutation to automatically search the parameters for LS-SVM, and is applied to real-time measurement problem of saturated vapor dryness in gas driving oil extraction. The proposed algorithm is based on statistical learning theory to map the complex nonlinear relationship between dryness and its influence factors by learning from empirical data, therefore, saturated vapor dryness can be forecasted. The experimental results show that soft sensor modeling based on particle swarm optimization with mutation has high precision, adaptability, and ease of practical application.

Application of LS-SVM in Speaker Recognition

Abstract: Speaker recognition is regarded as a kind of voice recognitionIt is one of the current research hotspotsThe support vector machines(SVM) based on ethe statistical learning theory is a new machine learning algorithm as the hotspots of machine learning researchAn improved SVM,the least square support vector machines(LS-SVM) is discussed in this paperThe experimental results demonstrate that the LS-SVM-based speaker recognition is less computational complexity and more effient than the SVM-based speaker recognitionThen it has high adaptability for the speaker recognition

A nonasymptotic law of iterated logarithm for robust online estimators.

Abstract: In this paper, we provide tight deviation bounds for M-estimators, which are valid with a prescribed probability for every sample size. M-estimators are ubiquitous in machine learning and statistical learning theory. They are used both for defining prediction strategies and for evaluating their precision. Our deviation bounds can be seen as a non-asymptotic version of the law of iterated logarithm. They are established under general assumptions such as Lipschitz continuity of the loss function and (local) curvature of the population risk. These conditions are satisfied for most examples used in machine learning, including those that are known to be robust to outliers and to heavy tailed distributions. To further highlight the scope of applicability of the obtained results, a new algorithm, with provably optimal theoretical guarantees, for the best arm identification in a stochastic multi-arm bandit setting is presented. Numerical experiments illustrating the validity of the algorithm are reported.

A Statistical Diagnosis Approach for Analyzing Design–Silicon Timing Mismatch

Abstract: Explaining the mismatch between predicted timing behavior from modeling and simulation, and the observed timing behavior measured on silicon chips can be very challenging. Given a list of potential sources, the mismatch can be the aggregate result caused by some of them both individually and collectively, resulting in a very large search space. Furthermore, observed data are always corrupted by some unknown statistical random noises. In this paper, we examine how trying to explain the mismatch observed on silicon can be classified as an ill-posed problem, where ill posed means that the solution may not be unique or stable. Thus, a small change in the observed response can have a large change in the predicted solution. To solve ill-posed problems, a statistical learning theory uses a principle called regularization. This paper proposes using a statistical learning method called support vector (SV) analysis to statistically analyze all known sources of uncertainty with the objective to rank which sources contribute the most to the observed mismatch. Experimental results are presented under different error assumption models to compare two kinds of SV ranking approaches to four other ranking approaches, where some use the idea of regularization and others do not. This paper is concluded by showing a self cross-validation approach to validate the ranking results when there is no true ranking available, as the case with actual silicon.