Statistical learning theory

About: Statistical learning theory is a research topic. Over the lifetime, 1618 publications have been published within this topic receiving 158033 citations.

A Study of SVM-Constructions Based on Kernels

Abstract: The key to SVM-constuctions is to select appreciate kernel function.Study the primary properties of three kinds of current kernels by using machine learning and derive some other related functions.Introduce some more sophisticated kernels which can be used of SVM-constructions.This paper proposes some principles that choosing kernel functions when construct a Nonlinear classifier using SVM,also,analyzes the computability and generalization ability of kernel method by some examples.As a result,expand the application scopes of kernels and make the methods of SVM-construction more diversiform.

From bits to information with learning machines: theory and applications

Abstract: Summary form only given. Learning is becoming the central problem in trying to understand intelligence and in trying to develop intelligent machines. The paper outlines some previous efforts in developing machines that learn. It sketches the authors's work on statistical learning theory and theoretical results on the problem of classification and function approximation that connect regularization theory and support vector machines. The main application focus is classification (and regression) in various domains-such as sound, text, video and bioinformatics. In particular, the paper describe the evolution of a trainable object detection system for classifying objects-such as faces and people and cars-in complex cluttered images. Finally, it speculates on the implications of this research for how the brain works and review some data which provide a glimpse of how 3D objects are represented in the visual cortex.

Support vector machine model for predicting sand liquefaction based on clustering binary tree algorithm

Abstract: Support vector machine(SVM) is a novel and powerful learning method which is derived based on statistical learning theory(SLT) and the structural risk minimization principle.Traditional SVM only deals with the binary classification problems,however,there are large numbers of multi-classification problems in practical engineering.With a view to the main factors with important influence on sand liquefaction,multi-class support vector machines model based on clustering-binary tree is established.The nonlinear relationship between sand liquefaction and influencing factors is obtained from the finite empirical data by SVM.The results indicate that the structure of model base on SVM is reasonable;this algorithm is feasible;and it can predict the sand liquefaction more accurately.

Information Bottleneck and Representation Learning

Abstract: A grand challenge in representation learning is the development of computational algorithms that learn the different explanatory factors of variation behind high-dimensional data. Representation models (usually referred to as encoders) are often determined for optimizing performance on training data when the real objective is to generalize well to other (unseen) data. The first part of this chapter is devoted to provide an overview of and introduction to fundamental concepts in statistical learning theory and the Information Bottleneck principle. It serves as a mathematical basis for the technical results given in the second part, in which an upper bound to the generalization gap corresponding to the cross-entropy risk is given. When this penalty term times a suitable multiplier and the cross entropy empirical risk are minimized jointly, the problem is equivalent to optimizing the Information Bottleneck objective with respect to the empirical data distribution. This result provides an interesting connection between mutual information and generalization, and helps to explain why noise injection during the training phase can improve the generalization ability of encoder models and enforce invariances in the resulting representations.