Statistical learning theory

About: Statistical learning theory is a research topic. Over the lifetime, 1618 publications have been published within this topic receiving 158033 citations.

A statistical modeling approach to content based retrieval

Abstract: Statistical modeling for content based retrieval is examined in the context of recent TREC Video benchmark exercise. The TREC Video exercise can be viewed as a test bed for evaluation and comparison of a variety of different algorithms on a set of high-level queries for multimedia retrieval. We report on the use of techniques adopted from statistical learning theory. Our method, as in most statistical methods, depend on training of models based on large data sets. A plethora of statistical models such the Gaussian mixture models, support vector machines etc. can be thought of, only a few of which are exploited in this preliminary report. Training requires a large amount of annotated (labeled) data. Thus, we explore use of active learning for the annotation engine that minimizes the number of training samples to be labeled for satisfactory performance.

Towards Understanding Generalization via Analytical Learning Theory

Abstract: This paper introduces a novel measure-theoretic theory for machine learning that does not require statistical assumptions. Based on this theory, a new regularization method in deep learning is derived and shown to outperform previous methods in CIFAR-10, CIFAR-100, and SVHN. Moreover, the proposed theory provides a theoretical basis for a family of practically successful regularization methods in deep learning. We discuss several consequences of our results on one-shot learning, representation learning, deep learning, and curriculum learning. Unlike statistical learning theory, the proposed learning theory analyzes each problem instance individually via measure theory, rather than a set of problem instances via statistics. As a result, it provides different types of results and insights when compared to statistical learning theory.

An Evolutionary Statistical Learning Theory

Abstract: Statistical learning theory was developed by Vapnik. It is a learning theory based on Vapnik-Chervonenkis dimension. It also has been used in learning models as good analytical tools. In general, a learning theory has had several problems. Some of them are local optima and over-fitting problems. As well, statistical learning theory has same problems because the kernel type, kernel parameters, and regularization constant C are determined subjectively by the art of researchers. So, we propose an evolutionary statistical learning theory to settle the problems of original statistical learning theory. Combining evolutionary computing into statistical learning theory, our theory is constructed. We verify improved performances of an evolutionary statistical learning theory using data sets from KDD cup. Keywords—Evolutionary computing, Local optima, Over-fitting, Statistical learning theory

Constraint-Aware Data Analysis on Mobile Devices: An Application to Human Activity Recognition on Smartphones

Abstract: Most state-of-the-art machine learning (ML) algorithms do not consider the computational constraints of implementing their learned models on mobile devices. These constraints are, for example, the limited depth of the arithmetic unit, the memory availability, and the battery capacity. We propose a new learning framework, which relies on Statistical Learning Theory that includes these constraints inside the learning process itself. This new framework allows to train advanced resource-sparing ML models and to efficiently deploy them on smart mobile devices. The advantages of our proposal are presented on a smartphone-based Human Activity Recognition application and compared against a conventional ML approach.

The Application of SVR to Prediction of Chaotic Time Series

Abstract: Support vector machines (SVM) are a kind of novel machine learning methods based on statistical learning theory, which has been developed to solve classification and regression problems. This paper applies support vector regression (SVR) to chaotic time series prediction, and compares the prediction results with BP network and RBF network. The results of simulation experiments show that SVR has a good generalization ability and capability of tolerating noise.