Statistical learning theory

About: Statistical learning theory is a research topic. Over the lifetime, 1618 publications have been published within this topic receiving 158033 citations.

Study of Least Squares Support Vector Machines

Abstract: In this paper, we present a least squares version for support vector machines(SVM)classifiers and function estimation. Due to equality type constraints in the formulation, the solution follows from solving a set of linear equations, instead of quadratic programming for classical SVM. The approach is illustrated on a two-spiral benchmark classification problem. The results show that the LS-SVM is an efficient method for solving pattern recognition.

Theoretical learning guarantees applied to acoustic modeling

Abstract: In low-resource scenarios, for example, small datasets or a lack in computational resources available, state-of-the-art deep learning methods for speech recognition have been known to fail. It is possible to achieve more robust models if care is taken to ensure the learning guarantees provided by the statistical learning theory. This work presents a shallow and hybrid approach using a convolutional neural network feature extractor fed into a hierarchical tree of support vector machines for classification. Here, we show that gross errors present even in state-of-the-art systems can be avoided and that an accurate acoustic model can be built in a hierarchical fashion. Furthermore, we present proof that our algorithm does adhere to the learning guarantees provided by the statistical learning theory. The acoustic model produced in this work outperforms traditional hidden Markov models, and the hierarchical support vector machine tree outperforms a multi-class multilayer perceptron classifier using the same features. More importantly, we isolate the performance of the acoustic model and provide results on both the frame and phoneme level, considering the true robustness of the model. We show that even with a small amount of data, accurate and robust recognition rates can be obtained.

Intelligent Biometric Information Management

Abstract: We advance here a novel methodology for robust intelligent biometric information management with inferences and predictions made using randomness and complexity concepts. Intelligence refers to learning, adap- tation, and functionality, and robustness refers to the ability to handle incomplete and/or corrupt adversarial information, on one side, and image and or device variability, on the other side. The proposed methodology is model-free and non-parametric. It draws support from discriminative methods using likelihood ratios to link at the conceptual level biometrics and forensics. It further links, at the modeling and implementation level, the Bayesian framework, statistical learning theory (SLT) using transduction and semi-supervised lea- rning, and Information Theory (IY) using mutual information. The key concepts supporting the proposed methodology are a) local estimation to facilitate learning and prediction using both labeled and unlabeled data; b) similarity metrics using regularity of patterns, randomness deficiency, and Kolmogorov complexity (similar to MDL) using strangeness/typicality and ranking p-values; and c) the Cover – Hart theorem on the asymptotical performance of k-nearest neighbors approaching the optimal Bayes error. Several topics on biometric inference and prediction related to 1) multi-level and multi-layer data fusion including quality and multi-modal biometrics; 2) score normalization and revision theory; 3) face selection and tracking; and 4) identity management, are described here using an integrated approach that includes transduction and boosting for ranking and sequential fusion/aggregation, respectively, on one side, and active learning and change/ outlier/intrusion detection realized using information gain and martingale, respectively, on the other side. The methodology proposed can be mapped to additional types of information beyond biometrics.

Mine water discharge prediction based on least squares support vector machines

Abstract: In order to realize the prediction of a chaotic time series of mine water discharge, an approach incorporating phase space reconstruction theory and statistical learning theory was studied. A differential entropy ratio method was used to determine embedding parameters to reconstruct the phase space. We used a multi-layer adaptive best-fitting parameter search algorithm to estimate the LS-SVM optimal parameters which were adopted to construct a LS-SVM prediction model for the mine water chaotic time series. The results show that the simulation performance of a single-step prediction based on this LS-SVM model is markedly superior to that based on a RBF model. The multi-step prediction results based on LS-SVM model can reflect the development of mine water discharge and can be used for short-term forecasting of mine water discharge.

Forecasting daily ambient air pollution based on least squares support vector machines

Abstract: Meteorological and pollutions data are collected daily at monitoring stations of a city. This pollutant-related information can be used to build an early warning system, which provides forecast and also alarms health advice to local inhabitants by medical practicians and local government. In the literature, air quality or pollutant level predictive models using multi-layer perceptrons (MLP) have been employed at a variety of cities by environmental researchers. The practical applications of these models however suffer from different drawbacks so that good generalization may not be obtained. Least squares support vector machines (LS-SVM), a novel type of machine learning technique based on statistical learning theory, can be used for regression and time series prediction. LS-SVM can overcome most of the drawbacks of MLP and has been reported to show promising results.