Statistical learning theory

About: Statistical learning theory is a research topic. Over the lifetime, 1618 publications have been published within this topic receiving 158033 citations.

Research on Natural Gas Short-Term Load Forecasting Based on Support Vector Regression

Abstract: Natural gas load forecasting is a key process to the efficient operation of pipeline network. An accurate forecast is required to guarantee a balanced network operation and ensure safe gas supply at a minimum cost. Machine learning techniques have been increasingly applied to load forecasting. A novel regression technique based on the statistical learning theory, support vector machines (SVM), is investigated in this paper for natural gas short-term load forecasting. SVM is based on the principle of structure risk minimization as opposed to the principle of empirical risk minimization in conventional regression techniques. Using a data set with 2 years load values we developed prediction model using SVM to obtain 31 days load predictions. The results on city natural gas short-term load forecasting show that SVM provides better prediction accuracy than neural network. The software package natural gas pipeline networks simulation and load forecasting (NGPNSLF) based on support vector regression prediction has been developed, which has also been applied in practice.

Fluctuation-dissipation theorem and models of learning

Abstract: Advances in statistical learning theory have resulted in a multitude of different designs of learning machines. But which ones are implemented by brains and other biological information processors? We analyze how various abstract Bayesian learners perform on different data and argue that it is difficult to determine which learning-theoretic computation is performed by a particular organism using just its performance in learning a stationary target (learning curve). Basing on the fluctuation-dissipation relation in statistical physics, we then discuss a different experimental setup that might be able to solve the problem.

From Support Vector Machines to Hybrid System Identification

Abstract: The thesis focuses on three nonlinear modeling problems: classification (or pattern recognition), regression (or function approximation) and hybrid system identification. Amongst existing approaches, Support Vector Machines (SVMs) offer a general framework for both nonlinear classification and regression. These recent methods, based on statistical learning theory, rely on convex optimization to train black-box models with good generalization performances. The study first focuses on the evolution of these models towards grey-box models, which can benefit at the same time from the universal approximation capacity of black-box models and from prior knowledge. In particular, the thesis proposes a general framework for the incorporation of a wide variety of prior knowledge in SVMs for regression, which leads to a learning algorithm written as a convex optimization program. The last part of the thesis proposes to extend SVMs to the identification of hybrid systems, that switch between different dynamics. In this context, the classification and regression problems are intrinsically mixed together and cannot be considered separately. A method based on non-convex optimization is proposed to solve these problems simultaneously. The resulting algorithm is able to deal with hybrid systems switching between arbitrary and unknown dynamics.

Effect of choice of kernel in support vector machines on ambient air pollution forecasting

Abstract: Forecasting of air pollution is a popular and important topic in recent year due to the health impact caused by air pollution. It is necessary to build an early warning system, which provides forecast and also alerts health alarm to local inhabitants by medical practicians and local government. Meteorological and pollutions data collected daily at monitoring stations of Macau can be used in this study to build a forecasting system. Support vector machines (SVM), a novel type of machine learning technique based on statistical learning theory, can be used for regression and time series prediction. SVM is capable of good generalization while the performance of the SVM model is often hinged on the appropriate choice of the kernel.

Permutational Rademacher Complexity

Abstract: Transductive learning considers situations when a learner observes m labelled training points and u unlabelled test points with the final goal of giving correct answers for the test points. This paper introduces a new complexity measure for transductive learning called Permutational Rademacher Complexity PRC and studies its properties. A novel symmetrization inequality is proved, which shows that PRC provides a tighter control over expected suprema of empirical processes compared to what happens in the standard i.i.d. setting. A number of comparison results are also provided, which show the relation between PRC and other popular complexity measures used in statistical learning theory, including Rademacher complexity and Transductive Rademacher Complexity TRC. We argue that PRC is a more suitable complexity measure for transductive learning. Finally, these results are combined with a standard concentration argument to provide novel data-dependent risk bounds for transductive learning.