Statistical learning theory

About: Statistical learning theory is a research topic. Over the lifetime, 1618 publications have been published within this topic receiving 158033 citations.

Classification of hyperspectral remote sensing image based on genetic algorithm and SVM

Abstract: Hyperspectral remote sensing data has been widely used in Terrain Classification for its high resolution. The classification of urban vegetation, identified as an indispensable and essential part of urban development system, is now facing a major challenge as different complex land-cover classes having similar spectral signatures. For a better accuracy in classification of urban vegetation, a classifier model was designed in this paper based on genetic algorithm (GA) and support vector machine (SVM) to address the multiclass problem, and tests were made with the classification of PHI hyperspectral remote sensing images acquired in 2003 which partially covers a corner of the Shanghai World Exposition Park, while PHI is a hyper-spectral sensor developed by Shanghai Institute of Technical Physics. SVM, based on statistical learning theory and structural risk minimization, is now widely used in classification in many fields such as two-class classification, and also the multi-class classification later due to its superior performance. On the other hand as parameters are very important factors affecting SVM's ability in classification, therefore, how to choose the optimal parameters turned out to be one of the most urgent problems. In this paper, GA was used to acquire the optimal parameters with following 3 steps. Firstly, useful training samples were selected according to the features of hyperspectral images, to build the classifier model by applying radial basis function (RBF) kernel function and decision Directed Acyclic Graph (DAG) strategy. Secondly, GA was introduced to optimize the parameters of SVM classification model based on the gridsearch and Bayesian algorithm. Lastly, the proposed GA-SVM model was tested for results' accuracy comparison with the maximum likelihood estimation and neural network model. Experimental results showed that GA-SVM model performed better classified accuracy, indicating the coupling of GA and SVM model could improve classification accuracy of hyperspectral remote sensing images, especially in vegetation classification.

Benders decomposition technique for support vector regression

Abstract: The theory of the support vector machine (SVM) algorithm is based on the statistical learning theory. Training of SVMs leads to either a quadratic programming (QP) problem, or linear programming (LP) problem. This depends on the specific norm that is used when the distance between the convex hulls of two classes are computed. The l/sub 1/ norm distance leads to a large scale linear programming problem in the case where the sample size is very large. We propose to apply the Benders decomposition technique to the resulting LP for the regression case. Preliminary results show that this technique is much faster than the QP formulation.

SMO Algorithm Applied in Time Series Model Building and Forecast

Abstract: As a novel learning machine, the support vector machine (SVM) based on statistical learning theory can be used for regression: support vector regression (SVR). SVR has been applied successfully to time-series analysis, but its optimization algorithm is usually built up from certain quadratic programming (QP) packages. Therefore, for small datasets this is practical and QP routines are the best choice, but for large datasets, data processing runtimes become lengthy, which limits its application. Sequential minimal optimization (SMO) algorithm can improve operation speed and reduce this long runtime. In this paper, SVR that is based on the SMO algorithm is used to forecast two typical time series models: Wolfer sunspot number data and Box and Jenkins gas furnace data. The results of simulation prove that the operational speed of SVR using the SMO algorithm is improved in comparison to SVR employing QP optimization algorithm; moreover, the forecasting precision is better than that of neural network and SVR using QP optimization algorithm.

Individual intelligent method-based fault diagnosis

Abstract: This chapter introduces the intelligent diagnosis methods based on individual intelligent techniques. The concept and advantages of intelligent diagnosis are first described, as well as the main steps commonly included in intelligent diagnosis. Second, three methods using artificial neural networks, which are able to learn and generalize nonlinear relationships between input data and output data, are presented for diagnosing the mechanical faults. Then, two diagnosis methods based on statistical learning theory are detailed and they can give better generalization abilities, especially for limited sample cases. Finally, two intelligent diagnosis methods are introduced based on the idea of deep learning, which uses advanced intelligent techniques for both feature extraction and fault classification. The effectiveness of each method is verified by various diagnosis cases, involving intelligent diagnosis of rub faults, bearing faults, and gear faults, and these methods can replace diagnosticians to efficiently process the massive collected signals and automatically diagnose the mechanical faults.