Statistical learning theory

About: Statistical learning theory is a research topic. Over the lifetime, 1618 publications have been published within this topic receiving 158033 citations.

Chaotic particle swarm optimization algorithm for support vector machine

Abstract: Statistical Learning Theory focuses on the machine learning theory for small samples Support vector machine (SVM) are new methods based on statistical learning theory There are many kinds of function can be used for kernel of SVM Wavelet function is a set of bases that can approximate arbitrary functions in arbitrary precision So Marr wavelet was used to construct wavelet kernel On the other hand, the parameter selection should to be done before training WSVM Modified chaotic particle swarm optimization (CPOS) was adopted to select parameters of SVM It is shown by simulation that the CPOS algorithm can derive a set of optimal parameters of WSVM, and WSVM model possess some advantages such as simple structure, fast convergence speed with high generalization ability

Application of SVM to engine parameter collector fault diagnosis

Abstract: Support Vector Machine (SVM), based on structural risk minimization principle, is now widely used in pattern recognition, classification and other research fields It shows better generalization performance than traditional statistical learning theory, especially in small samples In this paper, some dimensionless parameter is selected as SVM eigenvector, and then support vector machine is applied to fault diagnosis in engine parameter collector Result shows that it has good ability in fault pattern classification of engine parameter collector

The Key Theorem of Learning Theory Based on Random Sets Samples

Abstract: Statistical learning theory based on random samples is regarded as the best theory for dealing with small-sample learning problems at present. And it has become an interesting research after neural networks in machine learning. But it can hardly be handle by the learning problems based on random sets samples. In this paper, combined with the theory of random sets, the definition of the subtraction between the set and the real number is presented, and then some correlative theorems are proven. According to these, some of main concepts of statistical learning theory based on random sets samples are introduced, and at last, the key theorem of learning theory based on random sets samples is given and proven.

Estimating the Generalization Performance of Polynomial SVM Classifier for Text Categorization

Abstract: VC theory and structural risk minimization principle are key concepts of statistical learning theory. Developed from this theory, SVM is widly investigated and used for text categorization because of its high generalization performance. Previous work showed that polynomial SVM's performance was irrevelant of the order and it was appropriate for high dimensional text categorization problems without feature selection. The research indicates over-fitting problems occur as the polynomial order increases. SVM's generalization performance decreases drastically if too many features are used, so feature selection is necessary. Based on the structural risk minimization principle, this fact is analyzed via estimating functional classes's VC dimension. And the empirical results support the theoretical conclusions.

Deep Neural Networks for Choice Analysis: A Statistical Learning Theory Perspective

Abstract: While researchers increasingly use deep neural networks (DNN) to analyze individual choices, overfitting and interpretability issues remain as obstacles in theory and practice. By using statistical learning theory, this study presents a framework to examine the tradeoff between estimation and approximation errors, and between prediction and interpretation losses. It operationalizes the DNN interpretability in the choice analysis by formulating the metrics of interpretation loss as the difference between true and estimated choice probability functions. This study also uses the statistical learning theory to upper bound the estimation error of both prediction and interpretation losses in DNN, shedding light on why DNN does not have the overfitting issue. Three scenarios are then simulated to compare DNN to binary logit model (BNL). We found that DNN outperforms BNL in terms of both prediction and interpretation for most of the scenarios, and larger sample size unleashes the predictive power of DNN but not BNL. DNN is also used to analyze the choice of trip purposes and travel modes based on the National Household Travel Survey 2017 (NHTS2017) dataset. These experiments indicate that DNN can be used for choice analysis beyond the current practice of demand forecasting because it has the inherent utility interpretation, the flexibility of accommodating various information formats, and the power of automatically learning utility specification. DNN is both more predictive and interpretable than BNL unless the modelers have complete knowledge about the choice task, and the sample size is small. Overall, statistical learning theory can be a foundation for future studies in the non-asymptotic data regime or using high-dimensional statistical models in choice analysis, and the experiments show the feasibility and effectiveness of DNN for its wide applications to policy and behavioral analysis.