Topic
Statistical learning theory
About: Statistical learning theory is a research topic. Over the lifetime, 1618 publications have been published within this topic receiving 158033 citations.
Papers published on a yearly basis
Papers
More filters
TL;DR: It turns out that the oscillation bound is consistent with the experimental results about ELM obtained before and predicts that overfitting can be avoided even when the number of hidden nodes approaches infinity.
14 citations
01 Dec 2007
TL;DR: Using as a starting point one-side results from statistical learning theory, bounds on the number of required samples that are manageable for "reasonable" values of confidence delta and accuracy isin are obtained.
Abstract: In this paper, we study two general semi-infinite programming problems by means of statistical learning theory. The sample size results obtained with this approach are generally considered to be very conservative by the control community. The main contribution of this paper is to demonstrate that this is not necessarily the case. Using as a starting point one-side results from statistical learning theory, we obtain bounds on the number of required samples that are manageable for "reasonable" values of confidence delta and accuracy isin. In particular, we provide sample size bounds growing with 1/isin ln 1/isin instead of the usual 1/isin2 ln 1/isin2 dependence.
14 citations
TL;DR: Three general methods for obtaining exact bounds on the probability of overfitting are proposed within statistical learning theory: a method of generating and destroying sets, a recurrent method, and a blockwise method.
Abstract: Three general methods for obtaining exact bounds on the probability of overfitting are proposed within statistical learning theory: a method of generating and destroying sets, a recurrent method, and a blockwise method. Six particular cases are considered to illustrate the application of these methods. These are the following model sets of predictors: a pair of predictors, a layer of a Boolean cube, an interval of a Boolean cube, a monotonic chain, a unimodal chain, and a unit neighborhood of the best predictor. For the interval and the unimodal chain, the results of numerical experiments are presented that demonstrate the effects of splitting and similarity on the probability of overfitting.
14 citations
TL;DR: In this article, the authors analyze how various abstract Bayesian learners perform on different data and argue that it is difficult to determine which learning-theoretic computation is performed by a particular organism using just its performance in learning a stationary target (learning curve).
Abstract: Advances in statistical learning theory have resulted in a multitude of different designs of learning machines. But which ones are implemented by brains and other biological information processors? We analyze how various abstract Bayesian learners perform on different data and argue that it is difficult to determine which learning–theoretic computation is performed by a particular organism using just its performance in learning a stationary target (learning curve). Based on the fluctuation–dissipation relation in statistical physics, we then discuss a different experimental setup that might be able to solve the problem.
14 citations
TL;DR: The techniques leading to the second oracle inequality are based on the well-known approach of adding some artificial noise to the labeling process and lead to faster learning rates than those of Blanchard et al. (2003) whenever the set of weak learners does not perfectly approximate the target function.
Abstract: We derive two oracle inequalities for regularized boosting algorithms for classification. The first oracle inequality generalizes and refines a result from Blanchard et al. (2003), while the second oracle inequality leads to faster learning rates than those of Blanchard et al. (2003) whenever the set of weak learners does not perfectly approximate the target function. The techniques leading to the second oracle inequality are based on the well-known approach of adding some artificial noise to the labeling process.
14 citations