Open AccessProceedings Article
Algorithmic Stability and Generalization Performance
Olivier Bousquet,André Elisseeff +1 more
- Vol. 13, pp 196-202
Reads0
Chats0
TLDR
This work presents a novel way of obtaining PAC-style bounds on the generalization error of learning algorithms, explicitly using their stability properties, and demonstrates that regularization networks possess the required stability property.Abstract:
We present a novel way of obtaining PAC-style bounds on the generalization error of learning algorithms, explicitly using their stability properties. A stable learner is one for which the learned solution does not change much with small changes in the training set. The bounds we obtain do not depend on any measure of the complexity of the hypothesis space (e.g. VC dimension) but rather depend on how the learning algorithm searches this space, and can thus be applied even when the VC dimension is infinite. We demonstrate that regularization networks possess the required stability property and apply our method to obtain new bounds on their generalization performance.read more
Citations
More filters
Book
Kernel Methods for Pattern Analysis
TL;DR: This book provides an easy introduction for students and researchers to the growing field of kernel-based pattern analysis, demonstrating with examples how to handcraft an algorithm or a kernel for a new specific application, and covering all the necessary conceptual and mathematical tools to do so.
Book ChapterDOI
A Generalized Representer Theorem
TL;DR: The result shows that a wide range of problems have optimal solutions that live in the finite dimensional span of the training examples mapped into feature space, thus enabling us to carry out kernel algorithms independent of the (potentially infinite) dimensionality of the feature space.
Journal ArticleDOI
Stability and generalization
Olivier Bousquet,André Elisseeff +1 more
TL;DR: These notions of stability for learning algorithms are defined and it is shown how to use these notions to derive generalization error bounds based on the empirical error and the leave-one-out error.
Book
Adaptive computation and machine learning
TL;DR: This book attempts to give an overview of the different recent efforts to deal with covariate shift, a challenging situation where the joint distribution of inputs and outputs differs between the training and test stages.
Book ChapterDOI
Regularization and Semi-supervised Learning on Large Graphs
TL;DR: This work considers the problem of labeling a partially labeled graph, which may arise in a number of situations from survey sampling to information retrieval to pattern recognition in manifold settings.
References
More filters
Journal ArticleDOI
Theory of Reproducing Kernels.
TL;DR: In this paper, a short historical introduction is given to indicate the different manners in which these kernels have been used by various investigators and discuss the more important trends of the application of these kernels without attempting, however, a complete bibliography of the subject matter.
Journal ArticleDOI
Regularization algorithms for learning that are equivalent to multilayer networks.
Tomaso Poggio,Federico Girosi +1 more
TL;DR: A theory is reported that shows the equivalence between regularization and a class of three-layer networks called regularization networks or hyper basis functions.
Journal ArticleDOI
Algorithmic stability and sanity-check bounds for leave-one-out cross-validation
Michael Kearns,Dana Ron +1 more
TL;DR: This article proves sanity-check bounds for the error of the leave-one-out cross-validation estimate of the generalization error: that is, bounds showing that the worst-case error of this estimate is not much worse than that of the training error estimate.
Journal ArticleDOI
Scale-sensitive dimensions, uniform convergence, and learnability
TL;DR: A characterization of learnability in the probabilistic concept model, solving an open problem posed by Kearns and Schapire, and shows that the accuracy parameter plays a crucial role in determining the effective complexity of the learner's hypothesis class.