Journal ArticleDOI
Learnability and the Vapnik-Chervonenkis dimension
Reads0
Chats0
TLDR
This paper shows that the essential condition for distribution-free learnability is finiteness of the Vapnik-Chervonenkis dimension, a simple combinatorial parameter of the class of concepts to be learned.Abstract:
Valiant's learnability model is extended to learning classes of concepts defined by regions in Euclidean space En. The methods in this paper lead to a unified treatment of some of Valiant's results, along with previous results on distribution-free convergence of certain pattern recognition algorithms. It is shown that the essential condition for distribution-free learnability is finiteness of the Vapnik-Chervonenkis dimension, a simple combinatorial parameter of the class of concepts to be learned. Using this parameter, the complexity and closure properties of learnable classes are analyzed, and the necessary and sufficient conditions are provided for feasible learnability.read more
Citations
More filters
Proceedings Article
An Almost Optimal PAC Algorithm
TL;DR: It is shown that every consistent algorithm L (even a provably suboptimal one) induces a family of PAC algorithms which come very close to optimality: the number of labeled examples needed by LK exceeds the general lower bound only by factor ‘K(1= ) where ’K denotes (a truncated version of) the K-times iterated logarithm.
Proceedings ArticleDOI
Compressing and Teaching for Low VC-Dimension
TL;DR: This work shows that given an arbitrary set of labeled examples from an unknown concept in C, one can retain only a subset of exp(d) of them, in a way that allows to recover the labels of all other examples in the set, using additional exp( d) information bits.
Posted Content
Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets?
TL;DR: This work describes a natural task on which a provable sample complexity gap can be shown, for standard training algorithms, and demonstrates a single target function, learning which on all possible distributions leads to an $O(1)$ vs $Omega(d^2/\varepsilon)$ gap.
Proceedings Article
Algorithmic stability and uniform generalization
TL;DR: This paper proves that algorithmic stability in the inference process is equivalent to uniform generalization across all parametric loss functions, and establishes a relationship between algorithmic Stability and the size of the observation space, which provides a formal justification for dimensionality reduction methods.
Posted Content
A Theory of Universal Learning.
TL;DR: There are only three possible rates of universal learning, which aims to understand the performance of learning algorithms on every data distribution, but without requiring uniformity over the distribution: exponential, linear, or arbitrarily slow rates.
References
More filters
Book
Computers and Intractability: A Guide to the Theory of NP-Completeness
TL;DR: The second edition of a quarterly column as discussed by the authors provides a continuing update to the list of problems (NP-complete and harder) presented by M. R. Garey and myself in our book "Computers and Intractability: A Guide to the Theory of NP-Completeness,” W. H. Freeman & Co., San Francisco, 1979.
Book
The Art of Computer Programming
TL;DR: The arrangement of this invention provides a strong vibration free hold-down mechanism while avoiding a large pressure drop to the flow of coolant fluid.
Journal ArticleDOI
Pattern Classification and Scene Analysis.
Book
Pattern classification and scene analysis
Richard O. Duda,Peter E. Hart +1 more
TL;DR: In this article, a unified, comprehensive and up-to-date treatment of both statistical and descriptive methods for pattern recognition is provided, including Bayesian decision theory, supervised and unsupervised learning, nonparametric techniques, discriminant analysis, clustering, preprosessing of pictorial data, spatial filtering, shape description techniques, perspective transformations, projective invariants, linguistic procedures, and artificial intelligence techniques for scene analysis.