Learnability and the Vapnik-Chervonenkis dimension
TL;DR: This paper shows that the essential condition for distribution-free learnability is finiteness of the Vapnik-Chervonenkis dimension, a simple combinatorial parameter of the class of concepts to be learned.
Abstract: Valiant's learnability model is extended to learning classes of concepts defined by regions in Euclidean space En. The methods in this paper lead to a unified treatment of some of Valiant's results, along with previous results on distribution-free convergence of certain pattern recognition algorithms. It is shown that the essential condition for distribution-free learnability is finiteness of the Vapnik-Chervonenkis dimension, a simple combinatorial parameter of the class of concepts to be learned. Using this parameter, the complexity and closure properties of learnable classes are analyzed, and the necessary and sufficient conditions are provided for feasible learnability.
Citations
More filters
TL;DR: This paper investigates the problem of classifying objects which are given by feature vectors with Boolean entries by obtaining upper bounds for the required sample size which are small polynomials in the relevant parameters and which match the lower bounds known for these classes.
Abstract: In this paper, we investigate the problem of classifying objects which are given by feature vectors with Boolean entries. Our aim is to “(efficiently) learn probably almost optimal classifications” from examples. A classical approach in pattern recognition uses empirical estimations of the Bayesian discriminant functions for this purpose. We analyze this approach for different classes of distribution functions of Boolean features:kth order BahadurÂ?Lazarsfeld expansions andkth order Chow expansions. In both cases, we obtain upper bounds for the required sample size which are small polynomials in the relevant parameters and which match the lower bounds known for these classes. Moreover, the learning algorithms are efficient.
11 citations
01 Sep 1993
TL;DR: Results are derived that can be used to determine the accuracy of an induced concept when used to classify the original domain or another close domain.
Abstract: Inductive learning methods identify a concept from a training sample consisting of positive and negative examples of a target concept. Several studies have shown how such methods could be used to determine rules for expert systems. The question addressed in this paper is: how accurate is the induced concept when used to classify the original domain or another close domain? We derive results that can be used to determine the accuracy of an induced concept. Two previously published applications of inductive learning are used to illustrate our results.
11 citations
Cites background from "Learnability and the Vapnik-Chervon..."
...Recent work in machine learning has focused on establishing a theoretical foundation for learnability [4,7,9,23]....
[...]
...Bounds [4, 9] (1) For any given • and S, with 0 ~< E, 6 ~< 1, if the size of Q is at least...
[...]
TL;DR: A phenotype prediction tool helps 'fill in the blanks' for expression microarrays, extending their predictive power and uncovering once-hidden biases.
Abstract: A phenotype prediction tool helps 'fill in the blanks' for expression microarrays, extending their predictive power and uncovering once-hidden biases.
11 citations
Cites result from "Learnability and the Vapnik-Chervon..."
...From known PAC results [Blumer et al., 1989], the class of linear threshold formulas is learnable in the sense above....
[...]
01 Jun 1997
TL;DR: This work improves error bounds based on VC analysis for classes with sets of similar classifiers and applies the new error bounds to separating planes and artificial neural networks.
Abstract: We improve error bounds based on VC analysis for classes with sets of similar classifiers. We apply the new error bounds to separating planes and artificial neural networks.
11 citations
TL;DR: Valiant’s learning from examples model which formalizes the problem of generalization is presented and open questions are mentioned.
11 citations
References
More filters
01 Jan 1979
42,654 citations
Book•
01 Jan 1979
TL;DR: The second edition of a quarterly column as discussed by the authors provides a continuing update to the list of problems (NP-complete and harder) presented by M. R. Garey and myself in our book "Computers and Intractability: A Guide to the Theory of NP-Completeness,” W. H. Freeman & Co., San Francisco, 1979.
Abstract: This is the second edition of a quarterly column the purpose of which is to provide a continuing update to the list of problems (NP-complete and harder) presented by M. R. Garey and myself in our book ‘‘Computers and Intractability: A Guide to the Theory of NP-Completeness,’’ W. H. Freeman & Co., San Francisco, 1979 (hereinafter referred to as ‘‘[G&J]’’; previous columns will be referred to by their dates). A background equivalent to that provided by [G&J] is assumed. Readers having results they would like mentioned (NP-hardness, PSPACE-hardness, polynomial-time-solvability, etc.), or open problems they would like publicized, should send them to David S. Johnson, Room 2C355, Bell Laboratories, Murray Hill, NJ 07974, including details, or at least sketches, of any new proofs (full papers are preferred). In the case of unpublished results, please state explicitly that you would like the results mentioned in the column. Comments and corrections are also welcome. For more details on the nature of the column and the form of desired submissions, see the December 1981 issue of this journal.
40,020 citations
Book•
01 Jan 1968
TL;DR: The arrangement of this invention provides a strong vibration free hold-down mechanism while avoiding a large pressure drop to the flow of coolant fluid.
Abstract: A fuel pin hold-down and spacing apparatus for use in nuclear reactors is disclosed. Fuel pins forming a hexagonal array are spaced apart from each other and held-down at their lower end, securely attached at two places along their length to one of a plurality of vertically disposed parallel plates arranged in horizontally spaced rows. These plates are in turn spaced apart from each other and held together by a combination of spacing and fastening means. The arrangement of this invention provides a strong vibration free hold-down mechanism while avoiding a large pressure drop to the flow of coolant fluid. This apparatus is particularly useful in connection with liquid cooled reactors such as liquid metal cooled fast breeder reactors.
17,939 citations
14,948 citations
Book•
01 Jan 1973
TL;DR: In this article, a unified, comprehensive and up-to-date treatment of both statistical and descriptive methods for pattern recognition is provided, including Bayesian decision theory, supervised and unsupervised learning, nonparametric techniques, discriminant analysis, clustering, preprosessing of pictorial data, spatial filtering, shape description techniques, perspective transformations, projective invariants, linguistic procedures, and artificial intelligence techniques for scene analysis.
Abstract: Provides a unified, comprehensive and up-to-date treatment of both statistical and descriptive methods for pattern recognition. The topics treated include Bayesian decision theory, supervised and unsupervised learning, nonparametric techniques, discriminant analysis, clustering, preprosessing of pictorial data, spatial filtering, shape description techniques, perspective transformations, projective invariants, linguistic procedures, and artificial intelligence techniques for scene analysis.
13,647 citations