scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Learnability and the Vapnik-Chervonenkis dimension

TL;DR: This paper shows that the essential condition for distribution-free learnability is finiteness of the Vapnik-Chervonenkis dimension, a simple combinatorial parameter of the class of concepts to be learned.
Abstract: Valiant's learnability model is extended to learning classes of concepts defined by regions in Euclidean space En. The methods in this paper lead to a unified treatment of some of Valiant's results, along with previous results on distribution-free convergence of certain pattern recognition algorithms. It is shown that the essential condition for distribution-free learnability is finiteness of the Vapnik-Chervonenkis dimension, a simple combinatorial parameter of the class of concepts to be learned. Using this parameter, the complexity and closure properties of learnable classes are analyzed, and the necessary and sufficient conditions are provided for feasible learnability.

Content maybe subject to copyright    Report

Citations
More filters
Proceedings ArticleDOI
03 Nov 1993
TL;DR: The algorithm can learn an intersection of an arbitrary (polynomial) number of halfspaces over this distribution, if the subspace spanned by the normal vectors to the bounding hyperplanes has constant dimension.
Abstract: We present a polynomial-time algorithm to learn an intersection of a constant number of halfspaces in n dimensions, over the uniform distribution on an n-dimensional ball. The algorithm we present in fact can learn an intersection of an arbitrary (polynomial) number of halfspaces over this distribution, if the subspace spanned by the normal vectors to the bounding hyperplanes has constant dimension. This generalizes previous results for this distribution, in particular a result of E.B. Baum (1990) who showed how to learn an intersection of 2 halfspaces defined by hyperplanes that pass through the origin (his results in fact held for a variety of symmetric distributions). Our algorithm uses estimates of second moments to find vectors in a low-dimensional "relevant subspace". We believe that the algorithmic techniques studied here may be useful in other geometric learning applications. >

32 citations

Journal ArticleDOI
TL;DR: The philosophical innateness debate has long relied onpsychological evidence as discussed by the authors, however, a parallel debate hastaken place within neuroscience, and it is argued that the majority of natural systems are in fact innate.
Abstract: The philosophical innateness debate has long relied onpsychological evidence. For a century, however, a parallel debate hastaken place within neuroscience. In this paper, I consider theimplications of this neuroscience debate for the philosophicalinnateness debate. By combining the tools of theoretical neurobiologyand learning theory, I introduce the ``problem of development'' that alladaptive systems must solve, and suggest how responses to this problemcan demarcate a number of innateness proposals. From this perspective, Isuggest that the majority of natural systems are in fact innate. Lastly,I consider the acquistion strategies implemented by the human brain andsuggest that there is a rigorous way of characterizing these ``neuralconstructivist'' strategies as not being strongly innate. Alternatives toinnateness are thus both rigorously definable and empirically supported.

32 citations

Proceedings ArticleDOI
16 Jul 1994
TL;DR: This work improves on Aizenstein, Hellerstein, and Pitt's negative result for proper learning in the exact model and shows that read-thrice DNF formulas are not properly learnable in the extended PAC model, assuming RP.
Abstract: Bshouty, Goldman, Hancock and Matar have shown that up to log n-term DNF formulas can be properly learned in the exact model with equivalence and membership queries Given standard complexity-theoretical assumptions, we show that this positive result for proper learning cannot be significantly improved in the exact model or the PAC model extended to allow membership queries Our negative results are derived from two general techniques for proving such results in the exact model and the extended PAC model As a further application of these techniques, we consider read-thrice DNF formulas Here we improve on Aizenstein, Hellerstein, and Pitt's negative result for proper learning in the exact model in two ways First, we show that their assumption of NP ≠ co-NP can be replaced with the weaker assumption of P ≠ NP Second, we show that read-thrice DNF formulas are not properly learnable in the extended PAC model, assuming RP ≠ NP

32 citations


Cites methods from "Learnability and the Vapnik-Chervon..."

  • ...when the learner’s hypotheses are restricted to formulas from the subclass (proper learnability [ BEHW89 ])? A number of positive and negative results have been obtained with various restrictions on DNF formulas....

    [...]

Posted Content
TL;DR: A new algorithm is proposed that uses decision trees to learn candidate invariants in the form of arbitrary Boolean combinations of numerical inequalities in order to infer safe invariants for a range of challenging benchmarks and compares favorably to other ML-based invariant inference techniques.
Abstract: The problem of inferring an inductive invariant for verifying program safety can be formulated in terms of binary classification. This is a standard problem in machine learning: given a sample of good and bad points, one is asked to find a classifier that generalizes from the sample and separates the two sets. Here, the good points are the reachable states of the program, and the bad points are those that reach a safety property violation. Thus, a learned classifier is a candidate invariant. In this paper, we propose a new algorithm that uses decision trees to learn candidate invariants in the form of arbitrary Boolean combinations of numerical inequalities. We have used our algorithm to verify C programs taken from the literature. The algorithm is able to infer safe invariants for a range of challenging benchmarks and compares favorably to other ML-based invariant inference techniques. In particular, it scales well to large sample sets.

32 citations

Journal ArticleDOI
TL;DR: The notion of dynamic sampling is introduced, wherein the number of examples examined may increase with the complexity of the target concept, and this method is used to establish the learnability of various concept classes with an infinite Vapnik-Chervonenkis dimension.
Abstract: We consider the problem of learning a concept from examples in the distribution-free model by Valiant. (An essentially equivalent model, if one ignores issues of computational difficulty, was studied by Vapnik and Chervonenkis.) We introduce the notion of dynamic sampling, wherein the number of examples examined may increase with the complexity of the target concept. This method is used to establish the learnability of various concept classes with an infinite Vapnik-Chervonenkis dimension. We also discuss an important variation on the problem of learning from examples, called approximating from examples. Here we do not assume that the target concept T is a member of the concept class C from which approximations are chosen. This problem takes on particular interest when the VC dimension of C is infinite. Finally, we discuss the problem of computing the VC dimension of a finite concept set defined on a finite domain and consider the structure of classes of a fixed small dimension.

32 citations

References
More filters
Book
01 Jan 1979
TL;DR: The second edition of a quarterly column as discussed by the authors provides a continuing update to the list of problems (NP-complete and harder) presented by M. R. Garey and myself in our book "Computers and Intractability: A Guide to the Theory of NP-Completeness,” W. H. Freeman & Co., San Francisco, 1979.
Abstract: This is the second edition of a quarterly column the purpose of which is to provide a continuing update to the list of problems (NP-complete and harder) presented by M. R. Garey and myself in our book ‘‘Computers and Intractability: A Guide to the Theory of NP-Completeness,’’ W. H. Freeman & Co., San Francisco, 1979 (hereinafter referred to as ‘‘[G&J]’’; previous columns will be referred to by their dates). A background equivalent to that provided by [G&J] is assumed. Readers having results they would like mentioned (NP-hardness, PSPACE-hardness, polynomial-time-solvability, etc.), or open problems they would like publicized, should send them to David S. Johnson, Room 2C355, Bell Laboratories, Murray Hill, NJ 07974, including details, or at least sketches, of any new proofs (full papers are preferred). In the case of unpublished results, please state explicitly that you would like the results mentioned in the column. Comments and corrections are also welcome. For more details on the nature of the column and the form of desired submissions, see the December 1981 issue of this journal.

40,020 citations

Book
01 Jan 1968
TL;DR: The arrangement of this invention provides a strong vibration free hold-down mechanism while avoiding a large pressure drop to the flow of coolant fluid.
Abstract: A fuel pin hold-down and spacing apparatus for use in nuclear reactors is disclosed. Fuel pins forming a hexagonal array are spaced apart from each other and held-down at their lower end, securely attached at two places along their length to one of a plurality of vertically disposed parallel plates arranged in horizontally spaced rows. These plates are in turn spaced apart from each other and held together by a combination of spacing and fastening means. The arrangement of this invention provides a strong vibration free hold-down mechanism while avoiding a large pressure drop to the flow of coolant fluid. This apparatus is particularly useful in connection with liquid cooled reactors such as liquid metal cooled fast breeder reactors.

17,939 citations

Book
01 Jan 1973
TL;DR: In this article, a unified, comprehensive and up-to-date treatment of both statistical and descriptive methods for pattern recognition is provided, including Bayesian decision theory, supervised and unsupervised learning, nonparametric techniques, discriminant analysis, clustering, preprosessing of pictorial data, spatial filtering, shape description techniques, perspective transformations, projective invariants, linguistic procedures, and artificial intelligence techniques for scene analysis.
Abstract: Provides a unified, comprehensive and up-to-date treatment of both statistical and descriptive methods for pattern recognition. The topics treated include Bayesian decision theory, supervised and unsupervised learning, nonparametric techniques, discriminant analysis, clustering, preprosessing of pictorial data, spatial filtering, shape description techniques, perspective transformations, projective invariants, linguistic procedures, and artificial intelligence techniques for scene analysis.

13,647 citations