There has been a great deal of theoretical and experimental work in computer science on inductive inference systems, that is, systems that try to infer general rules from examples. However, a complete and applicable theory of such systems is still a distant goal. This survey highlights and explains the main ideas that have been developed in the study of inductive inference, with special emphasis on the relations between the general theory and the specific algorithms and implementations. 154 references.

Inductive Inference: Theory and Methods

The computational complexity of learning Boolean concepts from examples is investigated. It is shown for various classes of concept representations that these cannot be learned feasibly in a distribution-free sense unless R = NP. These classes include (a) disjunctions of two monomials, (b) Boolean threshold functions, and (c) Boolean formulas in which each variable occurs at most once. Relationships between learning of heuristics and finding approximate solutions to NP-hard optimization problems are given.

Computational limitations on learning from examples

This paper surveys recent results concerning the inference of deterministic finite automata (DFAs). The results discussed determine the extent to which DFAs can be feasibly inferred, and highlight a number of interesting approaches in computational learning theory.

/pdf/inductive-inference-dfas-and-computational-complexity-32lzzv3wea.pdf

Inductive Inference, DFAs, and Computational Complexity

Permission to copy without fee all or part of this material ie granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notica is given that copying is by permission of the Association for Computing Machinery To copy otherwise, or to republish, requires a fee and/or specific permission 24th ANNUAL ACM STOC 5/92/VICTORiA, B C, CANADA G 1992 ACM ()-8979151 2-7/92/0004/03~j -$1 ~() the only textbook in the field is Natarajan’s [101] Surveys by Laird [83] and Valiant [129] are valuable Somewhat more peripheral are the European meetings on Analogical and Inductive Inference, AH, and the AI machine learning communit y’s annual International Conference on Machine Learning In addition, the general AI meetings, AAAI and IJCAI, currently have a large number of papers devoted to learning, as do the neural net meetings

/pdf/computational-learning-theory-survey-and-selected-18y4duhcgf.pdf

Computational learning theory: survey and selected bibliography

We show that for some classes ℒ of recursive languages, from the characteristic function of any L in ℒ an approximate decision procedure for L with no more than n+1 mistakes can be (uniformly effectively) inferred in the limit; whereas, in general, a grammar (generation procedure) with no more than n mistakes cannot; for some classes an infinite sequence of perfectly correct decision procedures can be inferred in the limit, but single grammars with finitely many mistakes cannot; and for some classes an infinite sequence of decision procedures each with no more than n+1 mistakes can be inferred, but an infinite sequence of grammars each with no more than n mistakes cannot. This is true even though decision procedures generally contain more information than grammars. We also consider inference of grammars for r.e. languages from arbitrary texts, i.e., enumerations of the languages. We show that for any class of languages ℒ, if some, machine, from arbitrary texts for any L in ℒ, can infer in the limit an approximate grammar for L with no more than 2·n mistakes, then some machine can infer in the limit, for each language in ℒ, an infinite sequence of grammars each with no more than n mistakes. This reduction from 2·n to n is best possible. From these and other results we obtain and compare several natural, inference hierarchies. Lastly we show that if we restrict ourselves to recursive texts, there is a machine which, for any r.e. language, infers in the limit an infinite sequence of grammars each with only finitely many mistakes. We employ recursion theoretic methods including infinitary and ordinary recursion theorems.

Machine Inductive Inference and Language Identification

Comparison of identification criteria for machine inductive inference

Some results concerning inductive inference are surveyed. These results are interpreted with respect to automatic program synthesis, a special case of algorithmic inductive iruerence. The interpretations reinforce and refine opinions concerning automatic program synthesis, and artificial intelligence in general, which have been previously expressed in [9] and [15]. The final section digresses from science to speculation as the interpretations of the theorems surveyed are extended to differentiate various cognitive styles of learning. 1. Supported by NSF grant MCS-790S912

/pdf/a-general-theory-of-automatic-program-synthesis-1935dntcii.pdf

Carl H. Smith

Papers

Comparison of identification criteria for machine inductive inference

A General Theory of Automatic Program Synthesis