Showing papers by "Thomas G. Dietterich published in 1989"
TL;DR: An empirical study evaluates three methods for solving the problem of identifying a correct concept definition from positive examples such that the concept is some specialization of a target concept defined by a domain theory, and concludes that the new method, IOE, does not exhibit these shortcomings.
Abstract: This paper formalizes a new learning-from-examples problem: identifying a correct concept definition from positive examples such that the concept is some specialization of a target concept defined by a domain theory. It describes an empirical study that evaluates three methods for solving this problem: explanation-based generalization (EBG), multiple example explanation-based generalization (mEBG), and a new method, induction over explanations (IOE). The study demonstrates that the two existing methods (EBG and mEBG) exhibit two shortcomings: (a) they rarely identify the correct definition, and (b) they are brittle in that their success depends greatly on the choice of encoding of the domain theory rules. The study demonstrates that the new method, IOE, does not exhibit these shortcomings. This method applies the domain theory to construct explanations from multiple training examples as in mEBG, but forms the concept definition by employing a similarity-based generalization policy over the explanations. IOE has the advantage that an explicit domain theory can be exploited to aid the learning process, the dependence on the initial encoding of the domain theory is significantly reduced, and the correct concepts can be learned from few examples. The study evaluates the methods in the context of an implemented system, called Wyl2, which learns a variety of concepts in chess including “skewer” and “knight-fork.”
144 citations
01 Dec 1989
TL;DR: In this paper, the authors show that inductive learning from examples is fundamentally limited to learning only a small fraction of the total space of possible hypotheses, and prove an upper bound on the maximum number of concepts reliably learnable from m training examples.
Abstract: This paper explores the proposition that inductive learning from examples is fundamentally limited to learning only a small fraction of the total space of possible hypotheses. We begin by defining the notion of an algorithm reliably learning a good approximation to a concept C. An empirical study of three algorithms (the classical algorithm for maximally specific conjunctive generalizations, ID3, and back-propagation for feed-forward networks of logistic units) demonstrates that each of these algorithms performs very poorly for the task of learning concepts defined over the space of Boolean feature vectors containing 3 variables. Simple counting arguments allow us to prove an upper bound on the maximum number of concepts reliably learnable from m training examples.
38 citations
Book•
01 Jan 1989
TL;DR: Proceedings of the annual Conferences on Advances in Neural Information Processing Systems show progress in several areas, including machine learning, reinforcement learning, and more.
Abstract: Proceedings of the annual Conferences on Advances in Neural Information Processing Systems. Volumes 7 and later are available from The MIT Press.
26 citations
01 Jan 1989
TL;DR: The study demonstrates that the new method, IOE, has the advantage that an explicit domain theory can be exploited to aid the learning process, the dependence on the initial encoding of the domain theory is significantly reduced, and the correct concepts can be learned from few examples.
Abstract: This paper formalizes a new learning-from-examples problem: identifying a correct concept definition from positive examples such that the concept is some specialization of a target concept defined by a domain theory. It describes an empirical study that evaluates three methods for solving this problem: explanation-based generalization (EBG), multiple example explanation-based generalization (mEBG), and a new method, induction over explana- tions (IOE). The study demonstrates that the two existing methods (EBG and mEBG) exhibit two shortcomings: (a) they rarely identify the correct definition, and (b) they are brittle in that their success depends greatly on the choice of encoding of the domain theory rules. The study demonstrates that the new method, IOE, does not exhibit these shortcomings. This method applies the domain theory to construct explanations from multiple train- ing examples as in mEBG, but forms the concept definition by employing a similarity-based generalization policy over the explanations. IOE has the advantage that an explicit domain theory can be exploited to aid the learning process, the dependence on the initial encoding of the domain theory is significantly reduced, and the correct concepts can be learned from few examples. The study evaluates the methods in the context of an implemented system, called Wyl2, which learns a variety of concepts in chess including "skewer" and "knight-fork."
16 citations
01 Dec 1989
TL;DR: The most effective experimental strategy tried was a (very expensive) greedy algorithm that attempts to find an experiment that “splits the hypothesis space in half,” but a much cheaper strategy–that selects any experiment guaranteeing the elimination of at least one hypothesis from the set being considered–was found to be almost as effective.
Abstract: One way to study an unknown system is to perform experiments on it. How does the ability to control the inputs to a system affect the number of experiments (input/output pairs) needed to determine that system's function? How can a clever experiment selection strategy affect this number? An empirical study was performed using Boolean truth tables as hypotheses. Computer programs were constructed to model several different experimentation strategies. The average number of experiments needed to determine a target theory from a set of hypotheses was measured. Results demonstrated that control of the system inputs gave the most dramatic increase in performance. The most effective experimental strategy tried was a (very expensive) greedy algorithm that attempts to find an experiment that “splits the hypothesis space in half.”However, a much cheaper strategy–that selects any experiment guaranteeing the elimination of at least one hypothesis from the set being considered–was found to be almost as effective.
9 citations