Apprenticeship learning

About: Apprenticeship learning is a research topic. Over the lifetime, 266 publications have been published within this topic receiving 50546 citations.

Reinforcement Learning: An Introduction

Abstract: Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

Cognitive Apprenticeship: Teaching the Craft of Reading, Writing and Mathematics

Abstract: : Even today, many complex and important skills, such as those required for language use and social interaction, are learned informally through apprenticeshiplike methods -- i.e., methods involving not didactic teaching, but observation, coaching, and successive approximation while carrying out a variety of tasks and activities. The differences between formal schooling and apprenticeship methods are many, but for our purposes, one is most important. Perhaps as a by-product of the specialization of learning in schools, skills and knowledge taught in schools have become abstracted from their uses in the world. In apprenticeship learning, on the other hand, target skills are not only continually in use by skilled practitioners, but are instrumental to the accomplishment of meaningful tasks. Said differently, apprenticeship embeds the learning of skills and knowledge in the social and functional context of their use. This difference is not academic, but has serious implications for the nature of the knowledge that students acquire. This paper attempts to elucidate some of those implications through a proposal for the retooling of apprenticeship methods for the teaching and learning of cognitive skills. Specifically, we propose the development of a new cognitive apprenticeship to teach students the thinking and problem-solving skills involved in school subjects such as reading, writing and mathematics.

Apprenticeship learning via inverse reinforcement learning

Abstract: We consider learning in a Markov decision process where we are not explicitly given a reward function, but where instead we can observe an expert demonstrating the task that we want to learn to perform. This setting is useful in applications (such as the task of driving) where it may be difficult to write down an explicit reward function specifying exactly how different desiderata should be traded off. We think of the expert as trying to maximize a reward function that is expressible as a linear combination of known features, and give an algorithm for learning the task demonstrated by the expert. Our algorithm is based on using "inverse reinforcement learning" to try to recover the unknown reward function. We show that our algorithm terminates in a small number of iterations, and that even though we may never recover the expert's reward function, the policy output by the algorithm will attain performance close to that of the expert, where here performance is measured with respect to the expert's unknown reward function.

Maximum entropy inverse reinforcement learning

Abstract: Recent research has shown the benefit of framing problems of imitation learning as solutions to Markov Decision Problems. This approach reduces learning to the problem of recovering a utility function that makes the behavior induced by a near-optimal policy closely mimic demonstrated behavior. In this work, we develop a probabilistic approach based on the principle of maximum entropy. Our approach provides a well-defined, globally normalized distribution over decision sequences, while providing the same performance guarantees as existing methods. We develop our technique in the context of modeling real-world navigation and driving behaviors where collected data is inherently noisy and imperfect. Our probabilistic approach enables modeling of route preferences as well as a powerful new approach to inferring destinations and routes based on partial trajectories.

Robot Programming by Demonstration

Abstract: Also referred to as learning by imitation, tutelage, or apprenticeship learning, Programming by Demonstration (PbD) develops methods by which new skills can be transmitted to a robot. This book examines methods by which robots learn new skills through human guidance. Taking a practical perspective, it covers a broad range of applications, including service robots. The text addresses the challenges involved in investigating methods by which PbD is used to provide robots with a generic and adaptive model of control. Drawing on findings from robot control, human-robot interaction, applied machine learning, artificial intelligence, and developmental and cognitive psychology, the book contains a large set of didactic and illustrative examples. Practical and comprehensive machine learning source codes are available on the books companion website: http://www.programming-by-demonstration.org