Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

Fast and frugal heuristics - simple rules for making decisions with realistic mental resources - are presented here. These heuristics can enable both living organisms and artificial systems to make smart choices, classifications, and predictions by employing bounded rationality. But when and how can such fast and frugal heuristics work? What heuristics are in the mind's adaptive toolbox, and what building blocks compose them? Can judgments based simply on a single reason be as accurate as those based on many reasons? Could less knowledge even lead to systematically better predictions than more knowledge? This book explores these questions by developing computational models of heuristics and testing them through experiments and analysis. It shows how fast and frugal heuristics can yield adaptive decisions in situations as varied as choosing a mate, dividing resources among offspring, predicting high school drop-out rates, and playing the stock market.

Simple Heuristics That Make Us Smart

The study of decision making spans such varied fields as neuroscience, psychology, economics, statistics, political science, and computer science. Despite this diversity of applications, most decisions share common elements including deliberation and commitment. Here we evaluate recent progress in understanding how these basic elements of decision formation are implemented in the brain. We focus on simple decisions that can be studied in the laboratory but emphasize general principles likely to extend to other settings.

/pdf/the-neural-basis-of-decision-making-2p5agsty56.pdf

The Neural Basis of Decision Making

Foreword by Norbert Hornstein Preface Part I 1 Mind and Body 2 Structures, Capacities, and Conventions 3 Knowledge of Grammar 4 Some Elements of Grammar Part II 5 On the Biological Basis of Language Capacities 6 Language and Unconscious Knowledge Notes Index

https://www.cambridge.org/core/services/aop-cambridge-core/content/view/BB96E4E09C461EFC230F72A9D7BDF603/S0140525X00001515a.pdf/div-class-title-rules-and-representations-div.pdf

Rules and Representations

Beginning with Emlen (1966) and MacArthur and Pianka (1966) and extending through the last ten years, several authors have sought to predict the foraging behavior of animals by means of mathematical models. These models are very similar,in that they all assume that the fitness of a foraging animal is a function of the efficiency of foraging measured in terms of some "currency" (Schoener, 1971) -usually energy- and that natural selection has resulted in animals that forage so as to maximize this fitness. As a result of these similarities, the models have become known as "optimal foraging models"; and the theory that embodies them, "optimal foraging theory." The situations to which optimal foraging theory has been applied, with the exception of a few recent studies, can be divided into the following four categories: (1) choice by an animal of which food types to eat (i.e., optimal diet); (2) choice of which patch type to feed in (i.e., optimal patch choice); (3) optimal allocation of time to different patches; and (4) optimal patterns and speed of movements. In this review we discuss each of these categories separately, dealing with both the theoretical developments and the data that permit tests of the predictions. The review is selective in the sense that we emphasize studies that either develop testable predictions or that attempt to test predictions in a precise quantitative manner. We also discuss what we see to be some of the future developments in the area of optimal foraging theory and how this theory can be related to other areas of biology. Our general conclusion is that the simple models so far formulated are supported are supported reasonably well by available data and that we are optimistic about the value both now and in the future of optimal foraging theory. We argue, however, that these simple models will requre much modification, espicially to deal with situations that either cannot easily be put into one or another of the above four categories or entail currencies more complicated that just energy.

Citation classic - optimal foraging - a selective review of theory and tests

Replication and extension of Skinner's "superstition" experiment showed the development of two kinds of behavior at asymptote: interim activities (related to adjunctive behavior) occurred just after food delivery; the terminal response (a discriminated operant) occurred toward the end of the interval and continued until food delivery. These data suggest a view of operant conditioning (the terminal response) in terms of two sets of principles: principles of behavioral variation that describe the origins of behavior "appropriate" to a situation, in advance of reinforcement; and principles of reinforcement that describe the selective elimination of behavior so produced. This approach was supported by (a) an account of the parallels between the Law of Effect and evolution by means of natural selection, (fc) its ability to shed light on persistent problems in learning (e.g., continuity vs. noncontinuity, variability associated with extinction, the relationship between classical and instrumental conditioning, the controversy between behaviorist and cognitive approaches to learning), and (c) its ability to deal with a number of recent anomalies in the learning literature ("instinctive drift," auto-shaping, and auto-maintenance). The interim activities were interpreted in terms of interactions among motivational systems, and this view was supported by a review of the literature on adjunctive behavior and by comparison with similar phenomena in ethology (displacement, redirection, and "vacuum" activities). The proposed theoretical scheme represents a shift away from hypothetical "laws of learning" toward an interpretation of behavioral change in terms of interaction and competition among tendencies to action according to principles evolved in phylogeny.

The "supersitition" experiment: A reexamination of its implications for the principles of adaptive behavior.

Handbook of operant behavior

Handbook of Operant Behavior

Preface 1. The evolution, development and modification of behaviour 2. Variation and selection of behaviour 3. Direct orientation and feedback 4. Operant behaviour 5. Reward and punishment 6. Feeding regulation: a model motivational system 7. The optimal allocation of behaviour 8. Choice and decision rules 9. Foraging and behavioural ecology 10. Stimulus control and cognition 11. Stimulus control and performance 12. Memory and temporal control 13. Learning I: the acquisition of knowledge 14. Learning II: the guidance of action References Indexes.

John Staddon

Papers

The "supersitition" experiment: A reexamination of its implications for the principles of adaptive behavior.

Handbook of operant behavior

Handbook of Operant Behavior

Adaptive behavior and learning

Limits to action, the allocation of individual behavior