Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

Attention and Effort

The capacity to predict future events permits a creature to detect, model, and manipulate the causal structure of its interactions with its environment. Behavioral experiments suggest that learning is driven by changes in the expectations about future salient events such as rewards and punishments. Physiological work has recently complemented these studies by identifying dopaminergic neurons in the primate whose fluctuating output apparently signals changes or errors in the predictions of future salient and rewarding events. Taken together, these findings can be understood through quantitative theories of adaptive optimizing control.

/pdf/a-neural-substrate-of-prediction-and-reward-2tj1zqgoih.pdf

A Neural Substrate of Prediction and Reward

Tested the 2-process theory of detection, search, and attention presented by the current authors (1977) in a series of experiments. The studies (a) demonstrate the qualitative difference between 2 modes of information processing: automatic detection and controlled search; (b) trace the course of the

http://psycnet.apa.org/doi/10.1037/0033-295X.84.2.127

Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory.

A model of the neuropsychology of anxiety is proposed. The model is based in the first instance upon an analysis of the behavioural effects of the antianxiety drugs (benzodiazepines, barbiturates, and alcohol) in animals. From such psychopharmacologi-cal experiments the concept of a “behavioural inhibition system” (BIS) has been developed. This system responds to novel stimuli or to those associated with punishment or nonreward by inhibiting ongoing behaviour and increasing arousal and attention to the environment. It is activity in the BIS that constitutes anxiety and that is reduced by antianxiety drugs. The effects of the antianxiety drugs in the brain also suggest hypotheses concerning the neural substrate of anxiety. Although the benzodiazepines and barbiturates facilitate the effects of γ-aminobutyrate, this is insufficient to explain their highly specific behavioural effects. Because of similarities between the behavioural effects of certain lesions and those of the antianxiety drugs, it is proposed that these drugs reduce anxiety by impairing the functioning of a widespread neural system including the septo-hippocampal system (SHS), the Papez circuit, the prefrontal cortex, and ascending monoaminergic and cholinergic pathways which innervate these forebrain structures. Analysis of the functions of this system (based on anatomical, physiological, and behavioural data) suggests that it acts as a comparator: it compares predicted to actual sensory events and activates the outputs of the BIS when there is a mismatch or when the predicted event is aversive. Suggestions are made as to the functions of particular pathways within this overall brain system. The resulting theory is applied to the symptoms and treatment of anxiety in man, its relations to depression, and the personality of individuals who are susceptible to anxiety or depression.

https://www.cambridge.org/core/services/aop-cambridge-core/content/view/S0140525X00013066

Précis of The Neuropsychology of Anxiety: An Enquiry into the Functions of the Septo-Hippocampal System..

According to theories of selective attention, learning about a stimulus depends on attending to that stimulus; this is represented in two-stage models by saying that subjects switch in analyzers as well as learning stimulusresponse associations This assumption, however, is equally well represented in a formal model by the incorporation of a stimulus-specific learning-rate parameter, a, into the equations describing changes in the associative strength of stimuli Theories of selective attention have also assumed (a) that subjects learn to attend to and ignore relevant and irrelevant stimuli (ie, that a may increase or decrease depending on the correlation of a stimulus with reinforcement) and (b) that there is an inverse relationship between the probabilities of attending to different stimuli (ie, that an increase in a to one stimulus is accompanied by a decrease in a to others) The first assumption is used to explain the phenomena of acquired distinctiveness and dimensional transfer, the second those of overshadowing and blocking Although the first assumption is justified by the data, the second is not: Overshadowing and blocking are better explained by the choice of an appropriate rule for changing a, such that a decreases to stimuli that signal no change from the probability of reinforcement predicted by other stimuli

A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement

The psychology of animal learning

The study of conditioning in animals Classical and instrumental conditioning Theoretical analysis of classical conditioning Theoretical analysis of instrumental conditioning Appetitive and aversive reinforcement Avoidance learning Contiguity and contingency: excitatory and inhibitory conditioning Laws of association Discrimination learning References Indexes.

Conditioning and associative learning

Mechanisms of animal discrimination learning

1. The early development and uses of IQ tests 2. Psychometric theories of intelligence 3. The search for cognitive processes underlying components of IQ: Gs or speed and efficiency of information processing 4. Verbal, spatial and fluid abilities: Gc, Gv and Gf 5. Associative learning, working memory and executive control 6. Intelligence and the brain 7. Theories of g 8. The stability of IQ and the rise and fall of intelligence 9. The predictive validity of IQ - and its limits 10. Is this all? Multiple aspects of intelligence 11. Heritability: Kinship studies and single genes 12. The environment: secular changes and social class 13. Group differences 14. Sex differences 15. Epilogue

N. J. Mackintosh

Papers

A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement

The psychology of animal learning

Conditioning and associative learning

Mechanisms of animal discrimination learning

IQ and human intelligence