A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement
TL;DR: Overshadowing and blocking are better explained by the choice of an appropriate rule for changing a, such that a decreases to stimuli that signal no change from the probability of reinforcement predicted by other stimuli.
Abstract: According to theories of selective attention, learning about a stimulus depends on attending to that stimulus; this is represented in two-stage models by saying that subjects switch in analyzers as well as learning stimulusresponse associations This assumption, however, is equally well represented in a formal model by the incorporation of a stimulus-specific learning-rate parameter, a, into the equations describing changes in the associative strength of stimuli Theories of selective attention have also assumed (a) that subjects learn to attend to and ignore relevant and irrelevant stimuli (ie, that a may increase or decrease depending on the correlation of a stimulus with reinforcement) and (b) that there is an inverse relationship between the probabilities of attending to different stimuli (ie, that an increase in a to one stimulus is accompanied by a decrease in a to others) The first assumption is used to explain the phenomena of acquired distinctiveness and dimensional transfer, the second those of overshadowing and blocking Although the first assumption is justified by the data, the second is not: Overshadowing and blocking are better explained by the choice of an appropriate rule for changing a, such that a decreases to stimuli that signal no change from the probability of reinforcement predicted by other stimuli
Citations
More filters
Book•
01 Jan 1988TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Abstract: Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.
37,989 citations
TL;DR: Findings in this work indicate that dopaminergic neurons in the primate whose fluctuating output apparently signals changes or errors in the predictions of future salient and rewarding events can be understood through quantitative theories of adaptive optimizing control.
Abstract: The capacity to predict future events permits a creature to detect, model, and manipulate the causal structure of its interactions with its environment. Behavioral experiments suggest that learning is driven by changes in the expectations about future salient events such as rewards and punishments. Physiological work has recently complemented these studies by identifying dopaminergic neurons in the primate whose fluctuating output apparently signals changes or errors in the predictions of future salient and rewarding events. Taken together, these findings can be understood through quantitative theories of adaptive optimizing control.
8,163 citations
TL;DR: Tested the 2-process theory of detection, search, and attention presented by the current authors (1977) in a series of experiments and demonstrated the qualitative difference between 2 modes of information processing: automatic detection and controlled search.
Abstract: Tested the 2-process theory of detection, search, and attention presented by the current authors (1977) in a series of experiments. The studies (a) demonstrate the qualitative difference between 2 modes of information processing: automatic detection and controlled search; (b) trace the course of the
7,032 citations
TL;DR: Dopamine systems may have two functions, the phasic transmission of reward information and the tonic enabling of postsynaptic neurons.
Abstract: Schultz, Wolfram. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80: 1–27, 1998. The effects of lesions, receptor blocking, electrical self-stimulation, and drugs of abuse suggest t...
3,962 citations
TL;DR: A new model is proposed that deals with the explanation of cases in which learning does not occur in spite of the fact that the conditioned stimulus is a signal for the reinforcer by specifying that certain procedures cause a conditioned stimulus to lose effectiveness.
Abstract: Several formal models of excitatory classical conditioning are reviewed. It is suggested that a central problem for all of them is the explanation of cases in which learning does not occur in spite of the fact that the conditioned stimulus is a signal for the reinforcer. We propose a new model that deals with this problem by specifying that certain procedures cause a conditioned stimulus (CS) to lose effectiveness; in particular, we argue that a CS will lose associability when its consequences are accurately predicted. In contrast to other current models, the effectiveness of the reinforcer remains constant throughout conditioning. The second part of the article presents a reformulation of the nature of the learning produced by inhibitory-conditioning procedures and a discussion of the way in which such learning can be accommodated within the model outlined for excitatory learning.
2,779 citations
References
More filters
01 Jan 1972
6,206 citations
"A Theory of Attention: Variations i..." refers background in this paper
...The possibility of "configural" conditioning or "compounding" shows that this is not always true: With sufficient training, animals can learn to respond appropriately when AB is consistently reinforced but A and B separately are consistently not reinforced (e.g., Rescorla, 1972a; Woodbury, 1943)....
[...]
...As training continues, however, the difference in validity outweighs the difference in salience, and B loses control while A gains control (e.g., Jenkins, 1973; Rescorla, 1972b)....
[...]
4,181 citations
2,233 citations
"A Theory of Attention: Variations i..." refers background in this paper
...To this point, the elementary ideas being advanced do not represent any significant departure from the central assumptions of traditional theories of learning, at least of theories such as those advanced by Hull (1943), Spence (1936,1956), or Estes (1950, 1959)....
[...]
01 Dec 1967
TL;DR: The role of attention in Pavlovian conditioning, and use of auditory and visual stimuli to condition rats is discussed in this article, where the authors discuss the use of both visual and auditory stimuli.
Abstract: Role of attention in Pavlovian conditioning, and use of auditory and visual stimuli to condition rats
1,562 citations