The capacity to predict future events permits a creature to detect, model, and manipulate the causal structure of its interactions with its environment. Behavioral experiments suggest that learning is driven by changes in the expectations about future salient events such as rewards and punishments. Physiological work has recently complemented these studies by identifying dopaminergic neurons in the primate whose fluctuating output apparently signals changes or errors in the predictions of future salient and rewarding events. Taken together, these findings can be understood through quantitative theories of adaptive optimizing control.

/pdf/a-neural-substrate-of-prediction-and-reward-2tj1zqgoih.pdf

A Neural Substrate of Prediction and Reward

In this article we propose mechanisms that govern the processing of emotional information, particularly those involved in fear reduction. Emotions are viewed as represented by information structures in memory, and anxiety is thought to occur when an information structure that serves as program to escape or avoid danger is activated. Emotional processing is denned as the modification of memory structures that underlie emotions. It is argued that some form of exposure to feared situations is common to many psychotherapies for anxiety, and that confrontation with feared objects or situations is an effective treatment. Physiological activation and habituation within and across exposure sessions are cited as indicators of emotional processing, and variables that influence activation and habituation of fear responses are examined. These variables and the indicators are analyzed to yield an account of what information must be integrated for emotional processing of a fear structure. The elements of such a structure are viewed as cognitive representations of the stimulus characteristic of the fear situation, the individual's responses in it, and aspects of its meaning for the individual. Treatment failures are interpreted with respect to the interference of cognitive defenses, autonomic arousal, mood state, and erroneous ideation with reformation of targeted fear structures. Applications of the concepts advanced here to therapeutic practice and to the broader study of psychopathology are discussed.

/pdf/emotional-processing-of-fear-exposure-to-corrective-2ehs11haf7.pdf

Emotional processing of fear : exposure to corrective information

Several formal models of excitatory classical conditioning are reviewed. It is suggested that a central problem for all of them is the explanation of cases in which learning does not occur in spite of the fact that the conditioned stimulus is a signal for the reinforcer. We propose a new model that deals with this problem by specifying that certain procedures cause a conditioned stimulus (CS) to lose effectiveness; in particular, we argue that a CS will lose associability when its consequences are accurately predicted. In contrast to other current models, the effectiveness of the reinforcer remains constant throughout conditioning. The second part of the article presents a reformulation of the nature of the learning produced by inhibitory-conditioning procedures and a discussion of the way in which such learning can be accommodated within the model outlined for excitatory learning.

http://smash.psych.nyu.edu/courses/spring16/learnmem/papers/PearceHall1980.pdf

A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli.

According to theories of selective attention, learning about a stimulus depends on attending to that stimulus; this is represented in two-stage models by saying that subjects switch in analyzers as well as learning stimulusresponse associations This assumption, however, is equally well represented in a formal model by the incorporation of a stimulus-specific learning-rate parameter, a, into the equations describing changes in the associative strength of stimuli Theories of selective attention have also assumed (a) that subjects learn to attend to and ignore relevant and irrelevant stimuli (ie, that a may increase or decrease depending on the correlation of a stimulus with reinforcement) and (b) that there is an inverse relationship between the probabilities of attending to different stimuli (ie, that an increase in a to one stimulus is accompanied by a decrease in a to others) The first assumption is used to explain the phenomena of acquired distinctiveness and dimensional transfer, the second those of overshadowing and blocking Although the first assumption is justified by the data, the second is not: Overshadowing and blocking are better explained by the choice of an appropriate rule for changing a, such that a decreases to stimuli that signal no change from the probability of reinforcement predicted by other stimuli

A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement

\, SUMMARY In 1967, Overmier and Seligman found that dogs exposed to inescapable and unavoidable electric shocks in one situation later failed to learn to escape shock in a different situation where escape was possible. Shortly thereafter Seligman and Maier (1967) demonstrated that this effect was caused by the uncontrollability of the original shocks. In this article we review the effects of exposing organisms to aversive events which they cannot control, and we review the explanations which have been offered. There seem to be motivational, cognitive, and emotional effects of uncontrollability. (a) Motivation. Dogs that have been exposed to inescapable shocks do not subsequently initiate escape response in the presence of shock. We review parallel phenomena in cats, fish, rats, and man. Of particular interest is the discussion of learned helplessness in rats and man. Rats are of interest because learned helplessness has been difficult to demonstrate in rats. However, we show that inescapably shocked rats do fail to learn to escape if the escape task is reasonably difficult. With regard to man, we review a variety of studies using inescapable noise and unsolvable problems as agents which produce learned helplessness effects on both instrumental and cognitive tasks, (b) Cognition. We argue that exposure to uncontrollabl e events interferes with the organism's tendency to perceive contingent relationships between its behavior and outcomes. Here we review a variety of studies showing such a cognitive set. (c) Emotion. We review a variety of experiments which show that uncontrollable aversive events produce greater emotional disruption than do controllable aversive events. We have proposed an explanation for these effects, which we call the learned helplessness hypothesis. It argues that when events are uncontrollable the organism learns that its behavior and outcomes are independent, and that this learning produces the motivational, cognitive, and emotional effects of uncontrollabi lity. We describe the learned helplessness hypothesis and research which supports it. Finally, we describe and discuss in detail alternative hypotheses which have been offered as accounts of the learned helplessness effect. One set of hypotheses argues that organisms learn motor responses during exposure to uncontrollabl e shock that compete with the response required in the test task. Another explanation holds that uncontrollable shock is a severe stressor and depletes a neurochemical necessary for the mediation of movement. We examine the logical structure of these explanations and present a variety of evidence which bears on them directly.

Byron A. Campbell

Papers

Punishment and aversive behavior