Evidence supports at least two methods for learning about reward and punishment and making predictions for guiding actions. One method, called model-free, progressively acquires cached estimates of the long-run values of circumstances and actions from retrospective experience. The other method, called model-based, uses representations of the environment, expectations, and prospective calculations to make cognitive predictions of future value. Extensive attention has been paid to both methods in computational analyses of instrumental learning. By contrast, although a full computational analysis has been lacking, Pavlovian learning and prediction has typically been presumed to be solely model-free. Here, we revise that presumption and review compelling evidence from Pavlovian revaluation experiments showing that Pavlovian predictions can involve their own form of model-based evaluation. In model-based Pavlovian evaluation, prevailing states of the body and brain influence value computations, and thereby produce powerful incentive motivations that can sometimes be quite new. We consider the consequences of this revised Pavlovian view for the computational landscape of prediction, response, and choice. We also revisit differences between Pavlovian and instrumental learning in the control of incentive motivation.

/pdf/model-based-and-model-free-pavlovian-reward-learning-1dprmdccu5.pdf

Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation

Animals, in particular humans, frequently punish other individuals who behave negatively or uncooperatively towards them. In animals, this usually serves to protect the personal interests of the individual concerned, and its kin. However, humans also punish altruistically, in which the act of punishing is personally costly. The propensity to do so has been proposed to reflect the cultural acquisition of norms of behaviour, which incorporates the desire to uphold equity and fairness, and promotes cooperation. Here, we review the proximate neurobiological basis of punishment, considering the motivational processes that underlie punishing actions.

/pdf/the-neurobiology-of-punishment-23475jq7l7.pdf

The neurobiology of punishment

This chapter considers what factors influence the occurrence of attack and fear behavior (e.g., immobility, flight, distress signals). It is argued that, with the exception of conditioned attack and fear behavior, the same types of basic situations are capable of evoking either type of response.

The Organization of Aggression and Fear In Vertebrates

The validity of behavioural measures of aversion: a review

The development of early visual self-recognition

Male Siamese fighting fish (Betta splendens) were reinforced for swimming through an aperture by a mirror presentation which released an innate aggressive display. Moderately intense shock-punishment of the operant response increased the vigor of the display and, temporarily, the rates of the operant response. Gradual decreases in rate within and across nonpunished sessions reflected short-term and long-term habituation to the mirror image.

/pdf/facilitative-effects-of-punishment-on-aggressive-behavior-in-s7k9jj9x43.pdf

Facilitative effects of punishment on aggressive behavior in the Siamese fighting fish

A mirror presentation and the ensuing species-specific aggressive display served to reinforce the response of swimming through an aperture in paradise fish. Learning and extinction curves were shown with this type of reinforcement; however, non-contingent mirror presentations did not maintain performance. A declining trend in operant swimming performance with repeated image/display reinforcements reflected some habituation of the display.

John E. Anson

Papers

Facilitative effects of punishment on aggressive behavior in the Siamese fighting fish

Image-Induced Aggressive Display: Reinforcement in the Paradise Fish

Sources of reinforcement in the establishment of self-punitive behavior.