Many organisms, especially humans, are characterized by their capacity for intentional, goal-directed actions. However, similar behaviours often proceed automatically, as habitual responses to antecedent stimuli. How are goal-directed actions transformed into habitual responses? Recent work combining modern behavioural assays and neurobiological analysis of the basal ganglia has begun to yield insights into the neural basis of habit formation.

The role of the basal ganglia in habit formation

The prevailing theories of avoidance learning and the procedures that are usually used to study it seem to be totally out of touch with what is known about how animals defend themselves in nature. This paper suggests some alternative concepts, starting with the assumption that animals have innate species-specific defense reactions (SSDRs) such as fleeing, freezing, and fighting. It is proposed that if a particular avoidance response is rapidly acquired, then that response must necessarily be an SSDR. The learning mechanism in this case appears to be suppression of nonavoidance behavior by the avoidance contingency. The traditional approaches to avoidance learning appear to be slightly more valid in the case of responses that are slowly acquired, although in this case, too, the SSDR concept is relevant, and reinforcement appears to be based on the production of a safety signal rather than the termination of an aversive conditioned stimulus. Avoidance learning as we know it in the laboratory has frequently been used to "explain" how animals survive in the wild. The purpose of this paper is to turn this inferential process around and use the limited knowledge of natural defensive behavior to help account for some of the anomalies that have been found in laboratory studies of avoidance learning. Let us begin by recalling a little fable. It is a very familiar fable. It was already part of our lore when Hull gave his version of it in 1929, and the story has been told again many times since then. It goes something like this: Once upon a time there was a little animal who ran around in the forest. One day

Species-specific defense reactions and avoidance learning.

Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action

The history of 2-process learning theory is described, and the logical and empirical validity of its major postulates is examined. The assumption of 2 acquisition processes requires the demonstration of an empirical interaction between 2 types of reinforcement contingencies and (a) response classes, (b) reinforcing stimulus classes, or (c) characteristics of the learned behavior itself. The mediation postulates of 2-process theory which argue that CRs are intimately involved in the control of instrumental responding are emphasized, and 2 major lines of evidence that stem uniquely from these postulates are examined : (a) the concurrent development and maintenance of instrumental responses and conditioned reflexes, and (b) the interaction between separately conducted Pavlovian conditioning contingencies and instrumental training contingencies in the control of instrumental behavior. The evidence from concurrent measurement studies provides, at the very best, only weak support for the mediational hypotheses of 2-process theory. In contrast, the evidence from interaction studies shows the strong mediating control of instrumental responses by Pavlovian conditioning procedures, and demonstrates the surprising power of Pavlovian concepts in predicting the outcomes of many kinds of interaction experiments.

Two-process learning theory: Relationships between Pavlovian conditioning and instrumental learning

Goal-directed instrumental action: Contingency and incentive learning and their cortical substrates

The problem of maintaining independence between response rates and reinforcement probabilities when determining the effect of varying the response-reinforcement contingency upon free-operant behavior was solved by programming local reinforcement probabilities for response and no response on a second-by-second basis. Fifty-seven rats were trained to lever-press on schedules of water reinforcement involving different values of contingency. All rats were first trained on a high positive contingency and then shifted to less positive, zero, or negative contingencies. Under these conditions, rate of lever-pressing declined appropriately when the contingency between response and reinforcement decreased or was made negative. The decline in rate produced by a zero contingency cannot be attributed to extinction, since the probability of reinforcement given the occurrence of a response was the same as for the positive contingency from which the shift to zero was made. That is, there was no change in the opportunity for response-reinforcement contiguity. It was concluded that the technique of programming local reinforcement probabilities offers promise for more critical examinations of the effects of contingency upon free-operant behavior.

/pdf/the-effect-of-contingency-upon-the-appetitive-conditioning-45u2grhgus.pdf

The effect of contingency upon the appetitive conditioning of free-operant behavior.

Rats were given differential conditioning to a tone (signal for shock) and a light (signal for no shock) using the CER procedure Response rate in the presence of the CS− (light) was compared to the unconditioned effects of the light upon the VI response rate in a no-shock control group CER Ss showed enhanced response rates during the light, to an extent greater than that during the light presented to the control group

/pdf/increased-responding-to-cs-in-differential-cer-4khkazy25h.pdf

Increased responding to CS− in differential CER

Rats in an experimental group were given 30 trials of differential CER and then the CS+ and CS− were combined during CER extinction. The combination resulted in less suppression for the experimental group than shown by a control group which had a CS+ and a formerly random stimulus combined during extinction. This was interpreted as a demonstration of the active inhibitory properties of CS−.

https://link.springer.com/content/pdf/10.3758%2FBF03330761.pdf

A traditional demonstration of the active properties of Pavlovian inhibition using differential CER

Probabilistic contingency theories of animal conditioning: A critical analysis

Several experiments have found complete retention of conditioned suppression when overall suppression to the signal for shock was measured. The present experiment examined retention of conditioned suppression but did so with a paradigm that produced temporal discrimination of shock occurrence. Nine rats were exposed to a flashing light signal of 5-min duration that always terminated with a shock. After several months, a temporal discrimination was well established, as shown by maximum suppression toward the end of the signal period. After remaining in the home cage for 25 days, the rats were again subjected to the conditioning procedure. The overall level of suppression remained the same but the temporal discrimination was not observed.

Lynn J. Hammond

Papers

The effect of contingency upon the appetitive conditioning of free-operant behavior.

Increased responding to CS− in differential CER

A traditional demonstration of the active properties of Pavlovian inhibition using differential CER

Probabilistic contingency theories of animal conditioning: A critical analysis

Forgetting and conditioned suppression: role of a temporal discrimination1