The authors draw together and develop previous timing models for a broad range of conditioning phenomena to reveal their common conceptual foundations: First, conditioning depends on the learning of the temporal intervals between events and the reciprocals of these intervals, the rates of event occurrence. Second, remembered intervals and rates translate into observed behavior through decision processes whose structure is adapted to noise in the decision variables. The noise and the uncertainties consequent on it have both subjective and objective origins. A third feature of these models is their timescale invariance, which the authors argue is a very important property evident in the available experimental data. This conceptual framework is similar to the psychophysical conceptual framework in which contemporary models of sensory processing are rooted. The authors contrast it with the associative conceptual framework.

Time, rate, and conditioning.

Pigeons were given a small, immediate food reinforcement for pecking a key, and a larger, delayed reinforcement for not pecking this key. Most subjects pecked the key on more than 95% of trials. However, when pecking a differently colored key at an earlier time prevented this option from becoming available, three of 10 subjects consistently pecked it, thereby forcing themselves to wait for the larger reward. They did not peck the earlier key when it did not prevent this option. This is an experimental example of psychological impulse and a learnable device to control it. Although only a minority of the subjects learned it, the fact that such learning is possible at all argues for a theory of delayed reward that can predict change of preference as a function of elapsing time.

/pdf/impulse-control-in-pigeons-28i7eqhpb0.pdf

Impulse control in pigeons.

The sway that the response-reinforcement framework (Spencer, Thorndike, Hull, Skinner) has held on the behavioral sciences for nearly a hundred years is finally ending. The strength of this framework lay in providing concepts and methods for studying the effects of hedonic (reinforcing) stimuli on the repetition of specified responses acquired in instrumental training situations of various kinds. Its weakness lay in the invalidity of its central assumptions, stimulus-response association and response-reinforcement, which could not deal with motor-equivalence and flexibility (or “intelligence”) in behavior. To the four decades of incisive criticism on particular theoretical and empirical grounds, a more comprehensive challenge to the response-reinforcement framework is now added by the newer ideas about the nature of cognitive, motivational, and response-production processes that have emerged from the work of ethologists, neuroscientists, and cognitive psychologists. An alternative framework, incorporating the newer ideas, is clearly needed.The particular framework proposed here is based on the ideas of perceptual learning of stimulus-stimulus correlations and of a motivational (rather than reinforcing) role of hedonic (incentive) stimuli. According to it, an act is produced when its act-assembly is activated by a pexgo (perceptual representation) of a certain eliciting stimulus complex (ES). When certain eliciting stimuli are correlated with incentive stimuli, they acquire motivational properties that serve to strengthen the pexgos generated by those eliciting stimuli and thereby increase the probability of activation of the corresponding act-assemblies. Motivation thus influences response production, not by directly instigating “existing” responses, but by modulating the strength of pexgos of eliciting stimuli for the succession of acts that comprise a response. Therefore, a response is always constructed afresh on the basis of current perceptions; not even a stable and stereotyped response occurs as a mere activation of a preformed motor program. The topography of any response that emerges is determined by the nature of the motivational state and the momentary spatiotemporal distribution of eliciting stimuli of changing motivational valence.By suggesting that the animal learns the overlapping and nested correlations between the stimulus events that commonly occur in a given situation, and by separating what is learned from the processes of response production, the proposed perceptual-motivational framework seems capable of dealing with the problems of motor equivalence and flexibility in adaptive behavior. Some implications of this approach for further behavioral and brain research on such problems as behavior modification, learning by observation of models, analysis of causality, and search for neural substrates of learning and response production, are outlined.

How adaptive behavior is produced: a perceptual-motivational alternative to response reinforcements

In this article, we propose a theoretical framework for understanding and integrating people's and animals' covariation assessment. We argue that covariation perception is determined by the interaction between two sources of information: (a) the organism's prior expectations about the covariation between two events and (b) current situational information provided by the environment about the objective contingency between the events. Both accuracies and errors in people's and animals' covariation assessments are analyzed within this interactional theoretical framework. We then review four lines of research in support of this analysis. Finally, we consider the issue of accuracy versus rationality in covariation assessment. A consensus has been forming among learning, clinical, and social psychologists: The ability to detect the relationships or covariations among stimuli, behaviors, and outcomes in one's environment is an important component of adaptive behavior. The covariation between two events may be defined in terms of their co-occurrence (i.e., the degree to which one event occurs more often in the presence than in the absence of the other event). Information about the relationships or covariations between events in the world provides people and animals with a means of explaining the past, controlling the present, and predicting the future, thereby maximizing the likelihood that they can obtain desired outcomes and avoid aversive ones. The concept of covariation provides a cornerstone for a number of substantive areas within psychology. For example, contemporary learning theorists point to the role of ob

Assessment of covariation by humans and animals: The joint influence of prior expectations and current situational information.

A motivational view of learning, performance, and behavior modification.

Key pecking in pigeons can be engendered by associating response-independent food presentations with illumination of a key. Specific pairings of key and food are not necessary for this phenomenon. Differential positive association between key and food (defined in terms of relative densities of reinforcement), however, is necessary and sufficient to produce and maintain key pecking. Thus, the occurrence of key pecking in auto-shaping can be considered to depend on associative processes similar to classical conditioning. Consequently, auto-shaped pecking can be virtually eliminated by the addition of food presentations in the intertrial interval, thus removing the association between key and food. Initial exposure to random reinforcement, or reinforcement only in the absence of the key, results in lower rates of pecking in subsequent auto-shaping procedures.

Associative factors underlying the pigeon's key pecking in auto-shaping procedures.

The pigeon9s so-called "arbitrary" response of pecking an illuminated disk can be established and maintained by procedures resembling those of classical conditioning. This phenomenon was shown to be independent of the specific signaling relationships between illumination of the pecking disk and presentation of food; it will appear as long as the key is differentially associated with food. When a nondifferential condition is introduced, pecking "extinguishes" even if it has previously been established and even when the new condition involves as much reinforcement as the old one. Reinstating differential conditions reestablishes pecking. The initial conditions determine the speed and apparently the asymptote of pecking rates in the differential condition; initial exposure to a nondifferential procedure retards subsequent acquisition, possibly quite permanently. These findings are discussed in the context of mechanisms of adaptive learning, not involving reward and punishment, which lead to selection of effective behaviors on a nonarbitrary basis.

https://science.sciencemag.org/content/sci/171/3974/923.full.pdf

Classical Conditioning of a Complex Skeletal Response

The joint control of rate of key pecking in pigeons by stimulus-reinforcer and response-reinforcer relationships was studied in the context of a two-component multiple schedule of reinforcement. Food presentation was always associated with one component and extinction with the other. The stimulus-reinforcer relationship was manipulated by varying the relative durations of the two components. In the food-presentation component, a fixed rate of reinforcement, independent of rate of responding, was generated by a schedule referred to as "T*". One aspect of the response-reinforcer relationship, contiguity, was manipulated by varying the percentage of delayed reinforcers. With the multiple T* extinction schedule, stimulus-reinforcer and response-reinforcer relationships could be varied independently of one another. Rate of key pecking was sensitive to manipulations of both relationships. However, significant differential effects due to either the stimulus-reinforcer or response-reinforcer relationship were obtained only when the other relationship was weak: stimulus-reinforcer and response-reinforcer relationships interacted in the joint control of responding.

/pdf/associative-interaction-joint-control-of-key-pecking-by-2u2aj27nc5.pdf

Elkan Gamzu

Papers

Associative factors underlying the pigeon's key pecking in auto-shaping procedures.

Classical Conditioning of a Complex Skeletal Response

Associative interaction: joint control of key pecking by stimulus-reinforcer and response-reinforcer relationships.