Replication and extension of Skinner's "superstition" experiment showed the development of two kinds of behavior at asymptote: interim activities (related to adjunctive behavior) occurred just after food delivery; the terminal response (a discriminated operant) occurred toward the end of the interval and continued until food delivery. These data suggest a view of operant conditioning (the terminal response) in terms of two sets of principles: principles of behavioral variation that describe the origins of behavior "appropriate" to a situation, in advance of reinforcement; and principles of reinforcement that describe the selective elimination of behavior so produced. This approach was supported by (a) an account of the parallels between the Law of Effect and evolution by means of natural selection, (fc) its ability to shed light on persistent problems in learning (e.g., continuity vs. noncontinuity, variability associated with extinction, the relationship between classical and instrumental conditioning, the controversy between behaviorist and cognitive approaches to learning), and (c) its ability to deal with a number of recent anomalies in the learning literature ("instinctive drift," auto-shaping, and auto-maintenance). The interim activities were interpreted in terms of interactions among motivational systems, and this view was supported by a review of the literature on adjunctive behavior and by comparison with similar phenomena in ethology (displacement, redirection, and "vacuum" activities). The proposed theoretical scheme represents a shift away from hypothetical "laws of learning" toward an interpretation of behavioral change in terms of interaction and competition among tendencies to action according to principles evolved in phylogeny.

The "supersitition" experiment: A reexamination of its implications for the principles of adaptive behavior.

The authors draw together and develop previous timing models for a broad range of conditioning phenomena to reveal their common conceptual foundations: First, conditioning depends on the learning of the temporal intervals between events and the reciprocals of these intervals, the rates of event occurrence. Second, remembered intervals and rates translate into observed behavior through decision processes whose structure is adapted to noise in the decision variables. The noise and the uncertainties consequent on it have both subjective and objective origins. A third feature of these models is their timescale invariance, which the authors argue is a very important property evident in the available experimental data. This conceptual framework is similar to the psychophysical conceptual framework in which contemporary models of sensory processing are rooted. The authors contrast it with the associative conceptual framework.

Time, rate, and conditioning.

Maximization theory, which is borrowed from economics, provides techniques for predicing the behavior of animals - including humans. A theoretical behavioral space is constructed in which each point represents a given combination of various behavioral alternatives. With two alternatives - behavior A and behavior B - each point within the space represents a certain amount of time spent performing behavior A and a certain amount of time spent performing behavior B. A particular environmental situation can be described as a constraint on available points (a circumscribed area) within the space. Maximization theory assumes that animals always choose the available point with the highest numerical value. The task of maximization theory is to assign to points in the behavioral space values that remain constant across various environmental situations; as those situations change, the point actually chosen is always the one with the highest assigned value.Maximization theory is an alternative to reinforcement theory as a description of steady-state behavior. Situations to which reinforcement theory has been directly applied (such as reinforcement of rats pressing levers and pigeons pecking keys in Skinner boxes) and situations to which reinforcement theory has occasionally been extended (such as human economic behavior and human self-control) can be described by maximization theory. This approach views behavior as a quantitative outcome of the interaction of the putative instrumental response, the reinforcer, and the other activities available in the situation. It provides new insight into these situations and, because it takes context into account, has greater predictive power than reinforcement theory.

https://www.cambridge.org/core/services/aop-cambridge-core/content/view/S0140525X00009407

Charles P. Shimp

Papers

Optimal behavior in free-operant experiments.