Data on choice generally conform closely to an equation of the form: log(B1/B2)=a log(r1/r2+log k, where B1 and B2 are the frequencies of responding at Alternatives 1 and 2, r1 and r2 are the obtained reinforcement from Alternatives 1 and 2, and a and k are empirical constants. When a and k equal one, this equation is equivalent to the matching relation: B1/B2=r1/r2. Two types of deviation from matching can occur with this formulation: a and k not equal to one. In some experiments, a systematically falls short of one. This deviation is undermatching. The reasons for undermatching are obscure at present. Some evidence suggests, however, that factors favoring discrimination also favor matching. Matching (a=1) may represent the norm in choice when discrimination is maximal. When k differs from one, its magnitude indicates the degree of bias in choice. The generalized matching law predicts that bias should take this form (adding a constant proportion of responding to the favored alternative). Data from a variety of experiments indicate that it generally does.

/pdf/on-two-types-of-deviation-from-the-matching-law-bias-and-4g4uri3jwt.pdf

On two types of deviation from the matching law: bias and undermatching

The tendency to choose a larger, more delayed reinforcer over a smaller, less delayed one has frequently been termed “selfcontrol.” Three very different research traditions – two models emphasizing the control of local contingencies of reinforcement (Mischel's social learning theory and Herrnstein's matching law) and molar maximization models (specifically optimal foraging theory) – have all investigated behavior within the self-control paradigm. A framework is proposed to integrate research from all three research areas. This framework consists of three parts: a procedural analysis, a causal analysis, and a theoretical analysis. The procedural analysis provides a common procedural terminology for all three areas. The causal analysis establishes that, in all three research traditions, self-control varies directly with the current physical values of the reinforcers; that is, choices increase with reinforcer amount and decrease with reinforcer delay. But self-control also varies according to past events to which a subject has been exposed, and according to current factors other than the reinforcers. Each of the three models has therefore incorporated these indirect effects on self-control by postulating unobservable mechanisms. In all three cases, these mechanisms represent a subject's behavior as a function of a perceived environment. The theoretical analysis demonstrates that evolutionary theory can encompass the research from all three areas by considering differences in the adaptiveness of self-control in different situations. This integration provides a better and more predictive description of self-control.

Research on self-control: An integrating framework

Almost all of 103 sets of data from 23 different studies of choice conformed closely to the equation: log (B(1)/B(2)) = a log (r(1)/r(2)) + log b, where B(1) and B(2) are either numbers of responses or times spent at Alternatives 1 and 2, r(1) and r(2) are the rates of reinforcement obtained from Alternatives 1 and 2, and a and b are empirical constants. Although the matching relation requires the slope a to equal 1.0, the best-fitting values of a frequently deviated from this. For B(1) and B(2) measured as numbers of responses, a tended to fall short of 1.0 (undermatching). For B(1) and B(2) measured as times, a fell to both sides of 1.0, with the largest mode at about 1.0. Those experiments that produced values of a for both responses and time revealed only a rough correspondence between the two values; a was often noticeably larger for time. Statistical techniques for assessing significance of a deviation of a from 1.0 suggested that values of a between .90 and 1.11 can be considered good approximations to matching. Of the two experimenters who contributed the most data, one generally found undermatching, while the other generally found matching. The difference in results probably arises from differences in procedure. The procedural variations that lead to undermatching appear to be those that produce (a) asymmetrical pausing that favors the poorer alternative; (b) systematic temporal variation in preference that favors the poorer alternative; and (c) patterns of responding that involve changing over between alternatives or brief bouts at the alternatives.

Matching, undermatching, and overmatching in studies of choice.

Two multiple-schedule experiments with pigeons examined the effect of adding food reinforcement from an alternative source on the resistance of the reinforced response (target response) to the decremental effects of satiation and extinction. In Experiment 1, key pecks were reinforced by food in two components according to variable-interval schedules and, in some conditions, food was delivered according to variable-time schedules in one of the components. The rate of key pecking in a component was negatively related to the proportion of reinforcers from the alternative (variable-time) source. Resistance to satiation and extinction, in contrast, was positively related to the overall rate of reinforcement in the component. Experiment 2 was conceptually similar except that the alternative reinforcers were contingent on a specific concurrent response. Again, the rate of the target response varied as a function of its relative reinforcement, but its resistance to satiation and extinction varied directly with the overall rate of reinforcement in the component stimulus regardless of its relative reinforcement. Together the results of the two experiments suggest that the relative reinforcement of a response (the operant contingency) determines its rate, whereas the stimulus-reinforcement contingency (a Pavlovian contingency) determines its resistance to change.

Alternative reinforcement increases resistance to change: pavlovian or operant contingencies?

CHAPTER 5 – Melioration and Behavioral Allocation1

Responding by pigeons on one key of a two-key chamber alternated the color of the second key, on which responding produced food according to a variable-interval schedule of reinforcement. From time to time, reinforcement would be available for a response, but in the presence of a particular stimulus, either red or green light on the key. Red or green was chosen irregularly from reinforcement to reinforcement, so that a proportion of the total number of reinforcements could be specified for each color. Experimental manipulations involved variations of (1) the proportions for each color, (2) changeover delay, or, alternatively, (3) a fixed-ratio changeover requirement. The main findings were: (1) relative overall rates of responding and relative times in the presence of a key color approximated the proportions of reinforcements obtained in the presence of that color, while relative local rates of responding changed little; (2) changeover rate decreased as the proportions diverged from 0.50; (3) relative overall rate of responding and relative time remained constant as the changeover delay was increased from 2 to 32 sec, with reinforcement proportions for red and green of 0.75 and 0.25, but they increased above 0.90 when a fixed-ratio changeover of 20 responses replaced the changeover delay; (4) changeover rate decreased as the delay or fixed-ratio was increased.

/pdf/concurrent-responding-with-fixed-relative-rate-of-4136od9eft.pdf

D. Alan Stubbs

Papers

Concurrent responding with fixed relative rate of reinforcement.