Choice, rate of reinforcement, and the changeover delay
01 Mar 1970-Journal of the Experimental Analysis of Behavior (Society for the Experimental Analysis of Behavior)-Vol. 13, Iss: 2, pp 187-197
TL;DR: The present study shows that equality between proportions of responses and proportions of reinforcements ("matching") is obtained when the value of the changeover delay is varied.
Abstract: Pigeons distribute their responses on concurrently available variable-interval schedules in the same proportion as reinforcements are distributed on the two schedules only when a changeover delay is used. The present study shows that this equality between proportions of responses and proportions of reinforcements (“matching”) is obtained when the value of the changeover delay is varied. When responses are partitioned into the set of rapid response bursts occurring during the delay interval and the set of responses occurring subsequently, the proportion of neither set of responses matches the proportion of reinforcements. Instead, each set deviates from matching but in opposite directions. Matching on the gross level results from the interaction of two patterns evident in the local response rates: (I) the lengthening of the changeover delay response burst is accompanied by a commensurate decrease in the number of changeovers; (2) the changeover delay response burst is longer than the scheduled delay duration. When delay responses are eliminated by introducing a blackout during the delay interval, response matching is eliminated; the pigeon, however, continues to match the proportion of time spent responding on a key to the proportion of reinforcements obtained on that key.
Citations
More filters
TL;DR: The procedural variations that lead to undermatching appear to be those that produce (a) asymmetrical pausing that favors the poorer alternative; (b) systematic temporal variation in preference that Favorites 1 and 2; and (c) patterns of responding that involve changing over between alternatives or brief bouts at the alternatives.
Abstract: Almost all of 103 sets of data from 23 different studies of choice conformed closely to the equation: log (B(1)/B(2)) = a log (r(1)/r(2)) + log b, where B(1) and B(2) are either numbers of responses or times spent at Alternatives 1 and 2, r(1) and r(2) are the rates of reinforcement obtained from Alternatives 1 and 2, and a and b are empirical constants. Although the matching relation requires the slope a to equal 1.0, the best-fitting values of a frequently deviated from this. For B(1) and B(2) measured as numbers of responses, a tended to fall short of 1.0 (undermatching). For B(1) and B(2) measured as times, a fell to both sides of 1.0, with the largest mode at about 1.0. Those experiments that produced values of a for both responses and time revealed only a rough correspondence between the two values; a was often noticeably larger for time. Statistical techniques for assessing significance of a deviation of a from 1.0 suggested that values of a between .90 and 1.11 can be considered good approximations to matching. Of the two experimenters who contributed the most data, one generally found undermatching, while the other generally found matching. The difference in results probably arises from differences in procedure. The procedural variations that lead to undermatching appear to be those that produce (a) asymmetrical pausing that favors the poorer alternative; (b) systematic temporal variation in preference that favors the poorer alternative; and (c) patterns of responding that involve changing over between alternatives or brief bouts at the alternatives.
676 citations
TL;DR: The results of the two experiments suggest that the relative reinforcement of a response determines its rate, whereas the stimulus-reinforcement contingency (a Pavlovian contingency) determines its resistance to change.
Abstract: Two multiple-schedule experiments with pigeons examined the effect of adding food reinforcement from an alternative source on the resistance of the reinforced response (target response) to the decremental effects of satiation and extinction. In Experiment 1, key pecks were reinforced by food in two components according to variable-interval schedules and, in some conditions, food was delivered according to variable-time schedules in one of the components. The rate of key pecking in a component was negatively related to the proportion of reinforcers from the alternative (variable-time) source. Resistance to satiation and extinction, in contrast, was positively related to the overall rate of reinforcement in the component. Experiment 2 was conceptually similar except that the alternative reinforcers were contingent on a specific concurrent response. Again, the rate of the target response varied as a function of its relative reinforcement, but its resistance to satiation and extinction varied directly with the overall rate of reinforcement in the component stimulus regardless of its relative reinforcement. Together the results of the two experiments suggest that the relative reinforcement of a response (the operant contingency) determines its rate, whereas the stimulus-reinforcement contingency (a Pavlovian contingency) determines its resistance to change.
302 citations
TL;DR: A modified formulation with a multiplier for the overall rate of primary reinforcement obtained on each key provides a better description of choice in simple (concurrent) choice situations, an advantage not achieved by previous formulations.
Abstract: Pigeons' responses in the presence of two concurrently available (initial-link) stimuli produced one of two different (terminal-link) stimuli. Entrance into the mutually exclusive terminal links was arranged by different and independent variable-interval schedules for each key, while responses during the mutually exclusive terminal-link stimuli produced a single food reinforcement according to indentical and independent variable-interval schedules. The pigeons emitted more initial-link responses on the key with the shorter average interreinforcement interval in the initial link. This difference in initial-link response rates varied directly with the difference between the average inter-reinforcement intervals of the initial-link schedules and decreased when the initial-link schedule with the longer average interreinforcement interval was followed by several consecutive food reinforcements on the variable-interval schedule in the terminal link on that key. These results are incompatible with previous formulations of choice behavior with the concurrent-chains procedure. A modified formulation with a multiplier for the overall rate of primary reinforcement obtained on each key provides a better description of choice. In addition, the new formulation applies to behavior in simple (concurrent) choice situations, an advantage not achieved by previous formulations.
214 citations
TL;DR: The policy that maximizes overall reward rate on two variable-interval paradigms is derived and it is shown that for nearly all parameter values, a switch to the schedule with the longer interval should be followed immediately by a switch back to thedule with the shorter interval.
Abstract: Without assuming any constraints on behavior, we derive the policy that maximizes overall reward rate on two variable-interval paradigms. The first paradigm is concurrent variable time-variable time with changeover delay. It is shown that for nearly all parameter values, a switch to the schedule with the longer interval should be followed immediately by a switch back to the schedule with the shorter interval. The matching law does not hold at the optimum and does not uniquely specify the obtained reward rate. The second paradigm is discrete trial concurrent variable interval-variable interval. For given schedule parameters, the optimal policy involves a cycle of a fixed number of choices of the schedule with the shorter interval followed by one choice of the schedule with the longer interval. Molecular maximization sometimes results in optimal behavior.
170 citations
TL;DR: The extant data for pigeons' performance on concurrent variable-interval schedules suggested the generality that pigeons respond less to the richer reinforcement schedule than predicted by matching and a nonparametric test for distribution of points supported this concept of undermatching.
Abstract: The extant data for pigeons' performance on concurrent variable-interval schedules were examined in detail. Least-squares lines relating relative pecks and time to the corresponding relative reinforcements were obtained for four studies. The between-study group slopes for time and pecks and five of seven within-study group slopes from individual studies were less than 1.00. This suggested the generality that pigeons respond less to the richer reinforcement schedule than predicted by matching. For pecks, a nonparametric test for distribution of points also supported this concept of undermatching (to the richer reinforcement schedule). In addition, using mean squared error as the criterion, a cubic curve fit the peck proportion data better than any line or other polynomial. This indicates that the relation between peck and reinforcement proportions may be nonlinear.
156 citations
References
More filters
TL;DR: The present experiment is a study of strength of response of pigeons on a concurrent schedule under which they peck at either of two response-keys and investigates output as a function of frequency of reinforcement.
Abstract: A previous paper (Herrnstein, 1958) reported how pigeons behave on a concurrent schedule under which they peck at either of two response-keys The significant finding of this investigation was that the relative frequency of responding to each of the keys may be controlled within narrow limits by adjustments in an independent variable In brief, the requirement for reinforcement in this procedure is the emission of a minimum number of pecks to each of the keys The pigeon receives food when it completes the requirement on both keys The frequency of responding to each key was a close approximation to the minimum requirement The present experiment explores the relative frequency of responding further In the earlier study it was shown that the output of behavior to each of two keys may be controlled by specific requirements of outputs Now we are investigating output as a function of frequency of reinforcement The earlier experiment may be considered a study of differential reinforcement; the present one, a study of strength of response Both experiments are attempts to elucidate the properties of rdlative frequency of responding as a dependent variable
2,220 citations
376 citations
TL;DR: When a pigeon's pecks on two keys were reinforced concurrently by two independent variable-interval (VI) schedules, one for each key, the response rate on either key was given by the equation: R(1)=R(1)/(r(1)+r(2))(5/6), where R is response rate, r is reinforcement rate, and the subscripts 1 and 2 indicate keys 1 and 1.
Abstract: When a pigeon's pecks on two keys were reinforced concurrently by two independent variable-interval (VI) schedules, one for each key, the response rate on either key was given by the equation: R(1)=Kr(1)/(r(1)+r(2))(5/6), where R is response rate, r is reinforcement rate, and the subscripts 1 and 2 indicate keys 1 and 2. When the constant, K, was determined for a given pigeon in one schedule sequence, the equation predicted that pigeon's response rates in a second schedule sequence. The equation derived from two characteristics of the performance: the total response rate on the two keys was proportional to the one-sixth power of the total reinforcement rate provided by the two VI schedules; and, the pigeon matched the relative response rate on a key to the relative reinforcement rate for that key. The equation states that response rate on one key depends in part on reinforcement rate for the other key, but implies that it does not depend on response rate on the other key. This independence of response rates on the two keys was demonstrated by presenting a stimulus to the pigeon whenever one key's schedule programmed reinforcement. This maintained the reinforcement rate for that key, but reduced the response rate almost to zero. The response rate on the other key, nevertheless, continued to vary with reinforcement rates according to the equation.
241 citations
218 citations
TL;DR: The pigeon and the rat partition total response output between both schedules of a concurrent variable-interval pair is studied and the quantitative nature of a partition seems critically dependent on the relative rates with which the two schedules provide reinforcements for responding, in addition to the changeover delay.
Abstract: The pigeon and the rat partition total response output between both schedules of a concurrent variable-interval pair. The quantitative nature of a partition seems critically dependent on the relative rates with which the two schedules provide reinforcements for responding, in addition to the changeover delay. The manner in which the changeover delay controls the partition was studied by varying the duration of the changeover delay from 0 to 20 sec with each of two pairs of concurrent variable-interval schedules, viz., Conc VI 1.5-min VI 1.5-min and Conc VI 1-min VI 3-min. Rats served as the subjects and brain stimulation was employed as the reinforcer. When the schedules were Conc VI 1.5-min VI 1.5-min, relative response rate approximated 0.50 at all values of the changeover delay. When the schedules were Conc VI 1-min VI 3-min, relative response rate, computed with respect to the VI 1-min schedule, increased when the duration of the changeover delay increased. Changeover rate decreased when the duration of the changeover delay increased. The decrease was the same for both VI schedules of the Conc VI 1.5-min VI 1.5-min pair but was more rapid for the VI 3-min schedule of the Conc VI 1-min VI 3-min pair.
179 citations