scispace - formally typeset
Search or ask a question
Journal ArticleDOI

On the law of effect

01 Mar 1970-Journal of the Experimental Analysis of Behavior (Society for the Experimental Analysis of Behavior)-Vol. 13, Iss: 2, pp 243-266
TL;DR: Experiments on single, multiple, and concurrent schedules of reinforcement find various correlations between the rate of responding and the rate or magnitude of reinforcement, which can be accounted for by a coherent system of equations.
Abstract: Experiments on single, multiple, and concurrent schedules of reinforcement find various correlations between the rate of responding and the rate or magnitude of reinforcement. For concurrent schedules (i.e., simultaneous choice procedures), there is matching between the relative frequencies of responding and reinforcement; for multiple schedules (i.e., successive discrimination procedures), there are contrast effects between responding in each component and reinforcement in the others; and for single schedules, there are a host of increasing monotonic relations between the rate of responding and the rate of reinforcement. All these results, plus several others, can be accounted for by a coherent system of equations, the most general of which states that the absolute rate of any response is proportional to its associated relative reinforcement.

Content maybe subject to copyright    Report

Citations
More filters
Book
01 Jan 1988
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Abstract: Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

37,989 citations


Cites background from "On the law of effect"

  • ...Thorndike later modified the law to better account for accumulating data on animal learning (such as differences between the effects of reward and punishment), and the law in its various forms has generated considerable controversy among learning theorists (e.g., see Gallistel, 2005; Herrnstein, 1970; Kimble, 1961, 1967; Mazur, 1994)....

    [...]

Journal ArticleDOI
TL;DR: In this article, KlUGER and Denisi analyzed all the major reasons to reject a paper from the meta-analysis, even though the decision to exclude a paper came at the first identification of a missing inclusion criterion.
Abstract: the total number of papers may exceed 10,000. Nevertheless, cost consideration forced us to consider mostly published papers and technical reports in English. 4 Formula 4 in Seifert (1991) is in error—a multiplier of n, of cell size, is missing in the numerator. 5 Unfortunately, the technique of meta-analysis cannot be applied, at present time, to such effects because the distribution of dis based on a sampling of people, whereas the statistics of techniques such as ARIMA are based on the distribution of a sampling of observations in the time domain regardless of the size of the people sample involved (i.e., there is no way to compare a sample of 100 points in time with a sample of 100 people). That is, a sample of 100 points in time has the same degrees of freedom if it were based on an observation of 1 person or of 1,000 people. 258 KLUGER AND DENISI From the papers we reviewed, only 131 (5%) met the criteria for inclusion. We were concerned that, given the small percentage of usable papers, our conclusions might not fairly represent the larger body of relevant literature. Therefore, we analyzed all the major reasons to reject a paper from the meta-analysis, even though the decision to exclude a paper came at the first identification of a missing inclusion criterion. This analysis showed the presence of review articles, interventions of natural feedback removal, and papers that merely discuss feedback, which in turn suggests that the included studies represent 1015% of the empirical FI literature. However, this analysis also showed that approximately 37% of the papers we considered manipulated feedback without a control group and that 16% reported confounded treatments, that is, roughly two thirds of the empirical FI literature cannot shed light on the question of FI effects on performance—a fact that requires attention from future FI researchers. Of the usable 131 papers (see references with asterisks), 607 effect sizes were extracted. These effects were based on 12,652 participants and 23,663 observations (reflecting multiple observations per participant). The average sample size per effect was 39 participants. The distribution of the effect sizes is presented in Figure 1. The weighted mean (weighted by sample size) of this distribution is 0.41, suggesting that, on average, FI has a moderate positive effect on performance. However, over 38% of the effects were negative (see Figure 1). The weighted variance of this distribution is 0.97, whereas the estimate of the sampling error variance is only 0.09. A potential problem in meta-analyses is a violation of the assumption of independence. Such a violation occurs either when multiple observations are taken from the same study (Rosenthal, 1984) or when several papers are authored by the same person (Wolf, 1986). In the present investigation, there were 91 effects derived from the laboratory experiments reported by Mikulincer (e.g., 1988a, 1988b). This raises the possibility that the average effect size is biased, because his studies manipulated extreme negative FIs and used similar tasks. In fact, the weighted average d in Mikulincer's studies was —0.39; whereas in the remainder of the

5,126 citations


Cites background or result from "On the law of effect"

  • ...…form of FIs—increase performance (Arps, 1920; Book & Norvell, 1922; Brand, 1905; Brown, 1932; Crawley, 1926; Elwell & Grindley , 1938-1939; Gates, 1917; Gilliland, 1925; Johanason, 1922; Jones, 1910; Judd, 1905; Manzer, 1935; Smith, 1933; Spencer, 1923; Thorndike, 1927; Waters, 1933; Wright, 1906)....

    [...]

  • ...…affected performance, while actually manipulating expectations and never manipulating KR. Parenthetically, other researchers manipulated KR but labeled their manipulations as consciousness of habit (Judd, 1905), reward and punishment (Thorndike, 1927), and incentive (Crawley, 1926; Wright, 1906)....

    [...]

  • ...Whereas several reports were empirically consistent with these predictions (e.g., Thorndike , 1927), the law of effect was never sufficiently detailed to account for the inconsistent findings....

    [...]

Journal ArticleDOI
John Gibbon1

2,072 citations

Journal ArticleDOI
J. L. Evenden1
TL;DR: Evidence for varieties of impulsivity from several different areas of research, including human psychology, psychiatry and animal behaviour, suggests that several neurochemical mechanisms can influence impulsivity, and that impulsive behaviour has no unique neurobiological basis.
Abstract: The concept of impulsivity covers a wide range of ”actions that are poorly conceived, prematurely expressed, unduly risky, or inappropriate to the situation and that often result in undesirable outcomes”. As such it plays an important role in normal behaviour, as well as, in a pathological form, in many kinds of mental illness such as mania, personality disorders, substance abuse disorders and attention deficit/hyperactivity disorder. Although evidence from psychological studies of human personality suggests that impulsivity may be made up of several independent factors, this has not made a major impact on biological studies of impulsivity. This may be because there is little unanimity as to which these factors are. The present review summarises evidence for varieties of impulsivity from several different areas of research: human psychology, psychiatry and animal behaviour. Recently, a series of psychopharmacological studies has been carried out by the present author and colleagues using methods proposed to measure selectively different aspects of impulsivity. The results of these studies suggest that several neurochemical mechanisms can influence impulsivity, and that impulsive behaviour has no unique neurobiological basis. Consideration of impulsivity as the result of several different, independent factors which interact to modulate behaviour may provide better insight into the pathology than current hypotheses based on serotonergic underactivity.

1,844 citations

Journal ArticleDOI
TL;DR: A concurrent operants paradigm was used to compare the Pace et al. (1985) procedure with a modified procedure wherein clients were presented with two stimuli simultaneously and were given access only to the first stimulus approached, resulting in greater differentiation among stimuli.
Abstract: The development of effective training programs for persons with profound mental retardation remains one of the greatest challenges for behavior analysts working in the field of developmental disabilities. One significant advancement for this population has been the reinforcer assessment procedure developed by Pace, Ivancic, Edwards, Iwata, and Page (1985), which involves repeatedly presenting a variety of stimuli to the client and then measuring approach behaviors to differentiate preferred from nonpreferred stimuli. One potential limitation of this procedure is that some clients consistently approach most or all of the stimuli on each presentation, making it difficult to differentiate among these stimuli. In this study, we used a concurrent operants paradigm to compare the Pace et al. (1985) procedure with a modified procedure wherein clients were presented with two stimuli simultaneously and were given access only to the first stimulus approached. The results revealed that this forced-choice stimulus preference assessment resulted in greater differentiation among stimuli and better predicted which stimuli would result in higher levels of responding when presented contingently in a concurrent operants paradigm.

1,507 citations

References
More filters
Book
B. F. Skinner1
01 Jan 1953
TL;DR: The psychology classic "Walden Two" as mentioned in this paper is a detailed study of scientific theories of human nature and the possible ways in which human behavior can be predicted and controlled from one of the most influential behaviorists of the twentieth century.
Abstract: The psychology classic-a detailed study of scientific theories of human nature and the possible ways in which human behavior can be predicted and controlled-from one of the most influential behaviorists of the twentieth century and the author of Walden Two. "This is an important book, exceptionally well written, and logically consistent with the basic premise of the unitary nature of science. Many students of society and culture would take violent issue with most of the things that Skinner has to say, but even those who disagree most will find this a stimulating book." -Samuel M. Strong, The American Journal of Sociology "This is a remarkable book-remarkable in that it presents a strong, consistent, and all but exhaustive case for a natural science of human behavior...It ought to be...valuable for those whose preferences lie with, as well as those whose preferences stand against, a behavioristic approach to human activity." -Harry Prosch, Ethics

8,325 citations

Journal ArticleDOI
TL;DR: Most of the rat investigations, which I shall report, were carried out in the Berkeley laboratory, and a few, though a very few, were even carried out by me myself.
Abstract: I shall devote the body of this paper to a description of experiments with rats. But I shall also attempt in a few words at the close to indicate the significance of these findings on rats for the clinical behavior of men. Most of the rat investigations, which I shall report, were carried out in the Berkeley laboratory. But I shall also include, occasionally, accounts of the behavior of non-Berkeley rats who obviously have misspent their lives in out-of-State laboratories. Furthermore, in reporting our Berkeley experiments I shall have to omit a very great many. The ones I shall talk about were carried out by graduate students (or underpaid research assistants) who, supposedly, got some of their ideas from me. And a few, though a very few, were even carried out by me myself. Let me begin by presenting diagrams for a couple of typical mazes, an alley maze and an elevated maze. In the typical experiment a hungry rat is put at the entrance of the maze (alley or elevated), and wanders about through the various true path segments and blind alleys until he finally comes to the food box and eats. This is repeated (again in the typical experiment) one trial every 24 hours and the animal tends to make fewer and fewer errors (that is, blindalley entrances) and to take less and less time between start and goal-box until finally he is entering no blinds at all and running in a very few seconds from start to goal. The results are usually presented in the form of average curves of blind-entrances, or of seconds from start to finish, for groups of rats. All students agree as to the facts. They disagree, however, on theory and explanation. (1) First, there is a school of animal psychologists which believes that the maze behavior of rats is a matter of mere simple stimulus-response connections. Learning, according to them, consists in the strengthening of some of these connections and in the weakening of others. According to this ‘stimulus-response’ school the rat in progressing down the maze is helplessly responding to a succession of external stimulisights, sounds, smells, pressures, etc. impinging upon his external sense organs-plus internal stimuli coming from the viscera and from the skeletal muscles. Figure 1: Plan of maze 14-Unit T-Alley Maze. (From M.H. Elliot, The effect of change of reward on the maze performance of rats. University of California Publications in Psychology, 1928, 4, 20.)

5,735 citations

Journal ArticleDOI

4,181 citations

Book
01 Jan 1938

3,337 citations

Book
01 Jan 1957

3,148 citations


"On the law of effect" refers methods in this paper

  • ...The Skinner Box The “Skinner box” was designed as a ___________________ ___________________ with responding mechanisms and consequence delivery 7...

    [...]