Persistence and Fadeout in the Impacts of Child and Adolescent Interventions

TL;DR: It is argued that skill-building interventions should target “trifecta” skills—ones that are malleable, fundamental, and would not have developed eventually in the absence of the intervention.

Abstract: Many interventions targeting cognitive skills or socioemotional skills and behaviors demonstrate initially promising but then quickly disappearing impacts. Our paper seeks to identify the key features of interventions, as well as the characteristics and environments of the children and adolescents who participate in them, that can be expected to sustain persistently beneficial program impacts. We describe three such processes: skill-building, foot-in-the-door and sustaining environments. We argue that skill-building interventions should target "trifecta" skills - ones that are malleable, fundamental, and would not have developed eventually in the absence of the intervention. Successful foot-in-the-door interventions equip a child with the right skills or capacities at the right time to avoid imminent risks (e.g., grade failure or teen drinking) or seize emerging opportunities (e.g., entry into honors classes). The sustaining environments perspective views high quality of environments subsequent to the completion of the intervention as crucial for sustaining early skill gains. These three perspectives generate both complementary and competing hypotheses regarding the nature, timing and targeting of interventions that generate enduring impacts.

Figures

Figure 1. IQ impacts in Perry and Abecederian.

Table 1. Possible “trifecta” skills.

Figure 3. Fundamentality and malleability in skills, behaviors, and beliefs.

Figure 4. Math achievement during and after the pre-K building blocks program.

Figure 2. Cognitive impacts in 67 ECE studies.

Full text

UC Irvine
UC Irvine Previously Published Works
Title
Persistence and Fadeout in the Impacts of Child and Adolescent
Interventions.
Permalink
https://escholarship.org/uc/item/1kn271xs
Journal
Journal of research on educational effectiveness, 10(1)
ISSN
1934-5747
Authors
Bailey, Drew
Duncan, Greg J
Odgers, Candice L
et al.
Publication Date
2017
DOI
10.1080/19345747.2016.1232459
Peer reviewed
eScholarship.org Powered by the California Digital Library
University of California

Persistence and Fadeout in the Impacts of Child and Adolescent
Interventions
Drew Bailey
a
, Greg J. Duncan
a
, Candice L. Odgers
b
, and Winnie Yu
c
ABSTRACT
Many interventions targeti ng cognitive skills or socioemotion al skills and
behaviors demonstrate initially promising but then quickly disappearing
impacts. Our article seeks to iden tify the key features of interven tions, as
well as the characteristics and environments of the children and
adolescents who participate in them, that can be expected to sustain
persistently beneficial program impacts. We describe three such processes:
skill-building, foot-in-the-door and sustaining environments. We argue that
skill-building interventions should target “trifecta” skills—ones that are
malleable, fundamental, and would not have developed eventually in the
absence of the intervention. Successful foot-in-the-door interventions
equip a child with the right skills or capacities at the rig ht time to avoid
imminent risks (e.g., grade failure or teen drinking) or seize emerging
opportuniti es (e.g., entry into honors classes). The sustaining environments
perspective views high quality of environments subsequent to the
completion of the intervention as crucial for sustaining earlier skill gains.
These three perspectiv es generate both complementa ry and competing
hypotheses regarding the nature, timing, and targeting of interventions
that ge nerate enduring impacts.
KEYWORDS
interventions
fadeout
methodology
Far too often, impacts on outcomes targeted by intervention designers soon disappear. This
is readily apparent in interventions begun in early childhood, with perhaps the most famous
example being Perry Preschool, where the program’s large end-of-treatment impact on IQ
(.75 SD) at age 5 had dropped to a statistically insignificant .08 SD by age 8 (Schweinhart
et al. 2005; Figure 1). More generalizable—and worrisome—is the finding by Puma et al.
(2012), based on a random assignment of 4,442 children to a national sample of Head Start
centers, of noteworthy impacts at the end of the Head Start year, but virtually no statistically
significant impacts on any cognitive skill or socioemotional skill or behavior over the next
several years. On the other hand, a second famous early childhood intervention begun a
decade after Perry—the Abecedarian Project —generated IQ impacts that persisted well
beyond age 8 (Campbell, Pungello, Miller-Johnson, Burchinal, & Ramey, 2001; also shown
in Figure 1). Both Perry and Abecedarian produced substantial favorable impacts in adult-
hood, although not always on the same outcomes.
CONTACT Greg J. Duncan gduncan@uci.edu School of Education, University of California–Irvine, 2001 Education,
Irvine, CA 92697-5500, USA.
a
University of California–Irvine, Irvine, California, USA
b
Duke University, Durham, North Carolina, USA
c
Orange, California, USA
Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/uree.
© 2017 Taylor & Francis Group, LLC
JOURNAL OF RESEARCH ON EDUCATIONAL EFFECTIVENESS
2017, VOL. 10, NO. 1, 7–39
http://dx.doi.org/10.1080/193 45747.2016.1232459

Examining this and other seemingly contradictory evidence on fadeout, we seek to iden-
tify the key features of child and adolescent interventions, as well as the characteristics and
environments of their participants, that can be expected to generate persistent program
impacts. We will speak of impacts on “skills” but use that term broadly to encompass any
skill, behavior, capacity or psychological resource that helps individuals attain successful
outcomes.
1
With some disciplines most comfortable with the term “noncognitive skill” and
others most comfortable with some variant of “socioemotional skill or behavior,” we have
opted to use the two sets of terms interchangeably.
We integrate findings from across multiple disciplines by using a very broad definition of
skills, but where appropriate, provide examples of the specific socioemotional, behavioral, or
cognitive skill targeted by the intervention. We consider interventions that are quite diverse
in terms of their setting (both within and outside of classrooms), timing (encompassing vari-
ous stages of childhood and adolescence), and populations (mostly at-risk children and ado-
lescents in the United States but with some interventions offered to both at-risk and more
advantaged populations).
We begin in the section “Patterns of Fadeout and Persistence” with a selective review of
evidence on fadeout, choosing our examples to illustrate the diverse patterns of fadeout
across outcomes within and across interventions. We then formulate three distinct processes
that might sustain benefits for children and adolescents: skill building, foot-in-the-door skill
or capacity boosts, and sustaining environments.
As detailed in “Skill-Building Models,” the skill-building perspective undergirds most
cognitive theories of math and literacy learning. One version of it has been formalized in
economists’ human capital model of the skill accumulation process. Key to skill building is
that simpler skills support the learning of more sophisticated ones and, in the economic
models, that skills acquired prior to a given skill- or capacity-building intervention increase
the productivity of that investment.
Our main contribution here is to argue for the importance in this skill-building perspec-
tive of what we call “ trifecta” skills—ones that are malleable, fundamental, and would not
Figure 1. IQ impacts in Perry and Abecederian.
1
We do not concentrate on adult health outcomes because the physiological processes linking experiences in childhood, par-
ticularly early childhood, to adult health are only just beginning to be understood (Center on the Developing Child, 2010).
8 D. BAILEY ET AL.

have developed in the absence of the intervention. We argue that all three conditions are
needed to generate long-run effects, which limits substantially the kinds of interventions
that might be expected to produce persistent benefits to children and adolescents. In the
case of early childhood interventions, the third trifecta condition—eventual skill develop-
ment in counterfactual conditions—is particularly problematic for interventions that build
early literacy or math skills because most children are likely to eventually acquire at least
minimal levels of these skills soon after entering school. Indeed, much of the “fadeout”
effects of early childhood interventions have been attributed to this type of catch-up among
the larger population of children.
As explained in the section “Foot-in-the-Door Interventions,” developmental timing is
key to the foot-in-the-door perspective. Successful foot-in-the-door interventions equip a
child with the right skills or capacities at the right time to avoid imminent risks (e.g., grade
failure, teen drinking, or teen childbearing) or to seize emerging opportunities (e.g., entry
into honors classes, SAT prep). The skill or capacity boosts need not be permanent, as with
SAT prep that boosts chances of acceptance into a higher resourced college, a key step in a
positive cascade that might influence human capital and labor market outcomes. For SAT
prep, it is the enriched college resources, rather than any lingering test prep knowledge, that
leads to a higher paying job.
A third approach to understanding fadeout is what we call the “sustaining environments”
perspective (fifth section). It recognizes the importance of interventions that build important
skills and capacities, but views the quality of environments subsequent to the completion of
the intervention as crucial for maintaining initial skill advantages. The final section summa-
rizes some of the implications of our analysis.
Patterns of Fadeout and Persistence
Original calculations, using information from a meta-analytic database of the ev alua-
tions of 67 high-quality early childhood education (ECE) interventions published
between 1960 and 2007, produc e the pattern of geometrically declining effect sizes
shown in Figure 2 for cognitive outcomes.
2
At the end of the programs, effect sizes
averaged .23 standard deviations—considerably smaller than the end-of-treatment
impacts shown for Perry and Abe cedarian in Figure 1. Posttreatment impacts measured
no more than 12 months after the end of treatment had dropped by more than half, to
.10 SD, and again by half one to two years later. Figure 1 shows that a lthough Perry’s
IQ impacts approximate a geom etric decline, Abecedarian ’s IQ impacts were much
more persistent (although they did decline substantially during the treatment period),
2
Themeta-analyticdatabaseistheproductoftheNationalForumonEarlyChildhoodPolicyand Programs (http://developingchild.harvard.
edu/initiatives/forum/) based on a comprehensive search of the literature from 1960 to 2007, when the coding project began.
Studies had to have a treatment and control/comparison group, rather than simply assessing the growth of one group of chil-
dren over time. Early childhood education programs were defined as structured, center-based early childhood education clas-
ses, day care with some educational component, or center-based child care. These include full preschool programs such as
Head Start and other interventions conducted by researchers. Programs included were required to have provided services to
children, their families, or staff at the program sites, and assessed program impacts on children’s cognitive and achievement
outcomes. About one third of the ECE studies used random assignment, with the remainder following quasi-experimental
designs such as change models, individual or family fixed-effects models, regression discontinuity, difference in difference,
propensity score matching, interrupted time series, instrumental variables, and some other types of matching. Studies that
used quasi-experimental designs must have had pretest and posttest information on the outcome or established baseline
equivalence of groups on demographic characteristics determined by a joint test.
PERSISTENCE AND FADEOUT IN INTERVENTION IMPACTS 9

which suggests that fadeout patterns based on cross-study average i mpacts are likely to
conceal study-to-study variation.
Most interventions targeting children’s cognitive, social, or emotional development
fail to follow their subjects beyond the end of their program s (e.g., Durlak, Weissberg,
Dymnicki, Taylor, & Schellinger, 2011; Smit, Verdurmen, Monshouwer, & Smit, 2008).
3
When they do, complete fadeout is common. As mentioned above, Puma et al. (2012)
found v irtually no statistically significant impacts of Head Start on e ither cognitive or
noncognitive measures in kindergarten, first, or second grades. That said, Deming’s
(2009) sibling-based analysis shows that although initial impacts of Head Start on
achievement in the early grades had faded t o statistical i nsignificance by early adoles-
cence, a number of significant differences in attainment and behavioral domains were
detected in early adulthood.
A number of mathematics interventions for preschool or school-aged children also gener-
ate impressive initial effects that have been found to fade over time (Clements, Sarama,
Wolfe, & Spitler, 2013; Smith, Cobb, Farran, Cordray, & Munter, 2013). Bus & van
IJzendoorn’s(1999) meta-analysis of early phonological awareness training found substan-
tial effects on children’s initial reading skills (.44 SD) but much smaller effects on reading
skills (.16 SD) in the subset of studies with a follow-up assessment 18 months, on average,
after the completion of the programs. Unfortunately, none of these studies include longer
term follow-up information.
In some long-run studies such as Perry and Abecedarian, initial fadeout is followed by the
detection of impacts in adulthood, although not always on the same kinds of developmental
outcomes. In the case of teacher effects, Jacob, Lefgren, and Sims (2010) conclude that
teacher-induced (value-added) learning and other measures of teacher quality have low per-
sistence, with three quarters or more of teaching-year effects fading out within one year.
However, Chetty, Friedman, and Rockoff (2014) found longer run impacts on both
Figure 2. Cognitive impacts in 67 ECE studies.
3
In the meta-analytic database used in Figure 2, only about one third of the studies followed subjects beyond the end of
treatment. In a meta-analysis of adolescent alcohol using RCT designs, only 3 of 18 studies reported on long-term effects
(>48 months; Smit et al. (2008). In the case of prevention of depressive symptoms, only 12 of 30 studies collected data past
six months depression (Horowitz & Garber, 2006).
10 D. BAILEY ET AL.

Frequently asked questions

Q1. What are the contributions mentioned in the paper "Persistence and fadeout in the impacts of child and adolescent interventions" ?

The authors describe three such processes: skill-building, foot-in-the-door and sustaining environments. 

Q2. How do the authors encourage intervention design to extend beyond the typical fadeout window?

The authors also encourage intervention designs that allow for randomization to high-quality sustaining environments and for program evaluation plans to extend beyond the typical fadeout window of 12 months so that the reemergence of intervention effects via foot-in-the-door mechanisms can be more rigorously tested. 

Q3. Why is general intelligence the example of a “fundamental” capacity?

4Because it supports performance across a wide variety of important tasks, general intelligence, or g, is perhaps the best example of a “fundamental” capacity. 

Q4. What is the key implication of the Cunha and Heckman model?

The key intervention implication in this skill-building model is the need to identify fundamental cognitive and noncognitive skills, capacities, behaviors, or beliefs and develop them as early and efficiently as possible. 

Q5. What are the characteristics that are promising to target with interventions?

Lacking both malleability and fundamentality, it exemplifies the least promising kinds of characteristics to target with interventions. 

Q6. What is the important finding for their focus on impact persistence?

Most important for their focus on impact persistence, Dodge et al. (2015) found that test score impacts appearing in third grade were sustained through at least fifth grade. 

Q7. How long did Smit et al. (2008) report on the effects of Head Start?

In a meta-analysis of adolescent alcohol using RCT designs, only 3 of 18 studies reported on long-term effects (>48 months; Smit et al. (2008). 

Q8. What were the requirements for the evaluations of the programs?

Programs included were required to have provided services to children, their families, or staff at the program sites, and assessed program impacts on children’s cognitive and achievement outcomes. 

Q9. How many years of algebra courses do colleges require?

Most colleges require successful completion of three years of math courses in high school, and the more competitive colleges require four. 

Q10. How did Cortes and Goodman (2014) determine the effect of the extra algebra instruction on students’?

Using a regression discontinuity design, Cortes and Goodman (2014) estimated that children just below the cutoff who received the extra algebra instruction earned higher grades in ninth-grade algebra, outperformed controls on an 11th-grade mathematics exam, were 12 percentage22 D. BAILEY ET AL. 

Q11. What is the potential list of trifecta skills for children growing up facing?

the potential list of trifecta skills is likely to broaden for children growing up facing extreme forms of poverty and adversity. 

Q12. What is the probability that a child will learn more advanced skills?

if the probability is close to 100% that the child will learn more advanced skills or enjoy more positive environments if he or she has learned a precursor skill or been placed in a positive environment, foot-in-the-door processes may fully sustain initial impacts. 

Q13. What is the difference between skill building and foot-in-the-door?

The sensitive-period feature of foot-in-the-door processes differs fundamentally from skill building, which views the intervention task as one of identifying and improving key skills (e.g., grit, executive function, gratification delay, early numeracy) that will persist and generate lifelong benefits. 

Q14. What are the potential targets of interventions later in development?

Examples of potential targets of interventions later in development are listed in the top panel of Table 1 and include vocational skills, an understanding of fractions or algebra, vocabulary, or background knowledge that substantially exceeds typical levels. 

Q15. What is the strategy of focusing on skills, behaviors, or beliefs for disadvantaged children and?

The strategy of focusing on such skills, behaviors, or beliefs for disadvantaged children and adolescents is implicit in interventions such as Fast Track, double-dose algebra, and intensive tutoring programs aimed atPERSISTENCE AND FADEOUT IN INTERVENTION IMPACTS 29struggling readers.