Postprint
1
Department of Communication, Cornell University, Ithaca, NY, USA
2
Department of Medicine, Division of General Internal Medicine, Weill Cornell Medical College, New York, NY, USA
We would like to thank our team of research assistants, Laura Bagwell, Anjuli Fink, Lucus Tse, and Caiwei Zhu, for their
time and efforts in coding the articles for this manuscript. We would also like to thank Cornell University librarian Amelia
Kallaher for the guidance provided in developing the literature search strategy for this review.
We also thank Patrick Forscher, Bobbie Spellman, and two anonymous reviewers for their helpful comments on an earlier
version of this manuscript and Brenton Wiernik for this post print template.
Corresponding Author:
Neil Lewis, Jr., 467 Mann Library Building, Cornell University, Ithaca, New York, NY14853, USA.
Email: nlewisjr@cornell.edu.
Do We Report the Information
that is Necessary to Give
Psychology Away?
A Scoping Review of the Psychological
Intervention Literature 2000-2018
Bharathy Premachandra
1
and Neil A. Lewis, Jr.
1,2
Abstract
Psychologists are spending a considerable amount of time researching and developing interventions, in
hopes that our efforts can help to tackle some of society’s pressing problems. Unfortunately, those
hopes are often not realized—many interventions are developed and reported in our journals but do
not make their way into the broader world they were designed to change. One potential reason for
this is that there may be a gap between the information reported in our papers, and the information
others, such as practitioners, need to implement our findings. We explored this possibility in the
current paper. We conducted a scoping review to assess the extent to which the information needed
for implementation is reported in psychological intervention papers. Results suggest psychological
intervention papers report, at most, 64% of the information needed to implement interventions. We
discuss the implications of this for both psychological theories and applying them in the world.
Keywords
meta-science, psychological interventions, implementation, scaling, social cognition
Perspectives on Psychological Science
XX–XX
© The Author(s) 2020
Not the version of record.
2 Perspectives on Psychological Science XX(X)
“I get tired of research that never gets used…the question is when will we take all that energy and
all the wonderful things that you do and make it work for the people"
-Congressman Elijah Cummings, Society for Research on Child Development, 2019
One of psychology’s longest re-occurring calls is to do whatever we can to “give psychology
away” (Forscher, Vazire, & Anvari, 2020; Miller, 1969). In the 1940s, Kurt Lewin encouraged re-
searchers to engage in “action research,” particularly that which could improve intergroup rela-
tions (Lewin, 1946). In the 1990s, there were discussions about the role psychological research
could and/or should have in shaping our laws (Ellsworth, 1991). And most recently, in the 2010s,
there were calls for governments to develop councils of psychological advisers, similar to the
Council of Economic Advisors, an agency within the executive office of the President of the United
States, who could help to ensure that the policies we set are psychologically informed (Schwartz,
2012; Sunstein, 2016; Teachman, Norton, & Spellman, 2015). The rationale for this most recent
push is that psychologists have done so much intervention research that our expertise could osten-
sibly be leveraged to cultivate a healthier, happier, and more sustainable world (Gruber, Saxbe,
Bushman, Mcnamara, & Rhodes, 2019). On their face, these calls seem like a great idea. Psycholo-
gists have long conducted intervention research that can speak to and potentially help to address a
variety of pressing social problems (Suarez-Balcazar, Balcazar, & Fawcett, 1992).
In practice, however, it is presently unclear whether the decades of intervention research
psychologists and other social scientists have conducted is, in fact, ready for broader implementa-
tion and scaling (Goroff, Lewis, Scheel, Scherer, & Tucker, 2018). For psychological interventions to
have the societal impacts we aspire to have as a field (Teachman et al., 2015), psychologists
must—at the very least—report the information that practitioners wanting to implement our re-
search would need to implement it successfully. Practitioners would need to know: how differ-
ences in social contexts influence intervention efficacy; precisely what types of independent and
dependent variables have been examined and what that means for implementation and scaling ef-
ficacy; how attitudes, goals, or identity factors influence people’s responses to interventions; how
dosage and the timing of the intervention matter for uptake and maintenance; how the source or
other intervention agents of change (e.g., doctors, teachers, government officials) influence the effi-
cacy of the intervention, and much more (Earl & Lewis, 2019; Goroff et al., 2018). Without knowing
and reporting these things, well-intentioned interventions designed to improve outcomes may
have many unintended, life-altering consequences for the people being intervened on. Moreover,
policymakers and practitioners whose job it is to decide how to intervene, need to know what to
expect, how to plan, and forecast returns on intervention investments. These are necessary steps
in making decisions regarding the adoption, implementation, and scaling of interventions.
Unfortunately, psychologists have traditionally been ill-equipped to take on the challenge of
translating insights from our studies into effective interventions that can be brought to scale
(Yeager, et al., 2016). We often conduct our research in laboratory settings (Baumeister, Vohs, &
Premachandra and Lewis 3
Funder, 2007; Sears, 1986) with samples that are not representative of the broader world to which
we wish to generalize (Henrich, Heine, & Norenzayan, 2010; Lewis, 2019; Simons, Shoda, & Lind-
say, 2017). We are also not well-versed in considering and reporting the costs and benefits of im-
plementing interventions in a resource-constrained world (Forscher et al., 2020; Sunstein., 2015).
These may not be big problems for research aiming to make incremental advances to abstract the-
oretical propositions, but it is a problem in the realm of intervention research: it is extremely rare
for interventions that change outcomes in the laboratory to translate to the world outside of the
lab with high enough fidelity to be practically meaningful (DeAngelis, 2010). As Yeager and col-
leagues (2016) noted, “although promising, self-administered psychological interventions have not
often been tested in ways that are sufficiently relevant for policy and practice” (p. 375). This con-
tributes to the substantial gap that exists between research and practice (Institute of Medicine
(US) Committee on Health and Behavior: Research, 2001).
To address this gap, researchers and practitioners have developed a series of frameworks
and guidelines for conducting and reporting research in ways that would make it possible to use
the research for practice. One such framework is the Reach, Efficacy, Adoption, Implementation, and
Maintenance (RE-AIM) framework that was originally developed by Glasgow and colleagues
(1999) to provide guidance on how to conduct research in this way in the field of public health.
Since the first version in 1999, RE-AIM has evolved considerably in its usage over the past 20 years
(planning, reporting, reviews) and applied in various fields (e.g. environmental change, health pol-
icy, ageing, childcare, quality improvement) and in clinical, community, and corporate settings
(Glasgow, et al., 2019; Gaglio, Shoup, & Glasgow, 2013). RE-AIM outlines multi-level (individual
and setting) and multi-stage indicators (planning, evaluation, reporting) that are essential to con-
sider to achieve sustained population-level effectiveness of interventions (Stenhouse, 2017; Gaglio
et al., 2013). Reach assesses whether the sample being studied is representative of the target popu-
lation of interest by examining the number, proportion, and representativeness of the participants
in a study. Efficacy assesses the impacts of the intervention on participants by considering factors
like effects on quality of life, economic impact, as well as negative effects. Adoption assesses the fea-
sibility of translating between the setting of the intervention study and the setting(s) in which the
intervention would actually be disseminated, and considers factors like the number, proportion,
and representativeness of stakeholders who would be willing to implement the intervention. Im-
plementation assesses factors related to implementation fidelity: how closely must implementers
adhere to the original protocol to achieve the same outcomes as the original researchers and how
much time and money would that take. Finally, Maintenance assesses factors associated with long-
term follow up (to the extent that it is desirable); it focuses on how long researchers followed-up
with participants and organizations to determine long-term impact.
Current Study
In the current study, we used the guidelines described above from the implementation sci-
ence literature to assess the extent to which (a sample of ) the psychological intervention literature
provides the information required for successful implementation, herein referred to as
4 Perspectives on Psychological Science XX(X)
implementation information. To do this, we conducted a scoping review of, and used an implemen-
tation science inspired checklist to code, articles published in the past two decades about five cate-
gories of psychological interventions that were designed to drive social change in a variety of do-
mains, and have become well-known in public discourse: belonging (Walton & Cohen, 2011),
growth mindset (Aronson, Fried, & Good, 2002), utility-value (Hulleman & Harackiewicz, 2009),
self-affirmation (Cohen, Garcia, Apfel, & Master, 2006), identity-based motivation (Oyserman,
2015), as well as interventions that combined elements of these. We chose these interventions be-
cause they have been studied in a multitude of settings by researchers across various sub-disci-
plines in psychology (e.g. educational, social, organizational, clinical), as well as in neighboring dis-
ciplines (e.g., business, communication, education, and public health). As such, we are biasing the
sample of articles in our scoping review toward articles about interventions that have been thor-
oughly studied and tested, and therefore should be the most “shovel ready” for dissemination and
implementation. To maximize the chances of implementation readiness of the interventions in the
articles we coded, we opted to include articles from the most recent years of research, the years
2000-2018, allowing some time after the initial stages of intervention creation to get them closer
to dissemination readiness. While these interventions were conducted in various settings to im-
prove specific outcomes for different target populations, to give readers some familiarity with the
general concepts behind each intervention, Table 1 provides a descriptive overview of how the in-
terventions were applied in educational settings to improve student outcomes.
Table 1. Descriptive Overview of the Five Interventions.
Intervention name
Intervention Description
Self-Affirmation
Self-affirmation interventions are “designed to specifically target self-confidence in an attempt to
find methods that lead to higher test scores. This self-affirmation exercise is aimed to function as a
‘‘catalyst’’ that boosts students’ self-confidence and self-integrity while allowing their abilities to be
unencumbered, translating, theoretically, into better performance (Purdie-Vaughns et al. 2009 as
cited in Bratter, Rowley, & Chukhray, 2016).
Identity-based mo-
tivation
IBM interventions are used to improve student outcomes by helping “students imagine school as
the path to their future, generate strategies to succeed on that path, and see obstacles and fail-
ures along the way as signaling importance and value” (Horowitz, Sorensen, Yoder, & Oyserman,
2018, p. 12)
Growth Mindset
Growth mindset interventions “aim to convince students that rather than being fixed and finite,
intelligence is malleable, and one can become smarter and more successful in school by working
harder” (Broda, et al., 2018, p. 319)
Utility Value
“The UV intervention targets different psychological processes critical to student achievement:
perceived value of and engagement in coursework. It is a curricular intervention in which students
write short essays about the personal relevance of course material.”
(Harackiewicz et al.,2015, p.3)
Belonging
Belonging interventions “aim to help disadvantaged students reframe worries they may have
about fitting in as normal, rather than as reinforcement of societal and institutional signals that
they do not belong or are unable to succeed.” (Broda et al.,2018, p. 319)
Premachandra and Lewis 5
We had four primary descriptive goals in this endeavor: first, to get initial estimates of the state of
implementation information available in the literature on psychological interventions; second, to
determine the types of implementation information that are typically published in the psychologi-
cal intervention literature; third, to identify gaps between implementation information that is pub-
lished in the psychological intervention literature and the information required for practitioners to
implement them; and fourth, to discuss the implications of any gaps that exist for both theory and
practice. To be clear, the goal of the current paper is to begin to describe the state of implementa-
tion information in published psychology research on social interventions; descriptive research is,
in its own right, essential for the advancement of our science (Rozin, 2009). The goal is not to com-
pare or rank sub-fields or researchers. Because our goal is description rather than comparison, we
will employ the method of a scoping review rather than a systematic review or other meta-analytic
techniques.
Method
Brief Primer on Scoping Reviews
Scoping reviews provide an overview of topics or fields by systematically mapping large
bodies of literature. Unlike traditional systematic reviews that draw on literature pertaining to a
specific question, scoping reviews synthesize and analyze topic and/or field related research to
provide coherent insight into the conceptual state of research. See Table 2 for a brief overview of
the similarities and differences between scoping reviews and traditional systematic reviews.
Table 2. Differences between Scoping Reviews and Systematic Reviews (Arksey & O'Malley, 2005)
Scoping Reviews
Systematic Reviews
The goal is to provide a descriptive overview of the re-
viewed material as a whole, not by individual studies
The goal is to synthesize research question specific evi-
dence from reviewed studies
Includes material with a range of designs and methods
Heterogeneity in method and design is minimized for
meaningful synthesis.
Scoping reviews map existing literature in terms of volume, nature, and characteristics of
research. Scoping reviews are suited for topics that have not been extensively reviewed and are an
appropriate method for initial explorations regarding the extent, range, and nature of research ac-
tivity and allow scholars to identify gaps in the existing literature. Since our goal is to provide a de-
scriptive overview of the presence/absence of implementation information in psychological inter-
vention research, we required a systematic method that enabled a conceptual mapping of topic-
related research material that is not bound by specific research questions, designs, or methods;
scoping review is an ideal method for this kind of inquiry (Davis, Drey, & Gould, 2009).
