Are interventions theory-based? Development of a theory coding scheme.

TL;DR: The Theory Coding Scheme is an important methodological innovation, providing a research tool to reliably describe the theoretical base of interventions, inform evidence syntheses within literature reviews and meta-analyses, and stimulate the use of empirical data for theory development.

Abstract: Objective: Interventions are increasingly described as theory-based; however, the basis for this is often not clear. Advancing behavioral science requires a good understanding of how interventions are informed by, and test, theory. This study aims to develop a reliable method for assessing the extent to which behavioral interventions are theory-based. Design: The reliability, usability, and comprehensiveness of an initial coding scheme were improved in 13 iterative stages on the basis of its application to 29 papers, from a systematic review of interventions to promote physical activity and healthy eating. Results: The final Theory Coding Scheme contained 19 items, each with satisfactory interrater reliabilities, coding whether a theory or model was mentioned, how theories were used in intervention design, how intervention evaluations tested theory, and the implications of the results for future theory development. Conclusion: The Theory Coding Scheme is an important methodological innovation, providing a research tool to reliably describe the theoretical base of interventions, inform evidence syntheses within literature reviews and meta-analyses, and stimulate the use of empirical data for theory development.

Summary

Introduction

• There is growing recognition in the discipline of psychology, and the broader health services research and public health community (Campbell et al., 2000; Craig et al., 2008) , that the development and implementation of behaviour change technologies are enhanced by applying theory.
• Third, as well as using theory to inform intervention development, theory-based interventions can aid understanding of why interventions are effective or ineffective, thus facilitating an understanding of mechanisms of change and providing a basis for refining and developing better theory (Jemmott & Jemmott, 2000; Michie & Abraham, 2004) .
• Other reviews that cross several behavioural domains are Baban & Craciun (2007) and Webb & Sheeran (2006) .

Conceptualisation

• The starting point for the coding frame came from Ellis et al. (2003) who posited that theory-based interventions require the following; named "modifying factors", explanations as to how these factors will bring about change, methods to demonstrate changes in the modifying factors, and demonstration of how those changes contributed to behaviour change.
• The authors operationalised these concepts as three categories: whether the relevant theoretical constructs ("modifying factors") were targeted by intervention techniques, whether these constructs were measured and whether mediation effects were tested.

Initial development

• These 10 items reflected the three categories outlined in the conceptualisation section above.
• Specifically, items 1-5 related to whether the relevant theoretical constructs were targeted by intervention techniques, item 6-7 related to whether theoretical constructs were measured and items 8-10 referred to whether mediation effects were tested and how the intervention brought about behaviour change.
• Following discussions between the coders, some items were reworded and the response options were simplified to either yes/no or all/some/none.
• Definitions of theory (or model), theory-relevant construct, predictor and intervention technique were then generated.

Refinement

• The initial scheme was tested by four independent raters in a series of 13 iterative stages coding papers from a systematic review of interventions to increase healthy eating in the general population (Michie, Abraham, Whittington, & McAteer, 2007) .
• Ten of these stages involved two raters, one involved three and two involved one.
• The scheme was refined on the basis of discussion between the authors concerning comprehensiveness, clarity and ease of use, feedback from independent raters, and the inter-rater reliability of each item.
• The final modification was made following the comments of an anonymous reviewer, revising items 12 and 15 and adding items 13 and 19.

Theory-relevant construct:

• A construct (a key concept, excluding behaviour) within a theory/model upon which the intervention is based.
• A "Table of Theories" was included in the coding guidance to assist coders in identifying whether a particular construct belongs to the specified theory.
• Predictors were only coded if the author had presented evidence that the construct predicts/correlates with/causes behaviour.
• Predictors did not include actual behaviour, self-reported or otherwise (e.g. amount of time spent exercising), or biological factors (e.g., age, gender), or demographic factors (e.g.

Results

• Each item required a yes/no/don"t know response and the coder to identify the supporting evidence.
• The items are presented within the following six categories of coded items: reference to underpinning theory (items 1 to 3), targeting of relevant theoretical constructs (items 2, 5, 7-11), using theory to select recipients or tailor interventions (items 4 and 6), measurement of constructs (items 12 and 13), testing of mediation effects (items 14-18) and refining theory (item 19).
• These items assessed stated or suggested, rather than demonstrated, theoretical base.
• Often studies use a wide variety of behaviour change techniques in the hope that it will change something but this something is often unspecified and not linked to the theory upon which the intervention is stated to have been based.

Discussion

• The authors have developed a reliable method for assessing the extent to which theory has been applied to developing and evaluating interventions to change behaviour.
• The Theory Coding Scheme comprises 19 items covering a broad range of areas in which theory can play a role in psychological research.
• Specifically, it can provide a means for a more rigorous and systematic examination of the use of theory within intervention research than the approaches adopted in evidence syntheses of intervention effectiveness (e.g., Albarracin et al., 2005; Baban & Craciun, 2007; Trifiletti et al., 2005; Webb & Sheeran, 2006) .
• The CONSORT (Consolidated Standards for Reporting Trials) guidelines have been used to improve the way in which studies, particularly their methodologies, are conveyed to readers.
• In addition, demonstrating mediation statistically does not demonstrate a causal relationship between the mediator and the dependent measure; a better approach is to experimentally manipulate the proposed mediator, guided by the theory under test (Sigall & Mills, 1998) .

Stage Basis for development Development

• See text ("Initial Development") for further details.
• Coding response options modified to yes/no/don"t know for all items except those relating to measurement of, and change in, theoretical constructs (which were coded as all/some/none).
• Theory/constructs used to tailor intervention techniques to recipients; item 5 from Scheme 1 separated into further items; Support for theory stated with/without appropriate mediation; Results used to refine theory.
• Item linking techniques to constructs separated into 2 items.
• Greater emphasis at this stage on the explicit use of theory cited within the paper.

Targeted construct mentioned as predictor of behaviour

• ("Targeted" construct refers to a psychological construct that the study intervention is hypothesised to change).
• Evidence that the psychological construct relates to (correlates/predicts/causes) behaviour should be presented within the introduction or method (rather than the Discussion).
• Location of evidence that construct relates to behaviour: Location that this predictor is targeted by the intervention:.

Figures

Figure 1. Conceptual underpinnings of the coding scheme (numbers refer to coding scheme items)

Table 1: Overview of Coding Scheme Development

Figure 2. Diagrammatic Representation of Coding Scheme: numbers refer to items

Table 2.Overview of Theory Coding Scheme

Full text

promoting access to White Rose research papers
White Rose Research Online
eprints@whiterose.ac.uk
Universities of Leeds, Sheffield and York
http://eprints.whiterose.ac.uk/
This is an author produced version of a paper published in Health Psychology.
White Rose Research Online URL for this paper:
http://eprints.whiterose.ac.uk/76353/
Paper:
Michie, S and Prestwich, A (2010) Are interventions theory-based? Development
of a Theory Coding Scheme. Health Psychology, 29 (1). 1 - 8 (8).
http://dx.doi.org/10.1037/a0016939

1
APA Copyright Notice: This article may not exactly replicate the final version
published in the APA journal. It is not the copy of record. This is a preprint of an article
whose final and definitive form has been published in Health Psychology © 2010 American
Psychological Association DOI: 10.1037/a0016939; Health Psychology is available online
at http://www.apa.org/pubs/journals/hea/index.aspx . The final and definitive form of the
article can be found at http://psycnet.apa.org/journals/hea/29/1/1/
Running Head: CODING THEORY
Are interventions theory-based? Development of a Theory Coding Scheme
Susan Michie
Centre for Outcomes Research and Effectiveness, University College London, UK
Andrew Prestwich
Institute of Psychological Sciences, University of Leeds, UK
Author Note
We would like to thank Falko Sniehotta, Kali Demes, Ben Gardner, John McAteer
and Lizzie Rogers for their helpful input in developing the coding scheme and Robert
West, Mark Conner, Rebecca Lawton, and David French for helpful comments on an
earlier draft. Correspondence should be addressed to Professor Susan Michie, Centre
for Outcomes Research and Effectiveness, University College London, 1-19
Torrington Place, London WC1E 7HB, United Kingdom, email: s.michie@ucl.ac.uk
Tel. +44 (0) 207 679 5930; Fax: +44 (0) 207 916 8511.

2
Abstract
Objective
Interventions are increasingly described as theory-based; however, the basis for this is
often not clear. Advancing behavioural science requires a good understanding of how
interventions are informed by, and test, theory. This study aims to develop a reliable
method for assessing the extent to which behavioural interventions are theory-based.
Design
The reliability, usability, and comprehensiveness of an initial coding scheme were
improved in 13 iterative stages on the basis of its application to 29 papers from a
systematic review of interventions to promote physical activity and healthy eating.
Results
The final Theory Coding Scheme contained 19 items, each with satisfactory inter-
rater reliabilities, coding whether a theory or model was mentioned, how theories
were used in intervention design, how intervention evaluations tested theory and the
implications of the results for future theory development.
Conclusion
The Theory Coding Scheme is an important methodological innovation, providing a research
tool to reliably describe the theoretical base of interventions, inform evidence syntheses
within literature reviews and meta-analyses, and stimulate the use of empirical data for theory
development.
Key words: Theory, behaviour change, interventions

3
Introduction
There is growing recognition in the discipline of psychology, and the broader
health services research and public health community (Campbell et al., 2000; Craig et
al., 2008), that the development and implementation of behaviour change
technologies are enhanced by applying theory. Theory provides a common
description of what is known within an organising system. Within the behavioural
and social sciences, the term “theory” has been defined variously, but definitions tend
to share a core set of common ideas, elegantly expressed in the following:
“A theory presents a systematic way of understanding events or situations.
It is a set of concepts, definitions, and propositions that explain or predict
these events or situations by illustrating the relationships between variables.”
Glanz and Rimer (2005, p.4).
Applying theory leads both to “a steadily richer and more potent picture of how things
work” (Clarke, 1987, p.35) and to the refinement of the theory from which they
derived. The problem of research that lacks explicit theory is characterised by Clarke
as “pieces of a jigsaw which accumulate in journals ….despite the fact that a real jig-
saw puzzle can only be made by taking a picture and cutting it up into pieces, not by
making pieces and hoping they will form a picture.” (p.35)
Explicit use of theory in designing and evaluating interventions has several
benefits. First, theory can be used to inform interventions by identifying constructs
(key concepts in the theory) that are hypothesised to be causally related to behaviour
and are therefore appropriate targets for the intervention. Changing constructs that
cause behaviour will, theoretically, lead to behaviour change (Hardeman et al., 2005).
Using theory in this way can confer a range of benefits including potentially stronger
effects (Albarracin et al., 2005; Downing, Jones, Cook, & Bellis, 2006; Fisher &

4
Fisher, 2000; Gehrman & Hovell, 2003; Kim, Stanton, Li, Dickersin, & Galbraith,
1997). First, by identifying theoretical constructs to target, theory provides a means
for selecting component intervention techniques (Wingood & DiClemente, 1996;
Michie, Johnston, Francis, Hardeman, & Eccles, 2008) and for refining or tailoring
intervention techniques (Noar, Benac, & Harris, 2007). Second, collecting empirical
data within a theoretical framework facilitates the accumulation of evidence of
effectiveness across different contexts, populations and behaviours. Third, as well as
using theory to inform intervention development, theory-based interventions can aid
understanding of why interventions are effective or ineffective, thus facilitating an
understanding of mechanisms of change and providing a basis for refining and
developing better theory (Jemmott & Jemmott, 2000; Michie & Abraham, 2004). In
this way, there is a synergistic cycle of intervention development and evaluation.
Despite the apparent advantages of applying theory to interventions, a
substantial proportion of studies, identified in reviews, fails to make explicit reference
to theory, let alone explicitly apply or test it (see Albarracin et al, 2005; Trifiletti,
Gielen, Sleet, & Hopkins 2005; Noar & Zimmerman, 2005; Dombrowski, Sniehotta,
& Avenell, 2007). Studies are traditionally less likely to be developed within an
organising conceptual framework in the behavioural and social sciences than in the
biomedical and physical sciences (Clarke, 1987). Theory is too often used as a „loose
framework‟, to which passing reference is made, rather than as an integral part of a
rigorous scientific process. Where a theoretical base for an intervention is stated,
there is seldom reference to a method describing how the theory informed the design
of the intervention, or how the evaluation tests theory (Rothman, 2004). This may
reflect a lack of consideration as to how theory might be used to inform and evaluate
the intervention at the planning stage, or poor articulation of these issues at the