REVIEW PAPER
Learning theories and tools for the assessment of core
nursing competencies in simulation: A theoretical review
Patrick Lavoie
1,2
|
C
ecile Michaud
3
|
Marilou B
elisle
3
|
Louise Boyer
1
|
Emilie Gosselin
3
|
Myrian Grondin
4
|
Caroline Larue
1
|
St
ephan Lavoie
3
|
Jacinthe Pepin
1
1
Center for Innovation in Nursing Education
(CIFI), Faculty of Nursing, Universit
ede
Montr
eal, Montr
eal, QC, Canada
2
William F. Connell School of Nursing,
Boston College, Chestnut Hill, MA, USA
3
School of Nursing, Universit
ede
Sherbrooke, Sherbrooke, QC, Canada
4
Allied Health Sciences Library, Universit
e
de Montr
eal, Montr
eal, QC, Canada
Correspondence
Patrick Lavoie, Center for Innovation in
Nursing Education (CIFI), Faculty of Nursing,
Universit
e de Montr
eal, Pavillon Marguerite-
d’Youville, C.P. 6128 succ. Centre-Ville,
Montr
eal (QC), Canada, H3C 3J7.
Email: patrick.lavoie.1@umontreal.ca
Funding Information
This study was conducted by
Equipe FUTUR,
a research team funded with an
infrastructure grant from the Fonds de
recherche du Qu
ebec – Soci
et
e et culture
(FRQSC). No other grant from any funding
agency in the public, commercial, or not-for-
profit sectors was received to write this
paper
Abstract
Aim: To identify the theories used to explain learning in simulation and to examine
how these theories guided the assessment of learning outcomes related to core
competencies in undergraduate nursing students.
Background: Nurse educators face the challenge of making explicit the outcomes of
competency-based education, especially when competencies are conceptualized as
holistic and context dependent.
Design: Theoretical review.
Data Sources: Research papers (N = 182) published between 1999–2015 describing
simulation in nursing education.
Review Methods: Two members of the research team extracted data from the
papers, including theories used to explain how simulation could engender learning
and tools used to assess simulation outcomes. Contingency tables were created to
examine the associations between theories, outcomes and tools.
Results: Some papers (N = 79) did not provide an explicit theory. The 103 remain-
ing papers identified one or more learning or teaching theories; the most frequent
were the National League for Nursing/Jeffries Simulation Framework, Kolb’s theory
of experiential learning and Bandura’s social cognitive theory and concept of self-
efficacy. Students’ perceptions of simulation, knowledge and self-confidence were
the most frequently assessed, mainly via scales designed for the study where they
were used. Core competencies were mostly assessed with an observational
approach.
Conclusion: This review highlighted the fact that few studies examined the use of
simulation in nursing education through learning theories and via assessment of core
competencies. It also identified observational tools used to assess competencies in
action, as holistic and context-dependent constructs.
KEYWORDS
assessment, competency-based education, learning theories, literature review, nursing
education, simulation, undergraduate nursing students
Accepted: 14 July 2017
DOI: 10.1111/jan.13416
J Adv Nurs. 2017;1–12. wileyonlinelibrary.com/journal/jan © 2017 John Wiley & Sons Ltd
|
1
1
|
INTRODUCTION
Competency-based education (CBE) is characterized by learner-cen-
teredness and active learning, but is mainly oriented to learning out-
comes attainment (Frank et al., 2010; Goudreau et al., 2009). To
determine if learners achieve standards of competence, CBE requires
a robust and multifaceted approach to define and assess student
learning outcomes and to provides trainees with feedback about
their competency development (Holmboe, Sherbino, Long, Swing, &
Frank, 2010). Nurse educators who adopt CBE must use tools that
will make students’ learning outcomes explicit. Furthermore, it is rec-
ommended that assessment tools should include criteria that reflect
learners’ attainment of the milestones that make up the trajectory to
competence (Holmboe et al., 2010). However, this remains challeng-
ing when competencies are conceptualized as holistic and context-
dependent combinations of knowledge, skills and attitudes, as
opposed to task-specific behaviours (Cowan, Norman, & Coopamah,
2005).
1.1
|
Background
Morcke, Dornan, and Eika (2013) argued that the evolution of CBE can
be traced back to psychology in the 1940s, but that strong endorse-
ment of CBE by the medical community started in the new millennium
and was propelled by the Flexner centenary report on the future of
medical education (Cooke, Irby, & O’Brien, 2010). A central require-
ment of CBE is to define learning outcomes of education – competen-
cies – and to set clear expectations for learners. A competency can be
defined as a complex knowing of how to act based on the effective
mobilization and combination of a variety of internal and external
resources in a family of situations (Tardif, 2006). In nursing, different
sets of core competencies have been defined. The Quality and Safety
Education for Nurses’ competencies (Cronenwett et al., 2007) were
adapted from the Institute of Medicine’s (Greiner & Knebel, 2003) five
core competencies for all health professionals and include patient-
centred care, teamwork and collaboration, evidence-based practice,
quality improvement, safety and informatics. The Competency Out-
comes and Performance Assessment Model (Lenburg, Abdur-Rahman,
Spencer, Boyer, & Klein, 2011) described another set of core nursing
competencies: assessment and intervention, communication, critical
thinking, teaching, human caring relationships, management, leader-
ship and knowledge integration.
The way CBE is enacted is prone to variations, but active learning
remains one of its main features. Active learning involves engaging
students in meaningful learning activities and in reflection about what
they are doing (Bonwell & Eison, 1991; Prince, 2004). Examples of
active learning strategies include problem-based learning, classroom
response systems, games and case studies. Simulation is also an active
learning strategy, for which interest has grown tremendously in nurs-
ing education. Simulation has been described as “a technique – not a
technology – to replace or amplify real experiences with guided experi-
ences that evoke or replicate substantial aspects of the real world in a
fully interactive manner” (Gaba, 2004, p. i2).
Following the call for more valid and reliable tools to measure
the outcomes of simulation (Kardong-Edgren, Adamson, & Fitzgerald,
2010; Tanner, 2011), there has been considerable efforts in
developing such instruments. However, recent literature reviews
(Adamson, Kardong-Edgren, & Willhaus, 2013; Foronda et al. 2013;
Kardong-Edgren et al., 2010) showed that these tools often measure
knowledge, skills and attitudes as separate constructs. As such, it is
difficult for nurse educators to assess how students mobilize and
combine those resources in their encounters with simulated patients.
Why is this review needed?
•
Nurse educators face the challenge of making explicit
the outcomes of competency-based education, especially
when competencies are conceptualized as holistic and
context dependent.
•
Simulation is an active learning strategy coherent with
competency-based education; hence, it is crucial to
understand how it can contribute to the development of
core nursing competencies. Accordingly, there is a need
to determine which learning theories are currently guid-
ing simulation research.
•
Tools to assess students’ competencies in simulation
treat knowledge, skills and attitudes as separate con-
structs, which makes it difficult to assess how students
mobilize and combine those resources in action.
What are the key findings?
•
Most papers either did not cite a learning theory or cited
an instructional design framework for simulation. The
most frequently cited learning theories were Kolb’s expe-
riential learning and Bandura’s social cognitive theory.
•
Students’ perceptions and satisfaction, knowledge, proce-
dural skills and attitudes were the most frequently
assessed outcomes of simulation, mostly with tools
designed for the study where they were used.
•
Few tools assessed core competencies as learning out-
comes of simulation. Those that did relied on observa-
tion of students’ actions in simulation.
How should the findings be used to influence
policy/practice/research/education?
•
Further research is needed to enhance our understanding
of how simulation engenders learning.
•
Since it appears possible to assess core competencies by
observing students’ actions in simulation, future research
should aim at developing and testing new tools that cor-
respond to a holistic perspective of core competencies in
nursing. These tools should provide criteria to assess stu-
dents’ level of development.
2
|
LAVOIE ET AL.
Furthermore, a systematic review of 120 simulation studies by
Kaakinen and Arwood (2009) revealed that 94 studies discussed
simulation as a teaching strategy and only 16 studies referenced a
learning theory to explain how and why simulation was used. As
the authors explained, this suggests that simulation is executed
from a teaching paradigm rather than a learning paradigm. If simula-
tion is posited as an active learning strategy congruent with CBE, it
is important to understand the process of learning in simulation to
explain why it is used and how to assess the learning outcomes it
is expected to engender. Besides, detailing how a conceptual or
theoretical framework guided the development of a simulation
study was included as a criterion by which the quality of simula-
tion-based research articles should be assessed (Fey, Gloe, &
Mariani, 2015).
2
|
THE REVIEW
2.1
|
Aim
The aim of this review was to identify the theories used to explain
learning in simulation and to examine how these theories guided the
assessment of learning outcomes in simulation research. We aimed
to examine how core competencies were assessed in undergraduate
nursing students participating in simulation.
2.2
|
Design
Active learning strategies, including simulation, are complex interven-
tions, as they comprise various interacting components, involve great
outcome variability and require high skill levels for delivery (Craig
et al., 2008). To apprehend this complexity, it is important to attend
to the results of the interventions and to the mechanisms by which
they produce their effects. In this review, we focused on theories
used by authors to explain how simulation could engender learning
outcomes and tools used to assess those outcomes. The research
questions were as follows: What are the theories used to explain
learning in simulation? What would be the learning outcomes of sim-
ulation per those theories and do they correspond to learning out-
comes assessed in simulation studies? Which tools are used to
assess learning outcomes in simulation and are they compatible with
a holistic and context-dependent vision of competencies?
To answer these questions, we designed a theoretical review
(Campbell et al., 2014; Par
e, Trudel, Jaana, & Kitsiou, 2015). Theo-
retical reviews are explanatory by nature; their primary aim is to
identify and map theories that have become influential – or over-
looked – in a field of research to form new and more abstract theo-
retical understandings of the relationships between different
concepts or constructs. Through structured approaches, theoretical
reviews organize prior research and examine patterns and similitudes
to facilitate the development of new theories (Par
e et al., 2015).
To design this theoretical review, we followed guidelines for sys-
tematic reviews (Moher, Liberati, Tetzlaff, & Altman, 2009) to the
greatest possible extent. As discussed by Campbell et al. (2014),
reviewing theories using methods from systematic reviews presents
some methodological challenges. In our case, the most prominent
issues were quality appraisal and synthesis. We did not perform
quality appraisal, as the purpose of the review was to provide a
comprehensive picture of the theories, outcomes and tools used in
the field of simulation research, rather than appraise the effective-
ness of simulation. For synthesis, we organized the literature with an
iteratively developed coding framework and analysed the frequency
of codes. The appraisal of the appropriateness of the relationships
between theories, outcomes and assessment tools was inductive and
based on investigators’ knowledge of the field.
2.3
|
Search Methods
Two independent librarians defined an extensive search strategy to
retrieve research papers describing simulation in nursing education.
Keywords related to CBE, assessment, measurement and instrumen-
tation in research were included. The strategy was developed to
include core nursing competencies, as defined by Greiner and Knebel
(2003), Cronenwett et al. (2007) and Lenburg et al. (2011). These
included: (1) leadership; (2) clinical reasoning, clinical judgement or
critical thinking; (3) collaboration or teamwork; (4) informatics; (5)
learning to learn; (6) evidence-based practice; (7) interpersonal skills
or caring relationships and (8) clinical competence, clinical assess-
ment or clinical intervention. The keywords were subject to a Delphi
process with 14 educators and researchers involved in CBE, who
requested the addition of (9) cultural competence and (10) ethical
competence. The search strategies are available as supplementary
material (see Appendices S1 and S2).
The search was performed twice, in June 2014 (1999–2014) and
July 2015 (2014–2015). Inclusion criteria included: (1) use of role
playing, standardized patients, or low- to high-fidelity mannequins;
(2) undergraduate nursing students; (3) English or French; and (4)
description of a research methodology. Exclusion criteria included:
(1) secondary analysis; literature review or meta-analysis; (2) no
focus on a form of simulation; (3) simulation used solely as a data
collection method; (4) no student outcomes; (5) focus only on speci-
fic parts of a simulation, such as debriefing; and (6) psychometric
studies. We excluded studies using anatomical models, computer-
assisted instruction, games, task trainers and virtual reality, since
they were less representative of real experiences that students might
encounter in their practice.
2.4
|
Search outcome
As depicted in Figure 1, the first database search yielded 8,023 arti-
cles (CINAHL: 2,510; Education Source: 1,919; Embase, 2,636; ERIC:
74; MEDLINE: 744; and PsycInfo: 149). Two independent members
of the research team screened the titles and abstract of the non-
duplicate records (N=5,954) and assigned inclusion or exclusion
codes. Inter-rater agreement yielded a Kappa of 0.86 (95% CI 0.84–
0.88). The screening process left 659 full-text articles to be assessed
for eligibility. Full texts were retrieved and split in two equal sets.
LAVOIE ET AL.
|
3
Two researchers each read one set and 520 articles were excluded
per inclusion and exclusion criteria. When either investigator
doubted whether to include an article, another investigator examined
the article and consensus was reached. Ultimately, 143 studies iden-
tified in the first database search were included.
The second database search yielded 660 articles (CINAHL: 300;
Education Source: 16; Embase, 228; ERIC: 0; MEDLINE: 80; and
PsycInfo: 36). Of these, 547 were identified as non-duplicate
records. Given the high inter-rater agreement for the articles from
the first database search, a single researcher identified potentially
relevant studies from the second database search. Following the
screening process, 430 of these articles were excluded, leaving 117
full-text articles to be assessed for eligibility. Of these, 74 were
excluded. Ultimately, 43 studies identified in the second database
search were included in the review. In the end, 182 articles were
included and are listed as supplementary material (see Appendix S3).
2.5
|
Quality appraisal
None undertaken.
2.6
|
Data abstraction
Two researchers extracted data from the 182 selected articles per
the following categories: year, country, design, purpose, sample, form
of simulation, theory about how simulation could engender learning
outcomes, outcomes, assessment tools and instruments and results.
Two investigators worked independently to inductively code the
content of the grids and detail the categories presented above. Com-
parison of the investigators’ coding for 20% of the studies revealed
that it was identical.
2.7
|
Analysis and synthesis
Data were summarized as frequencies and percentages for categori-
cal variables and means and standard deviations for continuous vari-
ables. Contingency tables were created with the following
combinations of variables: theories and outcomes, outcomes and
tools and theories and tools. Based on these tables, we identified
the outcomes and tools most frequently associated with the preva-
lent theories. Outcomes were classified into categories, which were
arrived at by regrouping similar codes that were inductively gener-
ated from the content analysis. As in a previous review by Adamson
et al. (2013), our classification was influenced by Kirkpatrick and
Kirkpatrick’s (2006) model for evaluating training programme, but we
also took into account Tardif’s (2006) definition of competency. We
reviewed the original texts of the prevalent theories and summarized
their depiction of the learning process and possible learning out-
comes. We compared the association between learning theories and
outcomes in the studies to how they were described in the original
CINAHL
2,510 + 300 citations
Education Source
1,919 + 16 citations
Embase
2,636 + 228 citations
MEDLINE
744 + 80 citations
ERIC
74 + 0 citations
PsycInfo
149 + 36 citations
5,954 + 547 non-duplicated
records screened
Records excluded
(n = 5,295 + 430)
Full-text articles assessed for eligibility
(n = 659 + 117)
Full-text excluded, withreasons:
(n = 520 + 74)
• Not undergraduate nursing students
• Not in English or French
• No research methodology
• Secondary analysis, literature review
or meta-analysis
• No focus on a form of simulation
• Simulation used solely as a data
collection method
• No student out comes
• Focus on only specificpart of a
simulation
• Anatomical models, computer-
assisted instruction, games, task
trainers and virtual reality
Articles included in review
(n = 139 + 43)
FIGURE 1 Literature flow diagram [Colour figure can be viewed at wileyonlinelibrary.com]
4
|
LAVOIE ET AL.
texts. The results were presented to the entire research team in an
audit. Questions asked by the team allowed refinement of the findings.
3
|
RESULTS
The following section includes: (1) characteristics of the studies; (2)
prevalent theories; (3) the outcomes studied and the tools used to
assess them; and (4) associations between theories, outcomes and
tools in exemplar studies. It should be noted that the counts for the
studies sometimes exceeded the number of studies under review,
because some cited two or more theories, outcomes or tools/
instruments.
3.1
|
Characteristics of the Studies
Based on first authors’ affiliations, most studies (N = 131, 70.1%)
were conducted in North America, with 116 conducted in the USA
(63.7%) and 10 in Canada (5.5%). The remaining studies were con-
ducted in Europe (N=21, 11.5%) Asia (N=18, 9.9%), Oceania
(N=11, 6.0%) and the Middle East (N=6, 3.3%). The first study
included in this review was published in 1999. The number of stud-
ies published annually from 1999 to 2007 (M = 1.7) increased in
2008–2009 (M = 13.0) and peaked in 2010–2014 (M = 25.0).
Although the results appeared to show a decrease in this number in
2015 (N=17), they should not be interpreted as such, as the review
did not include studies published during the final 5 months of 2015.
Most studies (N=127, 69.8%) used mannequins exclusively,
most of which were of high fidelity (N=74, 40.7%). A smaller num-
ber of studies included role-play (N=16, 8.8%) or standardized
patients (N=12, 6.6%) exclusively. Fourteen studies (7.7%) used
two types of simulation, either mannequins and standardized
patients (N=10, 5.5%), mannequins and role-play (N=3, 1.6%), or
standardized patients and role-play (N=1, 0.05%). Of note, 13
(7.1%) and 28 (15.4%) studies did not define simulation type and
mannequin fidelity respectively.
3.2
|
Theories
As explained above, theories are to be understood as hypotheses
about how simulation engenders learning outcomes. Almost half of
the studies (N=79, 43.4%) did not cite an explicit theory. Instead,
the rationale for simulation included the standardization of learning
experiences, exposure to rare clinical events, the possibility of error
without risk to patients, the possibility for pausing or repeating simu-
lations and realism and authenticity. Authors also described simula-
tion as an active or interactive instructional strategy with
opportunities for immediate feedback. Simulation was considered a
means for bridging the theory-practice gap or a solution to the clini-
cal placement shortage and nursing staff overload. Other arguments
were drawn from previous studies examining simulation effective-
ness in various learning outcomes. The remaining papers (N = 103,
56.6%) explicitly identified one or more theory as depicted in
Table 1. The most frequently cited was an instructional design
framework, the National League for Nursing (NLN)/Jeffries Simula-
tion Framework (Jeffries, 2012; N = 35, 19.2%). Then, two learning
theories were the most frequent: Kolb’s (1984) experiential learning
theory and Learning Style Inventory (Kolb & Hay, 1999; N = 20,
11.0%), followed by Bandura’s (1986) social cognitive theory and
concept of self-efficacy (1977; N = 18, 9.9%).
TABLE 1 Theories cited in the studies
Theory
Citations
n (%)
NLN/Jeffries Simulation Framework
(Jeffries & Rizzolo, 2007)
35 (19.2)
Experiential learning theory, Learning Style
Inventory (Kolb, 1984; Kolb & Hay, 1999)
20 (11.0)
Self-efficacy, social cognitive theory (Bandura, 1977, 1986) 18 (9.9)
Clinical judgment model (Tanner, 2006) 6 (3.3)
Situated learning (Lave & Wenger, 1991) 6 (3.3)
Constructivism 4 (2.2)
Novice-to-expert model (Benner, 1984) 4 (2.2)
Taxonomy of learning domains and mastery
learning (Bloom, 1956, 1968)
3 (1.6)
Deliberate practice (Ericsson,
Krampe, & Tesch-R
€
omer, 1993)
3 (1.6)
Crisis resource management principles
(Gaba, Fish, & Howard, 1994)
3 (1.6)
Adult learning principles (Knowles,
Holton, & Swanson, 1998)
2 (1.1)
Assessment of clinical competence (Miller, 1990) 2 (1.1)
Cognitive apprenticeship model (Collins,
Brown, & Newman, 1989)
2 (1.1)
Dewey’s (1997) Experience and Education 2 (1.1)
Transformative learning (Mezirow, 1991) 2 (1.1)
Cognitive learning theory (as depicted in
Billings & Halstead, 1998)
1 (0.05)
Complexity integration nursing theory
(Van Sell & Kalofissudis, 2002)
1 (0.05)
Engagement theory of student learning
(Kuh, Kinzie, Schuh, & Whitt, 2005)
1 (0.05)
Four-phase teaching model for simulation
(Joyce & Weil, 1996)
1 (0.05)
Freire’s critical pedagogy (as depicted in
Roberts, 2000)
1 (0.05)
Multiple intelligence learning (Gardner, 2006) 1 (0.05)
Schema of cognitive and ethical
development (Perry, 1970)
1 (0.05)
Self-directed learning (as depicted in
Merriam, Caffarella, & Baumgartner, 2007)
1 (0.05)
Seven principles of good practice
(Chickering & Gamson, 1987)
1 (0.05)
Three-P (presage-process-product) model of
learning (Biggs, 1993)
1 (0.05)
Transfer or learning (Simons, 1999) 1 (0.05)
LAVOIE ET AL.
|
5
