Evaluating the Impact of Simulation on Translational Patient Outcomes
01 Aug 2011-Simulation in healthcare : journal of the Society for Simulation in Healthcare (NIH Public Access)-Vol. 6, Iss: 7
TL;DR: Rigorous SBME TSR can contribute to better patient care and improved patient safety, and can yield a favorable return on financial investment and contributes to long-term retention of acquired clinical skills.
Abstract: Introduction:A long and rich research legacy shows that under the right conditions, simulation-based medical education (SBME) is a powerful intervention to increase medical learner competence. SBME translational science demonstrates that results achieved in the educational laboratory (T1) transfer t
Citations
More filters
TL;DR: This Guide provides practical guidance to aid educators in effectively using simulation for training, and will focus on the educational principles that lead to effective learning, and include topics such as feedback and debriefing, deliberate practice, and curriculum integration – all central to simulation efficacy.
Abstract: Over the past two decades, there has been an exponential and enthusiastic adoption of simulation in healthcare education internationally. Medicine has learned much from professions that have established programs in simulation for training, such as aviation, the military and space exploration. Increased demands on training hours, limited patient encounters, and a focus on patient safety have led to a new paradigm of education in healthcare that increasingly involves technology and innovative ways to provide a standardized curriculum. A robust body of literature is growing, seeking to answer the question of how best to use simulation in healthcare education. Building on the groundwork of the Best Evidence in Medical Education (BEME) Guide on the features of simulators that lead to effective learning, this current Guide provides practical guidance to aid educators in effectively using simulation for training. It is a selective review to describe best practices and illustrative case studies. This Guide is the second part of a two-part AMEE Guide on simulation in healthcare education. The first Guide focuses on building a simulation program, and discusses more operational topics such as types of simulators, simulation center structure and set-up, fidelity management, and scenario engineering, as well as faculty preparation. This Guide will focus on the educational principles that lead to effective learning, and include topics such as feedback and debriefing, deliberate practice, and curriculum integration – all central to simulation efficacy. The important subjects of mastery learning, range of difficulty, capturing clinical variation, and individualized learning are also examined. Finally, we discuss approaches to team training and suggest future directions. Each section follows a framework of background and definition, its importance to effective use of simulation, practical points with examples, and challenges generally encountered. Simulation-based healthcare education has great potential for use throughout the healthcare education continuum, from undergraduate to continuing education. It can also be used to train a variety of healthcare providers in different disciplines from novices to experts. This Guide aims to equip healthcare educators with the tools to use this learning modality to its full capability.
715 citations
Cites background or result from "Evaluating the Impact of Simulation..."
...(McGaghie 2010b; McGaghie et al. 2011c)....
[...]
...Simulation-based mastery learning has been shown to be more effective than clinical training alone (McGaghie et al. 2011a; Barsuk et al. 2012) and to improve patient outcomes (Wayne et al. 2008a; Barsuk et al. 2009; Zendejas et al. 2011)....
[...]
...Simulation is also being increasingly considered as an enabling technology to facilitate implementation and translational sciences (McGaghie et al. 2011c)....
[...]
...Evidence increasingly shows that simulationbased healthcare education with deliberate practice leads to improved and lasting results compared with traditional clinical education (McGaghie et al. 2011b)....
[...]
TL;DR: This article critically review simulation‐based mastery learning research in medical education, evaluate its implementation and immediate results, and document measured downstream translational outcomes in terms of improved patient care practices, better patient outcomes and collateral effects.
Abstract: Objectives: This article has two objectives. Firstly, we critically review simulation-based mastery learning (SBML) research in medical education, evaluate its implementation and immediate results, and document measured downstream translational outcomes in terms of improved patient care practices, better patient outcomes and collateral effects. Secondly, we briefly address implementation science and its importance in the dissemination of innovations in medical education and health care.
Methods: This is a qualitative synthesis of SBML with translational (T) science research reports spanning a period of 7 years (2006-2013). We use the 'critical review' approach proposed by Norman and Eva to synthesise findings from 23 medical education studies that employ the mastery learning model and measure downstream translational outcomes.
Results: Research in SBML in medical education has addressed a range of interpersonal and technical skills. Measured outcomes have been achieved in educational laboratories (T1), and as improved patient care practices (T2), patient outcomes (T3) and collateral effects (T4).
Conclusions: Simulation-based mastery learning in medical education can produce downstream results. Such results derive from integrated education and health services research programmes that are thematic, sustained and cumulative. The new discipline of implementation science holds promise to explain why medical education innovations are adopted slowly and how to accelerate innovation dissemination.
450 citations
TL;DR: The quantity and quality of studies that contain an economic analysis of simulation-based medical education for the training of health professions learners are summarized and a comprehensive model for accounting and reporting costs in SBME is proposed.
288 citations
Cites background from "Evaluating the Impact of Simulation..."
...However, as training expenditures rise with an increased emphasis on technology-based education, we must carefully evaluate the costs of SBME against its outcomes to know how to best allocate resources.(4,5) Published evidence clearly establishes the effectiveness of SBME(2); however, the evidence on the costs of simulation in...
[...]
TL;DR: An evidence-based pedagogical framework for procedural skills training was developed based on a review of the literature using a critical synthesis approach and builds on earlier models of procedural skill training in medicine.
Abstract: Acquisition of competency in procedural skills is a fundamental goal of medical training. In this Perspective, the authors propose an evidence-based pedagogical framework for procedural skill training. The framework was developed based on a review of the literature using a critical synthesis approach and builds on earlier models of procedural skill training in medicine. The authors begin by describing the fundamentals of procedural skill development. Then, a six-step pedagogical framework for procedural skills training is presented: Learn, See, Practice, Prove, Do, and Maintain. In this framework, procedural skill training begins with the learner acquiring requisite cognitive knowledge through didactic education (Learn) and observation of the procedure (See). The learner then progresses to the stage of psychomotor skill acquisition and is allowed to deliberately practice the procedure on a simulator (Practice). Simulation-based mastery learning is employed to allow the trainee to prove competency prior to performing the procedure on a patient (Prove). Once competency is demonstrated on a simulator, the trainee is allowed to perform the procedure on patients with direct supervision, until he or she can be entrusted to perform the procedure independently (Do). Maintenance of the skill is ensured through continued clinical practice, supplemented by simulation-based training as needed (Maintain). Evidence in support of each component of the framework is presented. Implementation of the proposed framework presents a paradigm shift in procedural skill training. However, the authors believe that adoption of the framework will improve procedural skill training and patient safety.
214 citations
TL;DR: Residents who completed simulation-based mastery learning (SBML) showed significant improvement in LP procedural skills, and few neurology residents were competent to perform a simulated LP despite clinical experience with the procedure.
Abstract: Objective: To evaluate the effect of simulation-based mastery learning (SBML) on internal medicine residents9 lumbar puncture (LP) skills, assess neurology residents9 acquired LP skills from traditional clinical education, and compare the results of SBML to traditional clinical education. Methods: This study was a pretest-posttest design with a comparison group. Fifty-eight postgraduate year (PGY) 1 internal medicine residents received an SBML intervention in LP. Residents completed a baseline skill assessment (pretest) using a 21-item LP checklist. After a 3-hour session featuring deliberate practice and feedback, residents completed a posttest and were expected to meet or exceed a minimum passing score (MPS) set by an expert panel. Simulator-trained residents9 pretest and posttest scores were compared to assess the impact of the intervention. Thirty-six PGY2, 3, and 4 neurology residents from 3 medical centers completed the same simulated LP assessment without SBML. SBML posttest scores were compared to neurology residents9 baseline scores. Results: PGY1 internal medicine residents improved from a mean of 46.3% to 95.7% after SBML ( p p Conclusions: Residents who completed SBML showed significant improvement in LP procedural skills. Few neurology residents were competent to perform a simulated LP despite clinical experience with the procedure.
211 citations
References
More filters
Book•
01 Jan 2001TL;DR: In this article, the authors present experiments and generalized Causal inference methods for single and multiple studies, using both control groups and pretest observations on the outcome of the experiment, and a critical assessment of their assumptions.
Abstract: 1. Experiments and Generalized Causal Inference 2. Statistical Conclusion Validity and Internal Validity 3. Construct Validity and External Validity 4. Quasi-Experimental Designs That Either Lack a Control Group or Lack Pretest Observations on the Outcome 5. Quasi-Experimental Designs That Use Both Control Groups and Pretests 6. Quasi-Experimentation: Interrupted Time Series Designs 7. Regression Discontinuity Designs 8. Randomized Experiments: Rationale, Designs, and Conditions Conducive to Doing Them 9. Practical Problems 1: Ethics, Participant Recruitment, and Random Assignment 10. Practical Problems 2: Treatment Implementation and Attrition 11. Generalized Causal Inference: A Grounded Theory 12. Generalized Causal Inference: Methods for Single Studies 13. Generalized Causal Inference: Methods for Multiple Studies 14. A Critical Assessment of Our Assumptions
12,215 citations
Book•
01 Jan 1963
TL;DR: A survey drawn from social science research which deals with correlational, ex post facto, true experimental, and quasi-experimental designs and makes methodological recommendations is presented in this article.
Abstract: A survey drawn from social-science research which deals with correlational, ex post facto, true experimental, and quasi-experimental designs and makes methodological recommendations. Bibliogs.
10,916 citations
Book•
01 Jan 1970TL;DR: In this article, the authors discuss the use of tests and test interpretation for various purposes, e.g., personality measurement through self-report, personality test interpretation, and personnel selection.
Abstract: I.BASIC CONCEPTS. 1.Who Uses Tests? And for What Purposes? 2.Varieties of Tests and Test Interpretations. 3.Administering Tests. 4.Scores and Score Conversions. 5.How to Judge Tests: Validation. 6.How to Judge Tests: Reliability and other Qualities. II.TESTS OF ABILITY. 7.General Ability: Appraisal Methods. 8.The Meanings of General Ability. 9.Influences on Intellectual Development. 10.Multiple Abilities and Their Role in Counseling. 11.Personnel Selection. III.MEASURES OF TYPICAL RESPONSE. 12.Interest Inventories. 13.General Problems in Studying Personality. 14.Personality Measurement through Self-Report. 15.Judgments and Systematic Observations. 16.Inferences from Performance.
3,858 citations
TL;DR: An evidence-based intervention resulted in a large and sustained reduction (up to 66%) in rates of catheter-related bloodstream infection that was maintained throughout the 18-month study period.
Abstract: A b s t r ac t A total of 108 ICUs agreed to participate in the study, and 103 reported data. The analysis included 1981 ICU-months of data and 375,757 catheter-days. The median rate of catheter-related bloodstream infection per 1000 catheter-days decreased from 2.7 infections at baseline to 0 at 3 months after implementation of the study intervention (P≤0.002), and the mean rate per 1000 catheter-days decreased from 7.7 at baseline to 1.4 at 16 to 18 months of follow-up (P<0.002). The regression model showed a significant decrease in infection rates from baseline, with incidence-rate ratios continuously decreasing from 0.62 (95% confidence interval (CI), 0.47 to 0.81) at 0 to 3 months after implementation of the intervention to 0.34 (95% CI, 0.23 to 0.50) at 16 to 18 months. Conclusions An evidence-based intervention resulted in a large and sustained reduction (up to 66%) in rates of catheter-related bloodstream infection that was maintained throughout the 18-month study period.
3,844 citations
TL;DR: While research in this field needs improvement in terms of rigor and quality, high-fidelity medical simulations are educationally effective and simulation-based education complements medical education in patient care settings.
Abstract: SUMMARY Review date: 1969 to 2003, 34 years. Background and context: Simulations are now in widespread use in medical education and medical personnel evaluation. Outcomes research on the use and effectiveness of simulation technology in medical education is scattered, inconsistent and varies widely in methodological rigor and substantive focus. Objectives: Review and synthesize existing evidence in educational science that addresses the question, ‘What are the features and uses of high-fidelity medical simulations that lead to most effective learning?’. Search strategy: The search covered five literature databases (ERIC, MEDLINE, PsycINFO, Web of Science and Timelit) and employed 91 single search terms and concepts and their Boolean combinations. Hand searching, Internet searches and attention to the ‘grey literature’ were also used. The aim was to perform the most thorough literature search possible of peer-reviewed publications and reports in the unpublished literature that have been judged for academic quality. Inclusion and exclusion criteria: Four screening criteria were used to reduce the initial pool of 670 journal articles to a focused set of 109 studies: (a) elimination of review articles in favor of empirical studies; (b) use of a simulator as an educational assessment or intervention with learner outcomes measured quantitatively; (c) comparative research, either experimental or quasi-experimental; and (d) research that involves simulation as an educational intervention. Data extraction: Data were extracted systematically from the 109 eligible journal articles by independent coders. Each coder used a standardized data extraction protocol. Data synthesis: Qualitative data synthesis and tabular presentation of research methods and outcomes were used. Heterogeneity of research designs, educational interventions, outcome measures and timeframe precluded data synthesis using meta-analysis. Headline results: Coding accuracy for features of the journal articles is high. The extant quality of the published research is generally weak. The weight of the best available evidence suggests that high-fidelity medical simulations facilitate learning under the right conditions. These include the following:
3,176 citations