SUMMARY Review date: 1969 to 2003, 34 years. Background and context: Simulations are now in widespread use in medical education and medical personnel evaluation. Outcomes research on the use and effectiveness of simulation technology in medical education is scattered, inconsistent and varies widely in methodological rigor and substantive focus. Objectives: Review and synthesize existing evidence in educational science that addresses the question, ‘What are the features and uses of high-fidelity medical simulations that lead to most effective learning?’. Search strategy: The search covered five literature databases (ERIC, MEDLINE, PsycINFO, Web of Science and Timelit) and employed 91 single search terms and concepts and their Boolean combinations. Hand searching, Internet searches and attention to the ‘grey literature’ were also used. The aim was to perform the most thorough literature search possible of peer-reviewed publications and reports in the unpublished literature that have been judged for academic quality. Inclusion and exclusion criteria: Four screening criteria were used to reduce the initial pool of 670 journal articles to a focused set of 109 studies: (a) elimination of review articles in favor of empirical studies; (b) use of a simulator as an educational assessment or intervention with learner outcomes measured quantitatively; (c) comparative research, either experimental or quasi-experimental; and (d) research that involves simulation as an educational intervention. Data extraction: Data were extracted systematically from the 109 eligible journal articles by independent coders. Each coder used a standardized data extraction protocol. Data synthesis: Qualitative data synthesis and tabular presentation of research methods and outcomes were used. Heterogeneity of research designs, educational interventions, outcome measures and timeframe precluded data synthesis using meta-analysis. Headline results: Coding accuracy for features of the journal articles is high. The extant quality of the published research is generally weak. The weight of the best available evidence suggests that high-fidelity medical simulations facilitate learning under the right conditions. These include the following:

/pdf/features-and-uses-of-high-fidelity-medical-simulations-that-51wb8k5opy.pdf

Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review

Changes in medical practice that limit instruction time and patient availability, the expanding options for diagnosis and management, and advances in technology are contributing to greater use of simulation technology in medical education. Four areas of high-technology simulations currently being used are laparoscopic techniques, which provide surgeons with an opportunity to enhance their motor skills without risk to patients; a cardiovascular disease simulator, which can be used to simulate cardiac conditions; multimedia computer systems, which includes patient-centered, casebased programs that constitute a generalist curriculum in cardiology; and anesthesia simulators, which have controlled responses that vary according to numerous possible scenarios. Some benefits of simulation technology include improvements in certain surgical technical skills, in cardiovascular examination skills, and in acquisition and retention of knowledge compared with traditional lectures. These systems help to address the problem of poor skills training and proficiency and may provide a method for physicians to become self-directed lifelong learners.

Simulation Technology for Health Care Professional Skills Training and Assessment

Multiple pressures have stimulated the development of curricula to teach fundamental technical skills to surgeons in a laboratory setting. These include reduced resident work hours, increasing costs of operating room time, and the public and payers’ focus on medical errors and the ethics of learning basic skills on patients. In response to these demands, laparoscopic simulators have been developed using inanimate box trainers or computer-based virtual reality platforms.1,2 The goals of these simulator-based curricula are to provide an opportunity to learn and practice basic skills in a relaxed and inexpensive environment to attain a basic level of technical facility that can be transferred from the laboratory to the operating room environment.

Laparoscopy has been an area where simulator curricula have attracted much interest because unique skills had to be learned not only by surgeons in training but also by surgeons in practice. This latter group had to develop a strategy to acquire novel skills and incorporate these skills into their clinical practice. Since simulator training requires an investment in both the equipment and time required for training, it is important that this investment be justified by providing proof of the value of simulators.

The process to develop and prove the value of the MISTELS (McGill Inanimate System for Training and Evaluation of Laparoscopic Skills) physical laparoscopy simulator has followed a stepwise progression. First, the skills unique to laparoscopic surgery were identified, modeled into exercises that could be carried out in a physical simulator, and a measuring system (metrics) was developed for each exercise, providing a quantitative and objective assessment of performance based on efficiency and precision. Next, the metrics were evaluated for reliability and validity. The most important aspect of validity assessment was to evaluate the relationship of technical skill measured in the simulator to skill in the operating room. If this relationship was found to be robust, simulator performance could then be used to predict performance in the operating room. Once these steps had been completed, the simulator system could then be assessed as a means to verify that laparoscopic technical skill had reached a level thought necessary for the safe performance of basic laparoscopic surgery. In other words, we sought to determine whether the MISTELS score could be used as a summative assessment tool to separate a group that was considered competent from one that would not be considered competent purely from the technical-skill point of view. If so, the sensitivity, specificity, positive and negative predictive values could be determined.

The purpose of this paper is to review the process used to develop the MISTELS physical laparoscopy simulator and to summarize the data accumulated over a series of experiments to prove its value as an effective tool to teach and evaluate the fundamental skills required in laparoscopic surgery. These new data are put in context with previously published preliminary data on the MISTELS system.

Proving the Value of Simulation in Laparoscopic Surgery

Background There is a pressing need for an intraoperative assessment tool that meets high standards of reliability and validity to use as an outcome measure for different training strategies. The aim of this study was to develop a tool specific for laparoscopic skills and to evaluate its reliability and validity. Methods The Global Operative Assessment of Laparoscopic Skills (GOALS) consists of a 5-item global rating scale. A 10-item checklist and 2 visual analogue scales (VAS) for competence and case difficulty were also used. During laparoscopic cholecystectomy, 21 participants were evaluated by the attending surgeon, by 2 trained observers and by self-assessment while dissecting the gallbladder from the liver bed. Results The intraclass correlation coefficient (ICC) for the total GOALS score was .89 (95% confidence interval [CI] .74 to .95) between observers, .82 (95% CI .67 to .92) between observers and attending surgeons, and .70 (95% CI .37 to .87) between participants and attending surgeons. The ICCs (observers) for the VAS (competence) and the checklist were .69 and .70, respectively. The mean total GOALS score (observers) for novices (postgraduate years [PGYs] 1 through 3) was 13 (95% CI 10.3 to 15.7) compared with 19.4 (95% CI 17.2 to 21.5) for experienced (PGY 4 through attending surgeons, P = .0006). The VAS demonstrated a difference in scores between novice and experienced participants ( P = .001); however, the task checklist did not ( P = .09). Conclusions These data indicate that GOALS is feasible, reliable, and valid. They also suggest that it is superior to the task checklist and VAS for evaluation of technical skill by experienced raters. The findings support the use of GOALS in the training and evaluation of laparoscopic skills.

A global assessment tool for evaluation of intraoperative laparoscopic skills

Background: Developing technical skill is essential to surgical training, but using the operating room for basic skill acquisition may be inefficient and expensive, especially for laparoscopic operations This study determines if laparoscopic skills training using simulated tasks on a video-trainer improves the operative performance of surgery residents Study Design: Second- and third-year residents (n = 27) were prospectively randomized to receive formal laparoscopic skills training or to a control group At baseline, residents had a validated global assessment of their ability to perform a laparoscopic cholecystectomy based on direct observation by three evaluators who were blinded to the residents' randomization status Residents were also tested on five standardized video-trainer tasks The training group practiced the video-trainer tasks as a group for 30 minutes daily for 10 days The control group received no formal training All residents repeated the video-trainer test and underwent a second global assessment by the same three blinded evaluators at the end of the 1-month rotation Within-person improvement was determined; improvement was adjusted for differences in baseline performance Results: Five residents were unable to participate because of scheduling problems; 9 residents in the training group and 13 residents in the control group completed the study Baseline laparoscopic experience, video-trainer scores, and global assessments were not significantly different between the two groups The training group on average practiced the video-trainer tasks 138 times (range 94 to 171 times); the control group did not practice any task The trained group achieved significantly greater adjusted improvement in video-trainer scores (five of five tasks) and global assessments (four of eight criteria) over the course of the four-week curriculum, compared with controls Conclusions: Intense training improves video-eye-hand skills and translates into improved operative performance for junior surgery residents Surgical curricula should contain laparoscopic skills training

https://www.journalacs.org/article/S1072-7515(00)00339-2/pdf

Laparoscopic training on bench models: better and more cost effective than operating room experience?

Background: Interest in the training and evaluation of laparoscopic skills is extending beyond the realm of the operating room to the use of laparoscopic simulators. The purpose of this study was to develop a series of structured tasks to objectively measure laparoscopic skills. This model was then used to test for the effects of level of training and practice on performance. Methods: Forty-two subjects (6 each of surgical residents PGY1 to PGY5, 6 surgeons who practice laparoscopy and 6 who do not) were evaluated. Each subject viewed a 20-minute introductory video, then was tested performing 7 laparoscopic tasks (peg transfers, pattern cutting, clip and divide, endolooping, mesh placement and fixation, suturing with intracorporeal or extracorporeal knots). Performance was measured using a scoring system rewarding precision and speed. Each candidate repeated all 7 tasks and was rescored. Data were analyzed by linear regression to assess the relationship of performance with level of residency training for each task, and by ANOVA with repeated measures to test for effects of level of training, of repetition, and of the interaction between level of training and repetition on overall performance. Student's t test was used to evaluate differences between laparoscopic and nonlaparoscopic surgeons and between each of these groups and the PGY 5 level of surgical residents. Results: Significant predictors of overall performance were (a) level of training ( P = 0.002), (b) repetition ( P P = 0.001). There was also a significant interaction between level of training and the specific task on performance scores ( P = 0.006). When each task was evaluated individually for the 30 residents, 4 of the 7 tasks (tasks 1, 2, 6, 7) showed significant correlation between PGY level and score. A significant difference in performance scores between laparoscopic and nonlaparoscopic surgeons was seen for tasks 1, 2, and 6. Conclusions: A model was developed to evaluate laparoscopic skills. Construct validity was demonstrated by measuring significant improvement in performance with increasing residency training, and with practice. Further validation will require correlation of performance in the model with skill in vivo.

Development of a Model for Training and Evaluation of Laparoscopic Skills

Laparoscopic cholecystectomy (LC) has been performed increasingly in an outpatient setting. Conversion from LC to open cholecystectomy (OC) is sometimes required. To predict conversion to OC, a single institutional study of 1,676 consecutive patients in whom LC was attempted was performed. Factors evaluated were age, sex, history of acute cholecystitis, pancreatitis, or jaundice, previous abdominal surgery, abnormalities of liver function tests, thickened gallbladder wall identified by preoperative ultrasound, obesity or morbid obesity, and cumulative institutional experience in LC. Conversion to OC was required in 90 of 1,676 (5.4%) patients. Significant preoperative predictors of conversion were acute cholecystitis, increasing age, male sex, obesity, and thickened gallbladder wall found by ultrasound. Nonobese women younger than age 65 years with symptoms of biliary colic and normal gallbladder wall thickness found by preoperative ultrasound required conversion only 1.9% of the time. These predictors may be useful in planning a program of ambulatory or short stay surgical units for patients undergoing LC and for comparing data between series.

Factors determining conversion to laparotomy in patients undergoing laparoscopic cholecystectomy.

Background: Laparoscopic skill was measured objectively through a series of seven tasks in an inanimate laparoscopic simulator Seven analogous skills were tested in an in vivo porcine model These skills included transferring, cutting, clipping, placement of a ligating loop, mesh placement, and suturing with an intracorporeal and extracorporeal knot Scoring of each task rewarded precision and speed

Comparison of laparoscopic performance in vivo with performance measured in a laparoscopic simulator.

OBJECTIVE: This study evaluated the selective use of endoscopic retrograde cholangiopancreatography (ERCP) in the context of laparoscopic cholecystectomy (LC) while minimizing the use of operative cholangiography SUMMARY BACKGROUND DATA: There has been a long-standing debate between routine and selective operative cholangiography that has resurfaced with LC METHODS: Prospective data were collected on the first 1300 patients undergoing LC at McGill University Preoperative indications for ERCP were recorded, radiologic findings were standardized, and technical points for a safe LC were emphasized RESULTS: A total of 106 patients underwent 127 preoperative ERCPs Fifty patients were found to have choledocholithiasis (38%), and clearance of the common bile duct (CBD) with endoscopic sphincterotomy was achieved in 45 patients The other five patients underwent open cholecystectomy with common duct exploration Intraoperative cholangiography (IOC) was attempted in only 54 patients (42%), 6 of whom demonstrated choledocholithiasis Forty-nine postoperative ERCPs were performed in 33 patients and stones were detected in 17 (13%), with a median follow-up time of 22 months Endoscopic duct clearance was successful in all of these The incidence of CBD injury was 038%, and a policy of routine operative cholangiography might only have led to earlier recognition of duct injury in one case The rate of complication for all ERCPs was 9% and the associated median duration of the hospital stay was 4 days The median duration of the hospital stay after open CBD exploration was 13 days CONCLUSIONS: LC can be performed safely without routine IOC The selective use of preoperative and postoperative ERCP will clear the CBD of stones in 925% of patients

Cholecystectomy without operative cholangiography: Implications for common bile duct injury and retained common bile duct stones

Background: Laparoscopic skills can be measured objectively in a video-laparoscopic cart simulator system. These scores have been shown to be sufficiently sensitive to distinguish differences in performance between residents at different levels of training. The purpose of this study was to compare a simplified mirrored-box simulator to the video-laparoscopic cart system.

Harvey H. Sigman

Papers

Development of a Model for Training and Evaluation of Laparoscopic Skills

Factors determining conversion to laparotomy in patients undergoing laparoscopic cholecystectomy.

Comparison of laparoscopic performance in vivo with performance measured in a laparoscopic simulator.

Cholecystectomy without operative cholangiography: Implications for common bile duct injury and retained common bile duct stones

A simplified simulator for the training and evaluation of laparoscopic skills