Background Surgeons in the operating theatre deal constantly with high-demand tasks that require simultaneous processing of a large amount of information. In certain situations, high cognitive load occurs, which may impact negatively on a surgeon's performance. This systematic review aims to provide a comprehensive understanding of the different methods used to assess surgeons' cognitive load, and a critique of the reliability and validity of current assessment metrics. Methods A search strategy encompassing MEDLINE, Embase, Web of Science, PsycINFO, ACM Digital Library, IEEE Xplore, PROSPERO and the Cochrane database was developed to identify peer-reviewed articles published from inception to November 2016. Quality was assessed by using the Medical Education Research Study Quality Instrument (MERSQI). A summary table was created to describe study design, setting, specialty, participants, cognitive load measures and MERSQI score. Results Of 391 articles retrieved, 84 met the inclusion criteria, totalling 2053 unique participants. Most studies were carried out in a simulated setting (59 studies, 70 per cent). Sixty studies (71 per cent) used self-reporting methods, of which the NASA Task Load Index (NASA-TLX) was the most commonly applied tool (44 studies, 52 per cent). Heart rate variability analysis was the most used real-time method (11 studies, 13 per cent). Conclusion Self-report instruments are valuable when the aim is to assess the overall cognitive load in different surgical procedures and assess learning curves within competence-based surgical education. When the aim is to assess cognitive load related to specific operative stages, real-time tools should be used, as they allow capture of cognitive load fluctuation. A combination of both subjective and objective methods might provide optimal measurement of surgeons' cognition.

Systematic review of measurement tools to assess surgeons' intraoperative cognitive workload

Biomedical Aspects of the Laser The Introduction of Laser Applications into Biology and Medicine. By Leon Goldman. Pp. viii + 232. (Berlin and New York: Springer-Verlag, 1967.) 45.60 DM. $11.40.

Lasers in Medicine

Simulation-based training (SBT) has become a standard component of modern surgical education, yet successful implementation of evidence-based training programs remains challenging. In this narrative review, we use Kern's framework for curriculum development to describe where we are now and what lies ahead for SBT within surgery with a focus on technical skills in operative procedures. Despite principles for optimal SBT (proficiency-based, distributed, and deliberate practice) having been identified, massed training with fixed time intervals or a fixed number of repetitions is still being extensively used, and simulators are generally underutilized. SBT should be part of surgical training curricula, including theoretical, technical, and non-technical skills, and be based on relevant needs assessments. Furthermore, training should follow evidence-based theoretical principles for optimal training, and the effect of training needs to be evaluated using relevant outcomes. There is a larger, still unrealized potential of surgical SBT, which may be realized in the near future as simulator technologies evolve, more evidence-based training programs are implemented, and cost-effectiveness and impact on patient safety is clearly demonstrated.

Surgical simulation: Current practices and future perspectives for technical skills training.

Competency-based medical education (CBME) is rooted in the use of iterative assessments. We must ensure that the assessments used in CBME are valid, to make acceptable and accurate decisions regarding the competency of a trainee. Until recently, much of the educational and assessment literature in urology have used a now-outdated method of determining validity, based on theory and recommendations from over 50 years ago. We describe a contemporary approach to gathering construct validity evidence for the assessment of urologic trainees, for use in both clinical and simulation environments. Five sources of evidence make up Messick’s contemporary framework of validity: test content, response process, internal structure, relationship to other variables, and consequences. These are all components of construct validation and concern the accuracy, quality, reproducibility, generalizability, and wider impact of the scores generated by an assessment, respectively. When deciding the competency of a trainee, program directors and educators must have a clear understanding of how the validity is established and is determined in each assessment context. The contextual specificity of validity means that stakeholders must be prepared to defend the outcome of an assessment, particularly when making high-stake or summative decisions.

Surgical Education, Simulation, and Simulators-Updating the Concept of Validity.

Competency-based training has gained ground in surgical training and with it assessment tools to ensure that training objectives are met. Very few assessment tools are available for evaluating performance in thoracoscopic procedures. Video recordings would provide the possibility of blinded assessment and limited rater bias. This study aimed to provide validity evidence for a newly developed and dedicated tool for assessing competency in Video-Assisted Thoracoscopic Surgery (VATS) lobectomy. Participants with varying experience with VATS lobectomy were included from different countries. Video recordings from participants’ performance of a VATS right upper lobe lobectomy on a virtual reality simulator were rated by three raters using a modified version of a newly developed VATS lobectomy assessment tool (the VATSAT) and analyzed in relation to the unitary framework (content, response process, internal structure, relation to other variables, and consequences of testing). Fifty-three participants performed two consecutive simulated VATS lobectomies on the virtual reality simulator, leaving a total of 106 videos. Content established in previously published studies. Response process Standardized data collection was ensured by using an instructional element, uniform data collection, a special rating program, and automatic generation of the results to a database. Raters were carefully instructed in using the VATSAT, and tryout ratings were carried out. Internal structure Inter-rater reliability was calculated as intra-class correlation coefficients, to 0.91 for average measures (p < 0.001). Test/re-test reliability was calculated as Pearson’s r of 0.70 (p < 0.001). G-coefficient was calculated to be 0.79 with two procedures and three raters. By performing D-theory was found that either three procedures rated by two raters or five procedures rated by one rater were enough to reach an acceptable G-coefficient of ≥ 0.8. Relation to other variables Significant differences between groups were found (p < 0.001). The participants’ VATS lobectomy experience correlated significantly to their VATSAT score (p = 0.016). Consequences of testing The pass/fail score was found to be 14.9 points by the contrasting groups’ method, leaving five false positive (29%) and six false negatives (43%). Validity evidence was provided for the VATSAT according to the unitary framework. The VATSAT provides supervisors and assessors with a procedure-specific assessment tool for evaluating VATS lobectomy performance and aids with the decision of when the trainee is ready for unsupervised performance.

Evaluating competency in video-assisted thoracoscopic surgery (VATS) lobectomy performance using a novel assessment tool and virtual reality simulation

Direct Observation vs. Video-Based Assessment in Flexible Cystoscopy.

Nonspecialist Raters Can Provide Reliable Assessments of Procedural Skills

Simulation-based training for flexible cystoscopy - A randomized trial comparing two approaches.

During laparoscopy, the surgeon’s loss of depth perception and spatial orientation is problematic. Laser visual guidance (LVG) is an innovative technology that improves depth perception to enhance the visual field. In this trial, we examined the effect of LVG on surgical novices’ motor skills, quality of task performance, and cognitive workload. We designed a randomized controlled trial following the CONSORT statement. Thirty-two surgical novices completed the Training and Assessment of Basic Laparoscopic Techniques (TABLT) test. The first attempt allowed participants to familiarize themselves with the exercises. We then randomized the participants, and they completed a test session using either LVG or conventional two-dimensional vision. We found no significant difference between using the LVG tool and conventional 2D vision; however, both the mean completion time and movements used were less in the LVG group: Mean time used in the LVG group was 1288 s (95 % CI 1188–1388) versus 1354 s (95 % CI 1190–1518) (p = 0.45); mean angular path length used in the LVG group was 24,049° (95 % CI 20,761–27,336) versus 26,014° (95 % CI 22,059–29,970) (p = 0.42); mean path length in the LVG group was 4560 cm (95 % CI 3971–5,149 cm) versus 5062 cm (95 % CI 4328–5797), (p = 0.26). Moreover, the mean TABLT performance score was higher in the LVG group compared with the 2D group, although not significant: 379 (95 % CI 352–405) versus 338 (95 % CI 288–387) (p = 0.14). No significant difference was found between the groups’ cognitive workloads. We found no significant improvement of laparoscopic motor skills when using LVG, although a tendency toward improved performance was seen. LVG could have the potential to help novice surgeons acquire basic laparoscopic; however, further development of the concept and validation is needed to confirm this.

/pdf/laser-visual-guidance-versus-two-dimensional-vision-in-5e7w7evoim.pdf

Laser visual guidance versus two-dimensional vision in laparoscopy: a randomized trial

Purpose Point of care ultrasound (POCUS) is widely used, but the sensitivity and specificity of the findings are highly user-dependent. There are many different approaches to ultrasound training. The aim of this study was to explore the effects of hands-on practice when learning POCUS. Methods Junior doctors with no or limited ultrasound experience were included in the study and divided into three groups. They all completed a Focused Assessment with Sonography for Trauma (FAST) course with different amounts of hands-on practice: 40 minutes (n=67), 60 minutes (n=12), and 90 minutes of hands-on time (n=27). By the end of the course, they all completed a previously validated test. Results More hands-on time improved the mean test scores and decreased the test time. The scores of the 40-, 60-, and 90-minute groups were 11.6 (SD 2.1), 12.8 (SD 2.5), and 13.7 (SD 2.5), respectively (p<0.001). The 90-minute group completed the test significantly faster than the other two groups (20 versus 26 minutes, p=0.003). A large inter-individual variation was seen. Conclusion The necessary amount of hands-on training is unknown. This study demonstrates that performance increases with prolonged hands-on time but the inter-individual variation among trainees is very large, thereby making it impossible to define the “optimal” time. This supports the use of the concept of mastery learning where each individual trainee can continue training until proficiency is reached.

/pdf/hands-on-time-in-simulation-based-ultrasound-training-a-dose-3db2yfa6.pdf

Oria Mahmood

Papers

Direct Observation vs. Video-Based Assessment in Flexible Cystoscopy.

Nonspecialist Raters Can Provide Reliable Assessments of Procedural Skills

Simulation-based training for flexible cystoscopy - A randomized trial comparing two approaches.

Laser visual guidance versus two-dimensional vision in laparoscopy: a randomized trial

Hands-On Time in Simulation-Based Ultrasound Training – A Dose-Related Response Study