Showing papers in "Medical Education in 2015"

A contemporary approach to validity arguments: a practical guide to Kane's framework.

Abstract: Context Assessment is central to medical education and the validation of assessments is vital to their use. Earlier validity frameworks suffer from a multiplicity of types of validity or failure to prioritise among sources of validity evidence. Kane's framework addresses both concerns by emphasising key inferences as the assessment progresses from a single observation to a final decision. Evidence evaluating these inferences is planned and presented as a validity argument. Objectives We aim to offer a practical introduction to the key concepts of Kane's framework that educators will find accessible and applicable to a wide range of assessment tools and activities. Results All assessments are ultimately intended to facilitate a defensible decision about the person being assessed. Validation is the process of collecting and interpreting evidence to support that decision. Rigorous validation involves articulating the claims and assumptions associated with the proposed decision (the interpretation/use argument), empirically testing these assumptions, and organising evidence into a coherent validity argument. Kane identifies four inferences in the validity argument: Scoring (translating an observation into one or more scores); Generalisation (using the score[s] as a reflection of performance in a test setting); Extrapolation (using the score[s] as a reflection of real-world performance), and Implications (applying the score[s] to inform a decision or action). Evidence should be collected to support each of these inferences and should focus on the most questionable assumptions in the chain of inference. Key assumptions (and needed evidence) vary depending on the assessment's intended use or associated decision. Kane's framework applies to quantitative and qualitative assessments, and to individual tests and programmes of assessment. Conclusions Validation focuses on evaluating the key claims, assumptions and inferences that link assessment scores with their intended interpretations and uses. The Implications and associated decisions are the most important inferences in the validity argument.

Implementation of competency-based medical education: are we addressing the concerns and challenges?

Abstract: Context Competency-based medical education (CBME) has emerged as a core strategy to educate and assess the next generation of physicians. Advantages of CBME include: a focus on outcomes and learner achievement; requirements for multifaceted assessment that embraces formative and summative approaches; support of a flexible, time-independent trajectory through the curriculum; and increased accountability to stakeholders with a shared set of expectations and a common language for education, assessment and regulation. Objectives Despite the advantages of CBME, numerous concerns and challenges to the implementation of CBME frameworks have been described, including: increased administrative requirements; the need for faculty development; the lack of models for flexible curricula, and inconsistencies in terms and definitions. Additionally, there are concerns about reductionist approaches to assessment in CBME, lack of good assessments for some competencies, and whether CBME frameworks include domains of current importance. This study will outline these issues and discuss the responses of the medical education community. Methods The concerns and challenges expressed are primarily categorised as: (i) those related to practical, administrative and logistical challenges in implementing CBME frameworks, and (ii) those with more conceptual or theoretical bases. The responses of the education community to these issues are then summarised. Conclusions The education community has begun to address the challenges involved in implementing CBME. Models and guidance exist to inform implementation strategies across the continuum of education, and focus on the more efficient use of resources and technology, and the use of milestones and entrustable professional activities-based frameworks. Inconsistencies in CBME definitions and frameworks remain a significant obstacle. Evolution in assessment approaches from in vitro task-based methods to in vivo integrated approaches is responsive to many of the theoretical and conceptual concerns about CBME, but much work remains to be done to bring rigour and quality to work-based assessment.

A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment

Abstract: Context The relative advantages and disadvantages of checklists and global rating scales (GRSs) have long been debated. To compare the merits of these scale types, we conducted a systematic review of the validity evidence for checklists and GRSs in the context of simulation-based assessment of health professionals. Methods We conducted a systematic review of multiple databases including MEDLINE, EMBASE and Scopus to February 2013. We selected studies that used both a GRS and checklist in the simulation-based assessment of health professionals. Reviewers working in duplicate evaluated five domains of validity evidence, including correlation between scales and reliability. We collected information about raters, instrument characteristics, assessment context, and task. We pooled reliability and correlation coefficients using random-effects meta-analysis. Results We found 45 studies that used a checklist and GRS in simulation-based assessment. All studies included physicians or physicians in training; one study also included nurse anaesthetists. Topics of assessment included open and laparoscopic surgery (n = 22), endoscopy (n = 8), resuscitation (n = 7) and anaesthesiology (n = 4). The pooled GRS–checklist correlation was 0.76 (95% confidence interval [CI] 0.69–0.81, n = 16 studies). Inter-rater reliability was similar between scales (GRS 0.78, 95% CI 0.71–0.83, n = 23; checklist 0.81, 95% CI 0.75–0.85, n = 21), whereas GRS inter-item reliabilities (0.92, 95% CI 0.84–0.95, n = 6) and inter-station reliabilities (0.80, 95% CI 0.73–0.85, n = 10) were higher than those for checklists (0.66, 95% CI 0–0.84, n = 4 and 0.69, 95% CI 0.56–0.77, n = 10, respectively). Content evidence for GRSs usually referenced previously reported instruments (n = 33), whereas content evidence for checklists usually described expert consensus (n = 26). Checklists and GRSs usually had similar evidence for relations to other variables. Conclusions Checklist inter-rater reliability and trainee discrimination were more favourable than suggested in earlier work, but each task requires a separate checklist. Compared with the checklist, the GRS has higher average inter-item and inter-station reliability, can be used across multiple tasks, and may better capture nuanced elements of expertise.

How to improve the teaching of clinical reasoning: A narrative review and a proposal

Abstract: Context The development of clinical reasoning (CR) in students has traditionally been left to clinical rotations, which, however, often offer limited practice and suboptimal supervision. Medical schools begin to address these limitations by organising pre-clinical CR courses. The purpose of this paper is to review the variety of approaches employed in the teaching of CR and to present a proposal to improve these practices. Methods We conducted a narrative review of the literature on teaching CR. To that end, we searched PubMed and Web of Science for papers published until June 2014. Additional publications were identified in the references cited in the initial papers. We used theoretical considerations to characterise approaches and noted empirical findings, when available. Results Of the 48 reviewed papers, only 24 reported empirical findings. The approaches to teaching CR were shown to vary on two dimensions. The first pertains to the way the case information is presented. The case is either unfolded to students gradually – the ‘serial-cue’ approach – or is presented in a ‘whole-case’ format. The second dimension concerns the purpose of the exercise: is its aim to help students acquire or apply knowledge, or is its purpose to teach students a way of thinking? The most prevalent approach is the serial-cue approach, perhaps because it tries to directly simulate the diagnostic activities of doctors. Evidence supporting its effectiveness is, however, lacking. There is some empirical evidence that whole-case, knowledge-oriented approaches contribute to the improvement of students’ CR. However, thinking process-oriented approaches were shown to be largely ineffective. Conclusions Based on research on how expertise develops in medicine, we argue that students in different phases of their training may benefit from different approaches to the teaching of CR.

The impact of programmatic assessment on student learning: theory versus practice

Abstract: CONTEXT: It is widely acknowledged that assessment can affect student learning. In recent years, attention has been called to 'programmatic assessment', which is intended to optimise both learning functions and decision functions at the programme level of assessment, rather than according to individual methods of assessment. Although the concept is attractive, little research into its intended effects on students and their learning has been conducted. OBJECTIVES: This study investigated the elements of programmatic assessment that students perceived as supporting or inhibiting learning, and the factors that influenced the active construction of their learning. METHODS: The study was conducted in a graduate-entry medical school that implemented programmatic assessment. Thus, all assessment information, feedback and reflective activities were combined into a comprehensive, holistic programme of assessment. We used a qualitative approach and interviewed students (n = 17) in the pre-clinical phase of the programme about their perceptions of programmatic assessment and learning approaches. Data were scrutinised using theory-based thematic analysis. RESULTS: Elements from the comprehensive programme of assessment, such as feedback, portfolios, assessments and assignments, were found to have both supporting and inhibiting effects on learning. These supporting and inhibiting elements influenced students' construction of learning. Findings showed that: (i) students perceived formative assessment as summative; (ii) programmatic assessment was an important trigger for learning, and (iii) the portfolio's reflective activities were appreciated for their generation of knowledge, the lessons drawn from feedback, and the opportunities for follow-up. Some students, however, were less appreciative of reflective activities. For these students, the elements perceived as inhibiting seemed to dominate the learning response. CONCLUSIONS: The active participation of learners in their own learning is possible when learning is supported by programmatic assessment. Certain features of the comprehensive programme of assessment were found to influence student learning, and this influence can either support or inhibit students' learning responses.

Reclaiming a theoretical orientation to reflection in medical education research: a critical narrative review

Abstract: Context Reflection and reflective practice have become popular topics of scholarly dialogue in medical education. This popularity has given rise to checklists, portfolios and other tools to inspire and document reflection. We argue that some of the common ways in which reflection has been applied are influenced by broader discourses of assessment and evidence, and divorced from original theories of reflection and reflective practice. Methods This paper was developed using a critical narrative approach. First we present two theoretical lenses provided by theories of reflection. Next we present a summary of relevant literature, indexed in PubMed from 2004 to 2014, relating to the application of reflection or reflective practice to undergraduate and postgraduate medical education. We categorise these articles broadly by trends and problematise the trends relative to the two theoretical lenses of reflection. Results Two relevant theoretical orientations of reflection for medical education are: (i) reflection as epistemology of practice, and (ii) reflection as critical social inquiry. Three prevalent trends in the application of reflection to medical education are: (i) utilitarian applications of reflection; (ii) a focus on the self as the object of reflection, and (iii) reflection and assessment. These trends align with dominant epistemological positions in medicine, but not with those that underpin reflection. Conclusions We argue for continued theorising of and theoretically informed applications of reflection, drawing upon epistemologies of practice and critical reflection as critical social inquiry. These directions offer medical education research broad and deep potential in theories of reflection, particularly in relation to knowledge creation within uncertain and complex situations, and challenging of dominant discourses and structures. Future work could explore how dominant epistemological positions and discourses in medicine influence theories from other disciplines when these theories are deployed in medical education.

Student experiences across multiple flipped courses in a single curriculum

Abstract: Context The flipped classroom approach has garnered significant attention in health professions education, which has resulted in calls for curriculum-wide implementations of the model. However, research to support the development of evidence-based guidelines for large-scale flipped classroom implementations is lacking. Objectives This study was designed to examine how students experience the flipped classroom model of learning in multiple courses within a single curriculum, as well as to identify specific elements of flipped learning that students perceive as beneficial or challenging. Methods A qualitative analysis of students’ comments (n = 6010) from mid-course and end-of-course evaluations of 10 flipped courses (in 2012–2014) was conducted. Common and recurring themes were identified through systematic iterative coding and sorting using the constant comparison method. Multiple coders, agreement through consensus and member checking were utilised to ensure the trustworthiness of findings. Results Several themes emerged from the analysis: (i) the perceived advantages of flipped learning coupled with concerns about implementation; (ii) the benefits of pre-class learning and factors that negatively affect these benefits, such as quality and quantity of learning materials, as well as overall increase in workload, especially in the context of multiple concurrent flipped courses; (iii) the role of the instructor in the flipped learning environment, particularly in engaging students in active learning and ensuring instructional alignment, and (iv) the need for assessments that emphasise the application of knowledge and critical thinking skills. Conclusions Analysis of data from 10 flipped courses provided insight into common patterns of student learning experiences specific to the flipped learning model within a single curriculum. The study points to the challenges associated with scaling the implementation of the flipped classroom across multiple courses. Several core elements critical to the effective design and implementation of the flipped classroom model are identified.

Louder than words: power and conflict in interprofessional education articles, 1954–2013

Abstract: Context Interprofessional education (IPE) aspires to enable collaborative practice. Current IPE offerings, although rapidly proliferating, lack evidence of efficacy and theoretical grounding.

Variables that affect the process and outcome of feedback, relevant for medical training: a meta-review.

Abstract: Context: Feedback is considered important in medical education. The literature is not clear about the mechanisms that contribute to its effects, which are often small to moderate and at times contradictory. A variety of variables seem to influence the impact of feedback on learning. The aim of this study was to determine which variables influence the process and outcomes of feedback in settings relevant to medical education. Methods: A myriad of studies on feedback have been conducted. To determine the most researched variables, we limited our review to meta-analyses and literature reviews published in the period from January 1986 to February 2012. According to our protocol, we first identified features of the feedback process that influence its effects and subsequently variables that influence these features. We used a chronological model of the feedback process to categorise all variables found. Results: A systematic search of ERIC, PsycINFO and MEDLINE yielded 1101 publications, which we reduced to 203, rejecting papers on six exclusion criteria. Of these, 46 met the inclusion criteria. In our four-phase model, we identified 33 variables linked to task performance (e.g. task complexity, task nature) and feedback reception (e.g. self-esteem, goal-setting behaviour) by trainees, and to observation (e.g. focus, intensity) and feedback provision (e.g. form, content) by supervisors that influence the subsequent effects of the feedback process. Variables from all phases influence the feedback process and effects, but variables that influence the quality of the observation and rating of the performance dominate the literature. There is a paucity of studies addressing other, seemingly relevant variables. Conclusions: The larger picture of variables that influence the process and outcome of feedback, relevant for medical education, shows many open spaces. We suggest that targeted studies be carried out to expand our knowledge of these important aspects of feedback in medical education.

Limitations of subjective cognitive load measures in simulation‐based procedural training

Abstract: Context The effective implementation of cognitive load theory (CLT) to optimise the instructional design of simulation-based training requires sensitive and reliable measures of cognitive load. This mixed-methods study assessed relationships between commonly used measures of total cognitive load and the extent to which these measures reflected participants’ experiences of cognitive load in simulation-based procedural skills training. Methods Two groups of medical residents (n = 38) completed three questionnaires after participating in simulation-based procedural skills training sessions: the Paas Cognitive Load Scale; the NASA Task Load Index (TLX), and a cognitive load component (CLC) questionnaire we developed to assess total cognitive load as the sum of intrinsic load (how complex the task is), extraneous load (how the task is presented) and germane load (how the learner processes the task for learning). We calculated Pearson's correlation coefficients to assess agreement among these instruments. Group interviews explored residents’ perceptions about how the simulation sessions contributed to their total cognitive load. Interviews were audio-recorded, transcribed and subjected to qualitative content analysis. Results Total cognitive load scores differed significantly according to the instrument used to assess them. In particular, there was poor agreement between the Paas Scale and the TLX. Quantitative and qualitative findings supported intrinsic cognitive load as synonymous with mental effort (Paas Scale), mental demand (TLX) and task difficulty and complexity (CLC questionnaire). Additional qualitative themes relating to extraneous and germane cognitive loads were not reflected in any of the questionnaires. Conclusions The Paas Scale, TLX and CLC questionnaire appear to be interchangeable as measures of intrinsic cognitive load, but not of total cognitive load. A more complete understanding of the sources of extraneous and germane cognitive loads in simulation-based training contexts is necessary to determine how best to measure and assess their effects on learning and performance outcomes.

Exploring the impact of workplace cyberbullying on trainee doctors.

Abstract: OBJECTIVES: Workplace bullying is an occupational hazard for trainee doctors. However, little is known about their experiences of cyberbullying at work. This study examines the impact of cyberbullying among trainee doctors, and how attributions of blame for cyberbullying influence individual and work-related outcomes. METHODS: Doctors at over 6 months into training were asked to complete an online survey that included measures of cyberbullying, blame attribution, negative emotion, job satisfaction, interactional justice and mental strain. A total of 158 trainee doctors (104 women, 54 men) completed the survey. RESULTS: Overall, 73 (46.2%) respondents had experienced at least one act of cyberbullying. Cyberbullying adversely impacted on job satisfaction (β = - 0.19; p CONCLUSIONS: Acts of cyberbullying had been experienced by nearly half of the sample during their training and were found to significantly relate to ill health and job dissatisfaction. The deleterious impact of cyberbullying can be addressed through both workplace policies, and training for trainee doctors and experienced medical professionals. Language: en

Self-regulated learning in simulation-based training: a systematic review and meta-analysis.

Abstract: Context Self-regulated learning (SRL) requires an active learner who has developed a set of processes for managing the achievement of learning goals. Simulation-based training is one context in which trainees can safely practise learning how to learn. Objectives The purpose of the present study was to evaluate, in the simulation-based training context, the effectiveness of interventions designed to support trainees in SRL activities. We used the social-cognitive model of SRL to guide a systematic review and meta-analysis exploring the links between instructor supervision, supports or scaffolds for SRL, and educational outcomes. Methods We searched databases including MEDLINE and Scopus, and previous reviews, for material published until December 2011. Studies comparing simulation-based SRL interventions with another intervention for teaching health professionals were included. Reviewers worked independently and in duplicate to extract information on learners, study quality and educational outcomes. We used random-effects meta-analysis to compare the effects of supervision (instructor present or absent) and SRL educational supports (e.g. goal-setting study guides present or absent). Results From 11 064 articles, we included 32 studies enrolling 2482 trainees. Only eight of the 32 studies included educational supports for SRL. Compared with instructor-supervised interventions, unsupervised interventions were associated with poorer immediate post-test outcomes (pooled effect size: −0.34, p = 0.09; n = 19 studies) and negligible effects on delayed (i.e. > 1 week) retention tests (pooled effect size: 0.11, p = 0.63; n = 8 studies). Interventions including SRL supports were associated with small benefits compared with interventions without supports on both immediate post-tests (pooled effect size: 0.23, p = 0.22; n = 5 studies) and delayed retention tests (pooled effect size: 0.44, p = 0.067; n = 3 studies). Conclusions Few studies in the simulation literature have designed SRL training to explicitly support trainees’ capacity to self-regulate their learning. We recommend that educators and researchers shift from thinking about SRL as learning alone to thinking of SRL as comprising a shared responsibility between the trainee and the instructional designer (i.e. learning using designed supports that help prepare individuals for future learning).

Reading between the lines: faculty interpretations of narrative evaluation comments.

Abstract: Objectives Narrative comments are used routinely in many forms of rater-based assessment. Interpretation can be difficult as a result of idiosyncratic writing styles and disconnects between literal and intended meanings. Our purpose was to explore how faculty attendings interpret and make sense of the narrative comments on residents’ in-training evaluation reports (ITERs) and to determine the language cues that appear to be influential in generating and justifying their interpretations. Methods A group of 24 internal medicine (IM) faculty attendings each categorised a subgroup of postgraduate year 1 (PGY1) and PGY2 IM residents based solely on ITER comments. They were then interviewed to determine how they had made their judgements. Constant comparative techniques from constructivist grounded theory were used to analyse the interviews and develop a framework to help in understanding how ITER language was interpreted. Results The overarching theme of ‘reading between the lines’ explained how participants read and interpreted ITER comments. Scanning for ‘flags’ was part of this strategy. Participants also described specific factors that shaped their judgements, including: consistency of comments; competency domain; specificity; quantity, and context (evaluator identity, rotation type and timing). There were several perceived purposes of ITER comments, including feedback to the resident, summative assessment and other more socially complex objectives. Conclusions Participants made inferences based on what they thought evaluators intended by their comments and seemed to share an understanding of a ‘hidden code’. Participants’ ability to ‘read between the lines’ explains how comments can be effectively used to categorise and rank-order residents. However, it also suggests a mechanism whereby variable interpretations can arise. Our findings suggest that current assumptions about the purpose, value and effectiveness of ITER comments may be incomplete. Linguistic pragmatics and politeness theories may shed light on why such an implicit code might evolve and be maintained in clinical evaluation.

A critical appraisal of instruments to measure outcomes of interprofessional education

Abstract: Context Interprofessional education (IPE) is believed to prepare health professional graduates for successful collaborative practice. A range of instruments have been developed to measure the outcomes of IPE. An understanding of the psychometric properties of these instruments is important if they are to be used to measure the effectiveness of IPE. Objectives This review set out to identify instruments available to measure outcomes of IPE and collaborative practice in pre-qualification health professional students and to critically appraise the psychometric properties of validity, responsiveness and reliability against contemporary standards for instrument design. Methods Instruments were selected from a pool of extant instruments and subjected to critical appraisal to determine whether they satisfied inclusion criteria. The qualitative and psychometric attributes of the included instruments were appraised using a checklist developed for this review. Results Nine instruments were critically appraised, including the widely adopted Readiness for Interprofessional Learning Scale (RIPLS) and the Interdisciplinary Education Perception Scale (IEPS). Validity evidence for instruments was predominantly based on test content and internal structure. Ceiling effects and lack of scale width contribute to the inability of some instruments to detect change in variables of interest. Limited reliability data were reported for two instruments. Scale development and scoring protocols were generally reported by instrument developers, but the inconsistent application of scoring protocols for some instruments was apparent. Conclusions A number of instruments have been developed to measure outcomes of IPE in pre-qualification health professional students. Based on reported validity evidence and reliability data, the psychometric integrity of these instruments is limited. The theoretical test construction paradigm on which instruments have been developed may be contributing to the failure of some instruments to detect change in variables of interest following an IPE intervention. These limitations should be considered in any future research on instrument design.

Learning in student-run clinics: a systematic review

Abstract: Context Student-run clinics (SRCs) have existed for many years and may provide the most realistic setting for context-based learning and legitimate early clinical experiences with responsibility for patient care. We reviewed the literature on student outcomes of participation in SRCs. Methods A systematic literature review was performed using the PubMed, EMBASE, PsycINFO and ERIC databases. Included articles were reviewed for conclusions and outcomes; study quality was assessed using the Medical Education Research Study Quality Instrument (MERSQI). Results A total of 42 articles met the inclusion criteria and were included in the quantitative synthesis. The effects of participation on students’ attitudes were mainly positive: students valued the SRC experience. Data on the effects of SRC participation on students’ skills and knowledge were based mainly on expert opinions and student surveys. Students reported improved skills and indicated that they had acquired knowledge they were unlikely to have gained elsewhere in the curriculum. The quality of specific aspects of care delivered by students was comparable with that of regular care. Conclusions The suggestion that students should be trained as medical professionals with responsibility for patient care early in the curriculum is attractive. In an SRC this responsibility is central. Students valued the early training opportunity in SRCs and liked participating. However, little is known about the effect of SRC participation on students’ skills and knowledge. The quality of care provided by students seemed adequate. Further research is needed to assess the effect of SRC participation on students’ skills, knowledge and behaviour.

Measuring cognitive load: performance, mental effort and simulation task complexity.

Abstract: Context Interest in applying cognitive load theory in health care simulation is growing. This line of inquiry requires measures that are sensitive to changes in cognitive load arising from different instructional designs. Recently, mental effort ratings and secondary task performance have shown promise as measures of cognitive load in health care simulation. Objectives We investigate the sensitivity of these measures to predicted differences in intrinsic load arising from variations in task complexity and learner expertise during simulation-based surgical skills training. Methods We randomly assigned 28 novice medical students to simulation training on a simple or complex surgical knot-tying task. Participants completed 13 practice trials, interspersed with computer-based video instruction. On trials 1, 5, 9 and 13, knot-tying performance was assessed using time and movement efficiency measures, and cognitive load was assessed using subjective rating of mental effort (SRME) and simple reaction time (SRT) on a vibrotactile stimulus-monitoring secondary task. Results Significant improvements in knot-tying performance (F(1.04,24.95) = 41.1, p < 0.001 for movements; F(1.04,25.90) = 49.9, p < 0.001 for time) and reduced cognitive load (F(2.3,58.5) = 57.7, p < 0.001 for SRME; F(1.8,47.3) = 10.5, p < 0.001 for SRT) were observed in both groups during training. The simple-task group demonstrated superior knot tying (F(1,24) = 5.2, p = 0.031 for movements; F(1,24) = 6.5, p = 0.017 for time) and a faster decline in SRME over the first five trials (F(1,26) = 6.45, p = 0.017) compared with their peers. Although SRT followed a similar pattern, group differences were not statistically significant. Conclusions Both secondary task performance and mental effort ratings are sensitive to changes in intrinsic load among novices engaged in simulation-based learning. These measures can be used to track cognitive load during skills training. Mental effort ratings are also sensitive to small differences in intrinsic load arising from variations in the physical complexity of a simulation task. The complementary nature of these subjective and objective measures suggests their combined use is advantageous in simulation instructional design research.

Towards socio-material approaches in simulation-based education: lessons from complexity theory

Abstract: ContextReview studies of simulation-based education (SBE) consistently point out that theory-driven research is lacking. The literature to date is dominated by discourses of fidelity and authenticity - creating the real - with a strong focus on the developing of clinical procedural skills. Little of this writing incorporates the theory and research proliferating in professional studies more broadly, which show how professional learning is embodied, relational and situated in social - material relations. A key concern for medical educators concerns how to better prepare students for the unpredictable and dynamic ambiguity of professional practice; this has stimulated the movement towards socio-material theories in education that address precisely this question. Objectives and MethodsAmong the various socio-material theories that are informing new developments in professional education, complexity theory has been of particular importance for medical educators interested in updating current practices. This paper outlines key elements of complexity theory, illustrated with examples from empirical study, to argue its particular relevance for improving SBE. ResultsComplexity theory can make visible important material dynamics, and their problematic consequences, that are not often noticed in simulated experiences in medical training. It also offers conceptual tools that can be put to practical use. This paper focuses on concepts of emergence, attunement, disturbance and experimentation. These suggest useful new approaches for designing simulated settings and scenarios, and for effective pedagogies before, during and following simulation sessions. ConclusionsSocio-material approaches such as complexity theory are spreading through research and practice in many aspects of professional education across disciplines. Here, we argue for the transformative potential of complexity theory in medical education using simulation as our focus. Complexity tools open questions about the socio-material contradictions inherent in SBE, draw attention to important material dynamics of emergence, and suggest practical educative ways to expand and deepen student learning. Discuss ideas arising from the article at discuss.

How clinical supervisors develop trust in their trainees: a qualitative study.

Abstract: Context Clinical supervisors oversee trainees’ performance while granting them increasing opportunities to work independently. Although the factors contributing to supervisors’ trust in their trainees to conduct clinical work have been identified, how the development of trust is shaped by these factors remains less clear. Objectives This study was designed to determine how supervisors develop and experience trust in resident (postgraduate years 2 and 3) trainees in the clinical workplace. Methods Internal medicine in-patient supervisors at two institutions were interviewed about the meaning and experience of developing trust in resident trainees. Transcribed data were coded and analysed using a phenomenographic approach. Results Forty-three supervisors participated. Supervisors characterised the meaning of trust from the perspectives of trainee competence and leadership or from their own perspective of needing to provide more or less supervision. Supervisors initially considered trust to be usually independent of prior knowledge of the resident, and then used sources of information about trust to develop their judgements of trust. Sources, which incorporated inference, included supervisors’ comparisons with a standard, direct observation of the trainee as a team leader or care provider, and stakeholder input from team members, patients and families. Barriers against and accelerators to trust formation related to the resident, supervisor, resident–supervisor relationship, context and task. Trust formation had implications for supervisors’ roles, residents’ increasingly independent provision of care, and team functioning. Conclusions From a general starting point, supervisors develop trust in residents informed by observation, inference and information gathered from the team and patients. Judgements of trust yield outcomes defined by supervisors’ changing roles, the increasingly independent provision of care by residents, and team functioning. The implications of these findings for graded resident autonomy aligned with learning needs can inform the design of training environments to enable readiness for unsupervised practice.

Seeking inclusion in an exclusive process: discourses of medical school student selection.

Abstract: Context Calls to increase medical class representativeness to better reflect the diversity of society represent a growing international trend. There is an inherent tension between these calls and competitive student selection processes driven by academic achievement. How is this tension manifested? Methods Our three-phase interdisciplinary research programme focused on the discourses of excellence, equity and diversity in the medical school selection process, as conveyed by key stakeholders: (i) institutions and regulatory bodies (the websites of 17 medical schools and 15 policy documents from national regulatory bodies); (ii) admissions committee members (ACMs) (according to semi-structured interviews [n = 9]), and (iii) successful applicants (according to semi-structured interviews [n = 14]). The work is theoretically situated within the works of Foucault, Bourdieu and Bakhtin. The conceptual framework is supplemented by critical hermeneutics and the performance theories of Goffman. Results Academic excellence discourses consistently predominate over discourses calling for greater representativeness in medical classes. Policy addressing demographic representativeness in medicine may unwittingly contribute to the reproduction of historical patterns of exclusion of under-represented groups. In ACM selection practices, another discursive tension is exposed as the inherent privilege in the process is marked, challenging the ideal of medicine as a meritocracy. Applicants' representations of self in the ‘performance’ of interviewing demonstrate implicit recognition of the power inherent in the act of selection and are manifested in the use of explicit strategies to ‘fit in’. Conclusions How can this critical discourse analysis inform improved inclusiveness in student selection? Policymakers addressing diversity and equity issues in medical school admissions should explicitly recognise the power dynamics at play between the profession and marginalised groups. For greater inclusion and to avoid one authoritative definition of excellence, we suggest a transformative model of faculty development aimed at promoting multiple kinds of excellence. Through this multi-pronged approach, we call for the profession to courageously confront the cherished notion of the medical meritocracy in order to avoid unwanted aspects of elitism.

Exploring the factors influencing clinical students’ self‐regulated learning

Abstract: OBJECTIVES: The importance of self-regulated learning (SRL) has been broadly recognised by medical education institutions and regulatory bodies. Supporting the development of SRL skills has proven difficult because self-regulation is a complex interactive process and we know relatively little about the factors influencing this process in real practice settings. The aim of our study was therefore to identify factors that support or hamper medical students' SRL in a clinical context. METHODS: We conducted a constructivist grounded theory study using semi-structured interviews with 17 medical students from two universities enrolled in clerkships. Participants were purposively sampled to ensure variety in age, gender, experience and current clerkship. The Day Reconstruction Method was used to help participants remember their activities of the previous day. The interviews were transcribed verbatim and analysed iteratively using constant comparison and open, axial and interpretive coding. RESULTS: Self-regulated learning by students in the clinical environment was influenced by the specific goals perceived by students, the autonomy they experienced, the learning opportunities they were given or created themselves, and the anticipated outcomes of an activity. All of these factors were affected by personal, contextual and social attributes. CONCLUSIONS: Self-regulated learning of medical students in the clinical environment is different for every individual. The factors influencing this process are affected by personal, social and contextual attributes. Some of these are similar to those known from previous research in classroom settings, but others are unique to the clinical environment and include the facilities available, the role of patients, and social relationships pertaining to peers and other hospital staff. To better support students' SRL, we believe it is important to increase students' metacognitive awareness and to offer students more tailored learning opportunities.

The mixed impact of medical school on medical students’ implicit and explicit weight bias

Abstract: Context Health care trainees demonstrate implicit (automatic, unconscious) and explicit (conscious) bias against people from stigmatised and marginalised social groups, which can negatively influence communication and decision making. Medical schools are well positioned to intervene and reduce bias in new physicians. Objectives This study was designed to assess medical school factors that influence change in implicit and explicit bias against individuals from one stigmatised group: people with obesity. Methods This was a prospective cohort study of medical students enrolled at 49 US medical schools randomly selected from all US medical schools within the strata of public and private schools and region. Participants were 1795 medical students surveyed at the beginning of their first year and end of their fourth year. Web-based surveys included measures of weight bias, and medical school experiences and climate. Bias change was compared with changes in bias in the general public over the same period. Linear mixed models were used to assess the impact of curriculum, contact with people with obesity, and faculty role modelling on weight bias change. Results Increased implicit and explicit biases were associated with less positive contact with patients with obesity and more exposure to faculty role modelling of discriminatory behaviour or negative comments about patients with obesity. Increased implicit bias was associated with training in how to deal with difficult patients. On average, implicit weight bias decreased and explicit bias increased during medical school, over a period of time in which implicit weight bias in the general public increased and explicit bias remained stable. Conclusions Medical schools may reduce students’ weight biases by increasing positive contact between students and patients with obesity, eliminating unprofessional role modelling by faculty members and residents, and altering curricula focused on treating difficult patients.

Emotions and identity in the figured world of becoming a doctor.

Abstract: Context There is little room in clerkship curricula for students to express emotions, particularly those associated with the development of a caring identity. Yet it is recognised that competence, alone, does not make a good doctor. We therefore set out to explore the relationship between emotions and identity in clerkship education. Our exploration was conceptually oriented towards Figured Worlds theory, which is linked to Bakhtin's theory of dialogism. Methods Nine female and one male member of a mixed student cohort kept audio-diaries and participated in both semi-structured and cognitive individual interviews. The researchers identified 43 emotionally salient utterances in the dataset and subjected them to critical discourse analysis. They applied Figured Worlds constructs to within-case and cross-case analyses, supporting one another's reflexivity and openness to different interpretations, and constantly comparing their evolving interpretation against the complete set of transcripts. Results Students’ emotions were closely related to their identity development in the world of medicine. Patients were disempowered by their illnesses. Doctors were powerful because they could treat those illnesses. Students expressed positive emotions when they were granted positions in the world of medicine and were able to identify with the figures of doctors or other health professionals. They identified with doctors who behaved in caring and professionally appropriate ways towards patients and supportively towards students. Students expressed negative emotions when they were unable to develop their identities. Conclusions Critical discourse analysis has uncovered a link between students’ emotions and their identity development in the powerful world of becoming and being a doctor. At present, identity development, emotions and power are mostly tacit in undergraduate clinical curricula. We speculate that helping students to express emotions and exercise power in the most effective ways might help them to develop caring identities.

Impact of selection strategies on representation of underserved populations and intention to practise: international findings

Abstract: Context Socially accountable medical schools aim to reduce health inequalities by training workforces responsive to the priority health needs of underserved communities. One key strategy involves recruiting students from underserved and unequally represented communities on the basis that they may be more likely to return and address local health priorities. This study describes the impacts of different selection strategies of medical schools that aspire to social accountability on the presence of students from underserved communities in their medical education programmes and on student practice intentions. Methods A cross-sectional questionnaire was administered to students starting medical education in five institutions with a social accountability mandate in five different countries. The questionnaire assessed students’ background characteristics, rurality of background, and practice intentions (location, discipline of practice and population to be served). The results were compared with the characteristics of students entering medical education in schools with standard selection procedures, and with publicly available socio-economic data. Results The selection processes of all five schools included strategies that extended beyond the assessment of academic achievement. Four distinct strategies were identified: the quota system; selection based on personal attributes; community involvement, and school marketing strategies. Questionnaire data from 944 students showed that students at the five schools were more likely to be of non-urban origin, of lower socio-economic status and to come from underserved groups. A total of 407 of 810 (50.2%) students indicated an intention to practise in a non-urban area after graduation and the likelihood of this increased with increasing rurality of primary schooling (p = 0.000). Those of rural origin were statistically less likely to express an intention to work abroad (p = 0.003). Conclusions Selection strategies to ensure that members of underserved communities can pursue medical careers can be effective in achieving a fair and equitable representation of underserved communities within the student body. Such strategies may contribute to a diverse medical student body with strong intentions to work with underserved populations.

The interaction of socio-economic status and gender in widening participation in medicine.

Abstract: Context The lack of representation of people from low socio-economic and socio-educational backgrounds in the medical profession is of growing concern and yet research investigating the problem typically studies recruitment and selection in isolation. This study examines the impacts of home and school socio-economic status (SES) from application to selection in an undergraduate medical degree. Socio-cognitive career theory and stereotype bias are used to explain why those from backgrounds of low SES may be disadvantaged, especially if they are female. Methods Home and high school SES information for 2955 applicants and 202 medical students at one Australian medical school was related to application rates and performance on three selection tests (high school matriculation, the Undergraduate Medical and Health Sciences Admissions Test [UMAT] cognitive ability test, a multiple mini-interview) and academic performance in medical school. Interactions between gender and SES were assessed using moderated regression analyses. Results Applicants from backgrounds of low SES were under-represented. They were further disadvantaged at selection by the use of high school matriculation and cognitive ability tests, but not by the interview. They did not perform more poorly in medical school. Although females applied in greater numbers, a significant interaction between SES and gender indicated that female applicants of low SES were the most disadvantaged by the use of cognitive ability testing at selection. A targeted allowance of applicants from regions of low SES overcame this adverse impact to some extent. Conclusions Efforts to widen participation that focus on recruitment are insufficient when selection tests have adverse impacts on people from backgrounds of low SES. The addressing of low self-efficacy that arises from socio-cultural factors, together with reductions in stereotype threat, may reduce the current disadvantages imposed by SES in the medical profession.

Bridging the gap: the roles of social capital and ethnicity in medical student achievement

Abstract: OBJECTIVES: Within medical education, there is a discrepancy between the achievement level of White students and that of their ethnic minority peers. The processes underlying this disparity have not been adequately investigated or explained. This study utilises social network analysis to investigate the impact of relationships on medical student achievement by ethnicity, specifically by examining homophily (the tendency to interact with others in the same group) by ethnicity, age and role. METHODS: Data from a cross-sectional social network study conducted in one UK medical school are presented and are analysed alongside examination records obtained from the medical school. Participants were sampled across the four hospital placement sites; a total of 158 medical students in their clinical phase (Years 3 and 4) completed the survey. The research was designed and analysed using social capital theory. RESULTS: Although significant patterns of ethnic and religious homophily emerged, no link was found between these factors and achievement. Interacting with problem-based learning (PBL) group peers in study-related activities, and having seniors in a wider academic support network were directly linked to better achievement. Students in higher academic quartiles were more likely to be named by members of their PBL group in study activities and to name at least one tutor or clinician in their network. Students from lower-achieving groups were least likely to have the social capital enabling, and resulting from, interactions with members of more expert social groups. CONCLUSIONS: Lower levels of the social capital that mediates interaction with peers, tutors and clinicians may be the cause of underperformance by ethnic minority students. Because of ethnic homophily, minority students may be cut off from potential and actual resources that facilitate learning and achievement.

'It was serendipity': a qualitative study of academic careers in medical education.

Abstract: Context: Despite a demand for educational expertise in medical universities, little is known of the roles of medical educators and the sustainability of academic careers in medical education. We examined the experiences and career paths of medical educators from diverse professional backgrounds seeking to establish, maintain and strengthen their careers in medical schools. Methods: Semi-structured interviews were conducted with 44 lead and early-career medical educators from all 21 Australian and New Zealand medical schools. Questions explored career beginnings, rewards and challenges. Transcripts underwent systematic coding and independent thematic analysis. Final themes were confirmed by iterative review and member checking. Analysis was informed by Bourdieu's concepts of field (a social space for hierarchical interactions), habitus (individual dispositions which influence social interactions) and capital (economic, symbolic, social and cultural forms of power). Results: Participants provided diverse accounts of what constitutes the practice of medical education. Serendipitous career entry and little commonality of professional backgrounds and responsibilities suggest an ambiguous habitus with ill-defined career pathways. Within the field of medicine as enacted in medical schools, educators have invisible yet essential roles, experiencing tension between service expectations, a lesser form of capital, and demands for more highly valued forms of scholarship. Participants reported increasing expectations to produce research and obtain postgraduate qualifications to enter and maintain their careers. Unable to draw upon cultural capital accrued from clinical work, non-clinician educators faced additional challenges. To strengthen their position, educators consciously built social capital through essential service relationships, capitalising on times when education takes precedence, such as curriculum renewal and accreditation. Conclusions: Bourdieu's theory provides insight into medical educator career paths and the positioning of medical education within medical schools. Medical educators have an indistinct practice, and limited cultural capital in the form of research outputs. In order to maintain and strengthen their careers, educators must create alternative sources of capital, through fostering collaborative alliances.

Team cohesiveness, team size and team performance in team-based learning teams

Abstract: Objectives The purpose of this study was to explore the relationships among variables associated with teams in team-based learning (TBL) settings and team outcomes. Methods We administered the National Board of Medical Examiners (NBME) Psychiatry Subject Test first to individuals and then to teams of Year three students at four medical schools that used TBL in their psychiatry core clerkships. Team cohesion was analysed using the Team Performance Scale (TPS). Bivariate correlation and linear regression analysis were used to analyse the relationships among team-level variables (mean individual TPS scores for each team, mean individual NBME scores of teams, team size, rotation and gender make-up) and team NBME test scores. A hierarchical linear model was used to test the effects of individual TPS and individual NBME test scores within each team, as well as the effects of the team-level variables of team size, team rotation and gender on team NBME test scores. Individual NBME test and TPS scores were nested within teams and treated as subsampling units. Results Individual NBME test scores and individual TPS scores were positively and statistically significantly (p < 0.01) associated with team NBME test scores, when team rotation, team size and gender make-up were controlled for. Higher team NBME test scores were associated with teams rotating later in the year and larger teams (p < 0.01). Gender make-up was not significantly associated. Conclusions The results of an NBME Psychiatry Subject Test administered to TBL teams at four medical schools suggest that larger teams on later rotations score higher on a team NBME test. Individual NBME test scores and team cohesion were positively and significantly associated with team NBME test scores. These results suggest the need for additional studies focusing on team outcomes, team cohesion, team size, rotation and other factors as they relate to the effective and efficient performance of TBL teams in health science education.

Gaining access to a research site and participants in medical and nursing research: a synthesis of accounts

Abstract: ObjectivesThe literature contains few accounts of how access to a research site and participants in medical and nursing research is gained, and few efforts to synthesise the existing accounts. Therefore, this article has two main goals: (i) to synthesise our own account of access with others in the medical and nursing literature, and (ii) to derive from this synthesis considerations of access and implications for health professions education. MethodsBased on field notes from a study conducted in an orthopaedic surgical section of a Norwegian university hospital, we provide an account of how access to the operating theatre (research site) and surgical teams (participants) was achieved. We synthesise the findings by comparing our account with existing accounts on access. ResultsOur synthesis translates into a number of considerations related to the gaining of access to a research site and participants in medical and nursing research. These include conducting continuous negotiations to ensure the agreement and consent of participants and gatekeepers, and demonstrating transparency regarding the researcher's identity and the nature of the particular project and findings. These considerations can raise awareness and preparedness for the process of gaining access, which will benefit health professions education researchers planning or undertaking studies in educational or clinical settings. Health professions educators can also include the considerations in structured educational programmes, specifically methods courses, to instil similar awareness and preparedness in students. ConclusionsWe suggest that future medical and nursing research should emphasise a detailed, unvarnished documentation of the access process that incorporates existing accounts of the process. Discuss ideas arising from the article at discuss.

Taking context seriously : explaining widening access policy enactments in UK medical schools

Abstract: Context Since the 1970s, the UK medical student body has become increasingly diverse in terms of gender, ethnicity and age, but not in socio-economic background. This variance may be linked to large differences in how individual medical schools interpret and put into practice widening participation (WP) policy. However, attempts to theorise what happens when policy enters practice are neglected in medical education. We aimed to explore the dynamics of policy enactment to give a novel perspective on WP practices across UK medical schools. Methods We used a qualitative design employing individual telephone interviews to elicit views and concerns around WP from admissions deans and admissions staff within UK medical schools. We carried out interviews with representatives from 24 of 32 UK medical schools. Data coding and analysis were initially inductive, using framework analysis. After the themes emerged, we applied a deductive framework to group themes into four contextual dimensions of ‘situation’, ‘professional’, ‘material’ and ‘external’. Results Our participants held different positions in relation to the interpreting and translating of WP policy, which were influenced by a number of contextual factors including: geographical locality and positioning of the medical school; the expectations of the university and other key stakeholders, and resources. The latter were subtle and referred to resources for medical selection processes rather than for WP per se. The data hinted that the political goal of WP and medical education's goal of producing the best doctors may conflict. Conclusions This is the first study to explicitly explore WP policy enactment in medical education. Our analysis is useful for understanding differences in how WP policy is played out in local contexts, and for planning for future policy enactment and research. The messages identified will resonate internationally with all those engaged in efforts to widen participation in medical education.

Disadvantage and the 'capacity to aspire' to medical school

Abstract: Objectives This study was designed to elucidate why students from backgrounds of lower socio-economic status (SES) and who may be first in their family (FIF) to enter university continue to be under-represented in medical schools. Methods Academically able high school students (n = 33) from a range of socio-economic backgrounds participated in focus groups. School careers advisors (n = 5) were interviewed. Students discussed their career and education plans and ideas about a medical career. Careers advisors discussed enablers and barriers to a medical career for their students. Results Students of lower SES and of FIF status attending schools situated in poorer geographic locations had limited access to suitable work experience and, despite their participation in gifted and talented classes, were considered to be at greater risk of not achieving the high level of academic achievement required for admission to medical school. Conclusions There is utility in exploring intersecting differences and Appardurai's theory of the ‘capacity to aspire’ for the purpose of understanding the causes of the under-representation of disadvantaged students in medical schools. A focused materialist approach to building the aspirations of disadvantaged students, particularly those attending schools located in poorer areas, is required if effective pre-entry equity programmes are to be developed and evaluated. Alternatively, medical schools might rethink their reliance on very high academic attainment in the admission process.