Showing papers in "Medical Education in 2006"

The qualitative research interview

Abstract: BACKGROUND Interviews are among the most familiar strategies for collecting qualitative data. The different qualitative interviewing strategies in common use emerged from diverse disciplinary perspectives resulting in a wide variation among interviewing approaches. Unlike the highly structured survey interviews and questionnaires used in epidemiology and most health services research, we examine less structured interview strategies in which the person interviewed is more a participant in meaning making than a conduit from which information is retrieved. PURPOSE In this article we briefly review the more common qualitative interview methods and then focus on the widely used individual face-to-face in-depth interview, which seeks to foster learning about individual experiences and perspectives on a given set of issues. We discuss methods for conducting in-depth interviews and consider relevant ethical issues with particular regard to the rights and protection of the participants.

A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8

Abstract: Background: Preparing healthcare professionals for teaching is regarded as essential to enhancing teaching effectiveness. Although many reports describe various faculty development interventions, there is a paucity of research demonstrating their effectiveness.Objective: To synthesize the existing evidence that addresses the question: “What are the effects of faculty development interventions on the knowledge, attitudes and skills of teachers in medical education, and on the institutions in which they work?”Methods: The search, covering the period 1980–2002, included three databases (Medline, ERIC and EMBASE) and used the keywords: staff development; in-service training; medical faculty; faculty training/development; continuing medical education. Manual searches were also conducted.Articles with a focus on faculty development to improve teaching effectiveness, targeting basic and clinical scientists, were reviewed. All study designs that included outcome data beyond participant satisfaction were accepted....

The history of simulation in medical education and possible future directions.

Abstract: INTRODUCTION Clinical simulation is on the point of having a significant impact on health care education across professional boundaries and in both the undergraduate and postgraduate arenas. SCOPE OF SIMULATION The use of simulation spans a spectrum of sophistication, from the simple reproduction of isolated body parts through to complex human interactions portrayed by simulated patients or high-fidelity human patient simulators replicating whole body appearance and variable physiological parameters. GROWTH OF SIMULATION After a prolonged gestation, recent advances have made available affordable technologies that permit the reproduction of clinical events with sufficient fidelity to permit the engagement of learners in a realistic and meaningful way. At the same time, reforms in undergraduate and postgraduate education, combined with political and societal pressures, have promoted a safety-conscious culture where simulation provides a means of risk-free learning in complex, critical or rare situations. Furthermore, the importance of team-based and interprofessional approaches to learning and health care can be promoted. CONCLUSION However, at the present time the quantity and quality of research in this area of medical education is limited. Such research is needed to enable educators to justify the cost and effort involved in simulation and to confirm the benefit of this mode of learning in terms of the outcomes achieved through this process.

Making sense of qualitative data analysis: an introduction with illustrations from DIPEx (personal experiences of health and illness).

Abstract: Objectives This paper outlines an approach to analysing qualitative textual data from interviews and discusses how to ensure analytic procedures are appropriately rigorous. Overview Qualitative data analysis should begin at an early stage in data collection and be highly systematic. It is important to identify issues that emerge during the data collection and analysis as well as those that the researcher may have anticipated (from reading or experience). Analysis is very time-consuming, but careful sampling, the collection of rich material and analytic depth mean that a relatively small number of cases can generate insights that apply well beyond the confines of the study. One particular approach to thematic analysis is introduced with examples from the DIPEx (personal experiences of health and illness) project, which collects video- and audio-taped interviews that are freely accessible through http://www.dipex.org. Evaluation Qualitative analysis of patients' perspectives of illness can illuminate numerous issues that are important for medical education, some of which are unlikely to arise in the clinical encounter. Qualitative studies can also cover a much broader range of experiences – of both common and rare disease – than clinicians will see in practice. The DIPEx website is based on qualitative analysis of collections of interviews, illustrated with hundreds of video and audio clips, and is an innovative resource for medical education.

Development of a rating system for surgeons' non-technical skills.

Abstract: BACKGROUND Analyses of adverse events in surgery reveal that many underlying causes are behavioural, such as communication failure, rather than technical. Non-technical (i.e. cognitive and interpersonal) skills are not addressed explicitly in surgical training. However, surgeons need to demonstrate these skills, which underpin their technical excellence, to maximise patient safety in the operating theatre. This paper describes the method used to identify surgeons' non-technical skills, and the development of a skills taxonomy and behavioural rating system to structure observation and feedback in surgical training. METHODS Cognitive task analyses (critical incident interviews) were conducted with 27 consultant surgeons in general, cardiac and orthopaedic surgery. The interviews were coded and a multidisciplinary group of surgeons and psychologists used an iterative process to develop a skills taxonomy. This was supported by data gathered from an attitude survey, literature review, analysis of surgical mortality reports and observations in theatre. RESULTS Five categories of non-technical skills were identified, including situation awareness, decision making, task management, leadership and communication and teamwork. This provided a structure for a prototype skill taxonomy (v1.1), which comprised 14 non-technical skill elements. Observable behaviours (markers) indicative of good and poor performance were developed for each element by 16 consultant surgeons to form a prototype behaviour rating system. CONCLUSIONS The prototype skills taxonomy and behaviour rating system are grounded empirically in surgery. The reliability of the system is currently being tested using standardised scenarios. If this evaluation proves successful, the system could be used to structure feedback and guide non-technical skills training.

Making Sense of Grounded Theory in Medical Education

Abstract: BACKGROUND Grounded theory is a research methodology designed to develop, through collection and analysis of data that is primarily (but not exclusively) qualitative, a well-integrated set of concepts that provide a theoretical explanation of a social phenomenon. OBJECTIVE This paper aims to provide an introduction to key features of grounded theory methodology within the context of medical education research. OVERVIEW In this paper we include a discussion of the origins of grounded theory, a description of key methodological processes, a comment on pitfalls encountered commonly in the application of grounded theory research, and a summary of the strengths of grounded theory methodology with illustrations from the medical education domain. DISCUSSION The significant strengths of grounded theory that have resulted in its enduring prominence in qualitative research include its clearly articulated analytical process and its emphasis on the generation of pragmatic theory that is grounded in the data of experience. When applied properly and thoughtfully, grounded theory can address research questions of significant relevance to the domain of medical education.

A comparison of the effects of problem-based learning and lecturing on the development of students' critical thinking.

Abstract: Background Educational approaches are thought to have facilitative or hindering effects on students' critical thinking development. The aim of this study was to compare the effects of problem-based learning (PBL) and lecturing approaches on the development of students' critical thinking. Methods All 79 Year 1 undergraduate nursing students at a university in Hong Kong were randomly assigned to 1 of 2 parallel courses delivered by either PBL (n = 40) or lecturing (n = 39) over 1 academic year. The primary outcome measure was students' critical thinking disposition as measured by the California Critical Thinking Disposition Inventory (CCTDI). Individual interviews were also conducted to elicit the students' perceptions of their learning experience. Data were collected at 4 timepoints spanning 3 years. Results The overall CCTDI and subscale scores for the PBL group were not significantly different from those of the lecture group at the first timepoint (pretest). Compared with lecture students, PBL students showed significantly greater improvement in overall CCTDI (P = 0.0048), Truthseeking (P = 0.0008), Analyticity (P = 0.0368) and Critical Thinking Self-confidence (P = 0.0342) subscale scores from the first to the second timepoints; in overall CCTDI (P = 0.0083), Truthseeking (P = 0.0090) and Analyticity (P = 0.0354) subscale scores from the first to the third timepoints; and in Truthseeking (P = 0.0173) and Systematicity (P = 0.0440) subscale scores from the first to the fourth timepoints. Conclusions There were significant differences in the development of students' critical thinking dispositions between those who undertook the PBL and lecture courses, respectively.

Anatomy teaching: ghosts of the past, present and future

Abstract: INTRODUCTION Anatomy teaching has perhaps the longest history of any component of formalised medical education. In this article we briefly consider the history of dissection, but also review the neglected topic of the history of the use of living anatomy. CURRENT DEBATES The current debates about the advantages and disadvantages of cadavers, prosection versus dissection, and the use of living anatomy and radiology instead of cadavers are discussed. THE FUTURE Future prospects are considered, along with some of the factors that might inhibit change.

Can virtual reality improve anatomy education? A randomised controlled study of a computer-generated three-dimensional anatomical ear model.

Abstract: INTRODUCTION The use of computer-generated 3-dimensional (3-D) anatomical models to teach anatomy has proliferated. However, there is little evidence that these models are educationally effective. The purpose of this study was to test the educational effectiveness of a computer-generated 3-D model of the middle and inner ear. METHODS We reconstructed a fully interactive model of the middle and inner ear from a magnetic resonance imaging scan of a human cadaver ear. To test the model's educational usefulness, we conducted a randomised controlled study in which 28 medical students completed a Web-based tutorial on ear anatomy that included the interactive model, while a control group of 29 students took the tutorial without exposure to the model. At the end of the tutorials, both groups were asked a series of 15 quiz questions to evaluate their knowledge of 3-D relationships within the ear. RESULTS The intervention group's mean score on the quiz was 83%, while that of the control group was 65%. This difference in means was highly significant (P < 0.001). DISCUSSION Our findings stand in contrast to the handful of previous randomised controlled trials that evaluated the effects of computer-generated 3-D anatomical models on learning. The equivocal and negative results of these previous studies may be due to the limitations of these studies (such as small sample size) as well as the limitations of the models that were studied (such as a lack of full interactivity). Given our positive results, we believe that further research is warranted concerning the educational effectiveness of computer-generated anatomical models.

Broadening conceptions of learning in medical education: the message from teamworking

Abstract: Background There is a mismatch between the broad range of learning theories offered in the wider education literature and a relatively narrow range of theories privileged in the medical education literature. The latter are usually described under the heading of ‘adult learning theory’. Methods This paper critically addresses the limitations of the current dominant learning theories informing medical education. An argument is made that such theories, which address how an individual learns, fail to explain how learning occurs in dynamic, complex and unstable systems such as fluid clinical teams. Results Models of learning that take into account distributed knowing, learning through time as well as space, and the complexity of a learning environment including relationships between persons and artefacts, are more powerful in explaining and predicting how learning occurs in clinical teams. Learning theories may be privileged for ideological reasons, such as medicine's concern with autonomy. Conclusions Where an increasing amount of medical education occurs in workplace contexts, sociocultural learning theories offer a best-fit exploration and explanation of such learning. We need to continue to develop testable models of learning that inform safe work practice. One type of learning theory will not inform all practice contexts and we need to think about a range of fit-for-purpose theories that are testable in practice. Exciting current developments include dynamicist models of learning drawing on complexity theory.

Effect of practice on standardised learning outcomes in simulation-based medical education

Abstract: OBJECTIVES This report synthesises a subset of 31 journal articles on high-fidelity simulation-based medical education containing 32 research studies drawn from a larger qualitative review published previously. These studies were selected because they present adequate data to allow for quantitative synthesis. We hypothesised an association between hours of practice in simulation-based medical education and standardised learning outcomes measured as weighted effect sizes. METHODS Journal articles were screened using 5 exclusion and inclusion criteria. Response data were extracted and 3 judges independently coded each study. Learning outcomes were standardised using a common metric, the average weighted effect size (AWES), due to the heterogeneity of response measures in individual studies. ANOVA was used to evaluate AWES differences due to hours of practice on a highfidelity medical simulator cast in 5 categories. The eta squared (g 2 ) statistic was used to assess the association between AWES and simulator practice hours. RESULTS There is a strong association (g 2 ! 0.46) between hours of practice on high-fidelity medical simulators and standardised learning outcomes. The association approximates a dose‐response relationship. CONCLUSIONS Hours of high-fidelity simulator practice have a positive, functional relationship with standardised learning outcomes in medical education. More rigorous research methods and more stringent journal editorial policies are needed to advance this field of medical education research.

Longterm effects of problem-based learning: a comparison of competencies acquired by graduates of a problem-based and a conventional medical school.

Abstract: BACKGROUND: Problem-based learning (PBL) as an approach to the instruction of medical students has attracted much attention in recent years. However, its effect on the performance of its graduates is the subject of considerable debate. This article presents data from a large-scale study among graduates of a problem-based medical school and those of a conventional medical school to contribute to this discussion. PURPOSE: To study the longterm effects of problem-based medical training on the professional competencies of graduates. METHODS: A questionnaire was sent to all graduates since 1980 of a problem-based and a conventional medical school. Participants were requested to rate themselves on 18 professional competencies derived from the literature. RESULTS: The graduates of the PBL school scored higher on 14 of 18 professional competencies. Graduates of the problem-based school rated themselves as having much better interpersonal skills, better competencies in problem solving, self-directed learning and information gathering, and somewhat better task-supporting skills, such as the ability to work and plan efficiently. There were no sizeable differences with regard to general academic competencies, such as conducting research or writing a paper. Graduates from the conventional school rated themselves as having slightly more medical knowledge. The findings were shown to be valid and robust against possible response bias. CONCLUSION: The findings suggest that PBL not only affects the typical PBL-related competencies in the interpersonal and cognitive domains, but also the more general work-related skills that are deemed important for success in professional practice.

A conceptual framework for developing teaching cases: a review and synthesis of the literature across disciplines

Abstract: Context Case-based teaching is regarded as a superior instructional method compared with lectures in promoting a learner's critical thinking skills. While much is known about the role a discussion facilitator plays in case-based teaching, the debate on the influence of the format and structure of cases on learning is controversial. Objectives We sought to identify strategies for constructing cases based on studies from multiple disciplines, which report the development and use of cases in teaching and learning. The purpose was to offer the medical and other educational communities a conceptual framework that can be examined in future research. Results Based on a review of 100 studies, we synthesised 17 strategies around 5 core attributes of cases: relevant (level of learner, goals and objectives, setting of case narrative); realistic (authenticity, distractors, gradual disclosure of content); engaging (rich content, multiple perspectives, branching of content); challenging (difficulty, unusual cases, case structure, multiple cases), and instructional (build upon prior knowledge, assessment, feedback, and teaching aids). Discussion Despite the wide use of cases in disparate disciplines, there has been no overarching study that synthesises strategies of case development or tests these strategies in research settings. The framework we developed can serve as a menu of case development options that educators and researchers can pilot and evaluate in their local settings.

Professionalism in medical education, an American perspective: from evidence to accountability

Abstract: Context Professionalism is central to sustaining the public's trust in the medical profession; it is the essence of the doctor−patient relationship. Evidence exists that public trust is waning and that doctors are facing powerful contemporary threats to their professional values. The role of medical education is paramount in preparing future doctors to recognise and overcome these threats; to do so will require substantial change in the culture and environment of medical education. Objectives The aims of this paper are to provide a definition and framework for professionalism in the context of medical education, describe current threats to medical professionalism, and detail the role medical schools and academic medical centres can play in preparing tomorrow's doctors to recognise and resist these threats. Additionally, the paper reviews established and potential methods for measuring professionalism and thus assuring public accountability. Finally, specific recommendations are offered for medical schools and teaching hospitals to nurture and sustain professionalism. Discussion The progressive intrusion of commercialism into the realm of medicine is threatening to replace the ethics of professionalism with the irreconcilable ethics of the marketplace. Academic medicine must assume greater responsibility and accountability for strengthening the resolve of future doctors to sustain their commitment to the ethics of professionalism. It can do so by improving the medical school admission process, enhancing both formal and experiential teaching of professionalism, and purging the educational environment of unprofessional practices. Ten approaches that academic medicine might adopt to achieve these goals are provided.

Validating the readiness for interprofessional learning scale (RIPLS) in the postgraduate context : are health care professionals ready for IPL?

Abstract: AIMS This paper describes the process of validating the Readiness for Interprofessional Learning Scale (RIPLS) for use with postgraduate health care professionals. CONTEXT The RIPLS questionnaire has proved useful in the undergraduate context, enabling tutors to assess the readiness of students to engage in interprofessional learning (IPL). With the drive in the National Health Service (NHS) to deliver health care in interprofessional teams, it seems logical to ask whether postgraduate education should, or could, be delivered successfully in interprofessional contexts. As a preliminary to undertaking an extended IPL project, the researchers tested the validity of the RIPLS tool in the postgraduate health care context. METHOD A modified version of the RIPLS questionnaire was administered to all general practitioners, nurses, pharmacists and allied health professionals in the Dundee Local Health Care Cooperative (LHCC) (n = 799). A total of 546 staff responded (68%). RESULTS Three factors, comprising 23 statements, emerged from the statistical analysis of the survey data, namely, teamwork and collaboration, sense of professional identity and patient-centredness. The internal consistency measure was 0.76. Analysis of variance suggested some key differences between the different professions in respect of the factors. CONCLUSIONS The RIPLS questionnaire was validated for use in the postgraduate context, thus providing researchers with a tool for assessing health professionals' attitudes towards interprofessional learning at practice level, community health partnership level or at a national level of education and training. Significant differences between professional groups should be taken into account in designing any interprofessional learning programme.

Communication skills competencies: definitions and a teaching toolbox

Abstract: BACKGROUND Doctors' interpersonal and communication skills correlate with improved health care outcomes. International medical organisations require competency in communication skills. The Accreditation Council for Graduate Medical Education (ACGME) developed a toolbox for assessing this competency and 5 others, yet none initially for teaching these skills. PURPOSE AND METHODS The original focus in the development of the ACGME competencies was evaluation. This paper represents a significant step toward defining methods for teaching communication skills competencies. A total of 16 medical education leaders from medical schools worldwide, participating in the 2003 Harvard Macy Institute Program for Physician Educators, worked together to: (1) further define the ACGME competency in interpersonal and communication skills; (2) delineate teaching strategies for each level of medical education; and (3) create a teaching toolbox to integrate communication skills competencies into medical curricula. Four subgroups defined subcompetencies, identified teaching strategies for undergraduate, graduate and postgraduate medical training and brought their work to the larger group. The expanded communication competencies and teaching strategies were determined by a consensus of the larger group, presented to 80 Harvard Macy Scholars and Faculty for further discussion, then finalised by consensus. CONCLUSION The teaching toolbox expands the ACGME core communication competencies, adds 20 subcompetencies and connects these competencies to teaching strategies at each level of medical training. It represents the collaboration and consensus of a diverse international group of medical education leaders in a variety of medical specialities and institutions, all involved in teaching communication skills. The toolbox is applicable globally across different settings and specialities, and is sensitive to different definitions of health care.

Training the clinical eye and mind: using the arts to develop medical students' observational and pattern recognition skills.

Abstract: INTRODUCTION Observation, including identification of key pieces of data, pattern recognition, and interpretation of significance and meaning, is a key element in medical decision making. Clinical observation is taught primarily through preceptor modelling during the all-important clinical years. No single method exists for communicating these skills, and medical educators have periodically experimented with using arts-based training to hone observational acuity. The purpose of this qualitative study was to better understand the similarities and differences between arts-based and clinical teaching approaches to convey observation and pattern recognition skills. METHOD A total of 38 Year 3 students participated in either small group training with clinical photographs and paper cases (group 1), or small group training using art plus dance (group 2), both consisting of 3 2-hour sessions over a 6-month period. FINDINGS Students in both conditions found value in the training they received and, by both self- and instructor-report, appeared to hone observation skills and improve pattern recognition. The clinically based condition appeared to have been particularly successful in conveying pattern recognition concepts to students, probably because patterns presented in this condition had specific correspondence with actual clinical situations, whereas patterns in art could not be generalised so easily to patients. In the arts-based conditions, students also developed skills in emotional recognition, cultivation of empathy, identification of story and narrative, and awareness of multiple perspectives. CONCLUSION The interventions studied were naturally complementary and, taken together, can bring greater texture to the process of teaching clinical medicine by helping us see a more complete 'picture' of the patient.

An investigation of medical student reactions to feedback: a randomised controlled trial.

Abstract: Background Medical educators have indicated that feedback is one of the main catalysts required for performance improvement. However, medical students appear to be persistently dissatisfied with the feedback that they receive. The purpose of this study was to evaluate learning outcomes and perceptions in students who received feedback compared to those who received general compliments. Methods All subjects received identical instruction on two-handed surgical knot-tying. Group 1 received specific, constructive feedback on how to improve their knot-tying skill. Group 2 received only general compliments. Performance was videotaped before and after instruction and after feedback. Subjects completed the study by indicating their global level of satisfaction. Three faculty evaluators observed and scored blinded videotapes of each performance. Intra-observer agreement among expert ratings of performance was calculated using 2-way random effects intraclass correlation (ICC) methods. Satisfaction scores and performance scores were compared using paired samples t-tests and independent samples t-tests. Results Performance data from 33 subjects were analysed. Inter-rater reliability exceeded 0.8 for ratings of pre-test, pre-intervention and post-intervention performances. The average performance of students who received specific feedback improved (21.98 versus 15.87, P < 0.001), whereas there was no significant change in the performance score in the group who received only compliments (17.00 versus 15.39, P = 0.181) The average satisfaction rating in the group that received compliments was significantly higher than the group that received feedback (6.00 versus 5.00, P = 0.005). Discussion Student satisfaction is not an accurate measure of the quality of feedback. It appears that satisfaction ratings respond to praise more than feedback, while learning is more a function of feedback.

Stress, debt and undergraduate medical student performance.

Abstract: Introduction Against the background of current debate over university funding and widening access, we aimed to examine the relationships between student debt, mental health and academic performance. Methods We carried out an electronic survey of all medical undergraduate students at the University of Aberdeen during May–June 2004. The questionnaire contained items about demographics, debt, income and stress. Students were also asked for consent to access their examination results, which were correlated with their answers. Statistical analyses of the relationships between debt, performance and stress were performed. Results The median total outstanding debt was £7300 (interquartile range 2000–14 762.50). Students from lower socioeconomic backgrounds and postgraduate students had higher debts. There was no direct correlation between debt, class ranking or General Health Questionnaire (GHQ) score; however, a subgroup of 125 students (37.7%), who said that worrying about money affected their studies, did have higher debt and were ranked lower in their classes. Some of these students were also cases on the GHQ-12. Overall, however, cases on the GHQ had lower levels of debt and lower class ranking, suggesting that financial worries are only 1 cause of mental health difficulties. Discussion Students' perceptions of their own levels of debt rather than level of debt per se relates to performance. Students who worry about money have higher debts and perform less well than their peers in degree examinations. Some students in this subgroup were also identified by the GHQ and may have mental health problems. The relationships between debt, mental health and performance in undergraduate medical students are complex but need to be appreciated by medical education policy makers.

Not knowing that they do not know : self-assessment accuracy of third-year medical students

Abstract: PURPOSE The development of self-regulated learning is a major focus of our problem-based learning (PBL) medical programme. Students who are unsuccessful in assessments often seem to lack insight into the standard of their own performance, yet the ability to self-assess accurately is essential for the effective self-management of learning. The aim of this project was to evaluate the accuracy of self- and peer-assessment according to academic performance. METHOD In 2004, 175 3rd-year students undertook an integrated, case-based, short-essay, formative assessment. After the assessment they were provided with model answers and marking criteria. Students marked their own assessment paper and the paper of one of their peers. Assessment papers were subsequently marked by faculty members. The following data was available for each student: self-mark, faculty-mark, score awarded by a peer and the score that they awarded to their peer. Self-assessment and peer-assessment ability was compared to overall academic performance. RESULTS Low-achieving students score themselves and their peers generously. High-achieving students score themselves more harshly than faculty. However, they score their peers accurately. CONCLUSION In the 3rd year of the programme low-achieving students are unable to assess accurately the quality of their own work or the work of their peers in a formative written assessment. The PBL curriculum does not guarantee the appropriate development of self-assessment skills.

Caring for a common future: medical schools' social accountability

Abstract: ORIGINS AND CONTEXT: The concept of 'the social accountability of medical schools' is moving from the peripheral preoccupation of a few to a more central concern of medical schools themselves. Born of concerns about the professionalism and relevance of both the institutions and their graduates, it is seen increasingly as an urgent call to focus the considerable social resources vested in academic health science institutions on addressing the priority health concerns of the societies they serve. For a profession embedded in an ethos of service, this would seem an obvious transition. However, as with any movement towards transformative change, it runs the risk of being more mantra and rhetoric than mandate and responsibility. NEEDED RESPONSE: Proceeding from the assumption that good intentions alone are not enough, this paper seeks to outline the historical development and some current expression of the concept throughout the world. The sadly divergent wealth and health status of modern societies calls for very different actions by medical schools across the spectrum from the least endowed to the wealthiest of schools. In a profession claiming centuries of cohesive commitment to the welfare of others, it is increasingly urgent that the current generation of medical educators converge on a relevant set of principles and coherent activities. TOOLS FOR THE TASK: While recognising that they are closely intertwined, the paper outlines the difference between the social accountability of the institutions themselves and the social accountability of the graduates they produce. It outlines both individual examples and the international initiatives that are fostering and facilitating institutional collaborations to bring both progress and optimism to this daunting task. It provides connections to practical resources for those who are committed to that task. Other papers in this series add further practical insights into the central role that medical educators must play if we are to fulfil the responsibilities we carry with the privilege of our profession.

Empirical evidence for symbiotic medical education: a comparative analysis of community and tertiary-based programmes.

Abstract: BACKGROUND Flinders University has developed the Parallel Rural Community Curriculum (PRCC), a full year clinical curriculum based in rural general practice in South Australia. The examination performance of students on this course has been shown to be higher than that of their tertiary hospital-based peers. AIM To compare the learning experiences of students in the community-based programme with those of students in the tertiary hospital in order to explain these improved academic outcomes. METHOD A case study was undertaken, using an interpretivist perspective, with 3 structured interviews carried out over 2 academic years with each of 6 students from the community-based programme and 16 students from the tertiary hospital. The taped interviews were transcribed and analysed thematically using NUD*IST software. RESULTS The community-based programme was successful in immersing the students in the clinical environment in a meaningful way. Four key themes were found in the data. These represented clear differences between the experiences of the community-based and hospital-based students. These differences involved: the value that students perceived they were given by supervising doctors and their patients; the extent to which the student's presence realised a synergy between the work of the university and the health service; opportunities for students to meet the aspirations of both the community and government policy, and opportunities for students to learn how professional expectations can mesh with their own personal values. CONCLUSION This study has provided empirical evidence for the importance of the concept of symbiosis in understanding quality in medical education.

A plea for new psychometric models in educational assessment.

Abstract: OBJECTIVE To describe the weaknesses of the current psychometric approach to assessment as a scientific model. DISCUSSION The current psychometric model has played a major role in improving the quality of assessment of medical competence. It is becoming increasingly difficult, however, to apply this model to modern assessment methods. The central assumption in the current model is that medical competence can be subdivided into separate measurable stable and generic traits. This assumption has several far-reaching implications. Perhaps the most important is that it requires a numerical and reductionist approach, and that aspects such as fairness, defensibility and credibility are by necessity mainly translated into reliability and construct validity. These approaches are more and more difficult to align with modern assessment approaches such as mini-CEX, 360-degree feedback and portfolios. This paper describes some of the weaknesses of the psychometric model and aims to open a discussion on a conceptually different statistical approach to quality of assessment. FUTURE DIRECTIONS We hope that the discussion opened by this paper will lead to the development of a conceptually different statistical approach to quality of assessment. A probabilistic or Bayesian approach would be worth exploring.

Influence of clerkship experiences on clinical competence.

Abstract: BACKGROUND Clerkship experiences are considered crucial for the development of clinical competence. Yet whether there is a direct relationship between the nature and volume of patient encounters and learning outcomes is far from clear. Some evidence in the literature points towards the importance of clinical supervision on student learning, but the relationship between clinical supervision, patient encounters and student competence remains unclear. OBJECTIVES This study aimed firstly to determine the variation in students' clinical experiences within and across sites; secondly, to identify the causes of this variation, and thirdly, to investigate the consequences of this variation on students' competence. METHODS Clerkship students at 12 hospital sites recorded their patient encounters in logbooks. Site characteristics that might influence the variation in patient encounters were collected. Student competence was determined by 3 independent indicators: a practical end-of-clerkship examination; a theoretical end-of-clerkship examination, and an evaluation of professional performance. A model was developed to test the available clerkship data using structural equation modelling (SEM) software. RESULTS Analysis of the logbooks revealed a large variation in the number of patients encountered by students. The average length of patient stay, number of patients admitted, and quality of supervision accounted partly for this variation. An increased number of patient encounters did not directly lead to improved competence. Quality of supervision turned out to be crucially important because it directly impacted student learning and also positively influenced the number of patient encounters. CONCLUSION Monitoring the effectiveness of clerkship by merely asking students to keep a tally of the problems and diseases they encounter, without paying attention to the quality of supervision, does not contribute towards improving student learning.

Assessing procedural skills in context: exploring the feasibility of an Integrated Procedural Performance Instrument (IPPI)

Abstract: BACKGROUND The assessment of clinical procedural skills has traditionally focused on technical elements alone. However, in real practice, clinicians are expected to be able to integrate technical with communication and other professional skills. We describe an integrated procedural performance instrument (IPPI), where clinicians are assessed on 12 clinical procedures in a simulated clinical setting which combines simulated patients (SPs) with inanimate models or items of medical equipment. Candidates are observed remotely by assessors whose data are fed back to the clinician within 24 hours of the assessment. This paper describes the feasibility of IPPI. RESULTS A full-scale IPPI and 2 pilot studies with trainee and qualified health care professionals has yielded an extensive data set including 585 scenario evaluations from candidates, 60 from clinical assessors and 31 from simulated patients (SPs). Interview and questionnaire data showed that for the majority of candidates IPPI provided a powerful and valuable learning experience. Realism was rated highly. Remote and real-time assessment worked effectively, although for some procedures limited camera resolution affected observation of fine details. DISCUSSION IPPI offers an innovative approach to assessing clinical procedural skills. Although resource-intensive, it has the potential to provide insight into individual's performance over a spectrum of clinical scenarios and at no risk to the safety of patients. Additional benefits of IPPI include assessment in real time from experts (allowing remote rating by external examiners) as well as provision of feedback from simulated patients.

Students' views of reflective learning techniques: an efficacy study at a UK medical school

Abstract: Keywords: humans; students; medical/*psychology; *attitude of health personnel; teaching/*methods; *learning; education; medical; undergraduate/*methods; Wales Objective To describe the effects of a voluntary intervention using reflective learning techniques on students' learning. Design An interventional study with reflective learning techniques offered to medical students. Setting Year 3 of undergraduate medicine at Cardiff University where the curriculum is integrated with early clinical contact. Participants All 232 Year 3 students were invited to participate. A total of 65 attended an introductory lecture. After the lecture 35 students agreed to take part; 15 of these subsequently dropped out (some before attending tutorial groups, others after taking part for some weeks). Interventions Participants kept learning journals for 2 terms and attended fortnightly, facilitated tutorial groups where they discussed their reflective journal entries. Main outcome measures were qualitative interviews and examination results. Results Interviews were carried out with 19 full participants, 4 initial participants and 7 non-participants. Participants perceived that they gained a greater ability to identify learning objectives and to integrate learning. The tutorial groups encouraged students to compare progress with their peers. Some students did not take part because they thought that the large factual content of the curriculum would make reflective learning less useful. There were no differences between the groups in examination results. Conclusions Students among the small, self-selected group of participants were better able to identify what they needed to learn although there was no improvement in examination results. Students appear unlikely to take up voluntary reflective learning if they do not think it relates to the curriculum and assessments. Student culture exerts a potent effect on willingness to attend extra tutorial groups.

The relationship between assessment and learning.

Abstract: However, repetition of a statement does not mean that it is true or, at least, true in all contexts. It is also widely accepted that there are significant drivers of learning that are not related to assessment, and indeed, may be at odds with the intentions of assessment. These are the influence of the informal and ⁄or hidden curricula, frequently related to exposure to role models. It is possible to imagine a progression with a rather different outcome from that sometimes ascribed to Confucius:

Analysis of verbal interactions in tutorial groups: a process study.

Abstract: INTRODUCTION Collaborative learning, including problem-based learning (PBL), is a powerful learning method. Group interaction plays a crucial role in stimulating student learning. However, few studies on learning processes in medical education have examined group interactions. Most studies on collaboration within PBL used self-reported data rather than observational data. We investigated the following types of interactions in PBL tutorial groups: learning-oriented interactions (exploratory questioning, cumulative reasoning and handling conflicts about knowledge); procedural interactions, and irrelevant/off-task interactions. AIM The central question concerned how much time is spent on the different types of interaction during group sessions and how the types of interaction are distributed over the meeting. METHOD Four tutorial group sessions in Year 2 of the PBL undergraduate curriculum of Maastricht Medical School were videotaped and analysed. The sessions concerned the reporting phase of the PBL process. We analysed the interactions using a coding scheme distinguishing several verbal interaction types, such as questions, arguments and evaluations. RESULTS Learning-orientated interactions accounted for 80% of the interactions, with cumulative reasoning, exploratory questioning and handling conflicts about knowledge accounting for about 63%, 10% and 7% of the interactions, respectively. Exploratory questioning often preceded cumulative reasoning. Both types occurred throughout the meeting. Handling conflicts mainly occurred after the first 20 minutes. CONCLUSIONS Task involvement in the tutorial groups was high. All types of learning-orientated interactions were observed. Relatively little time was spent on exploratory questions and handling conflicts about knowledge. Problem-based learning training should pay special attention to stimulating discussion about contradictory information.

How specific is case specificity

Abstract: Objectives Case specificity implies that success on any case is specific to that case. In examining the sources of error variance in performance on case-based examinations, how much error variance results from differences between cases compared with differences between items within cases? What is the optimal number of cases and questions within cases to maximise test reliability given some fixed period of examination time? Methods G and D generalisability studies were conducted to identify variance components and reliability for each examination analysed, and to optimise the reliability of the given test composition (1, 1.5, 2, 3, 4 and 5 questions per case), using data from 3 key features examinations of the Medical Council of Canada (n = 6342 graduating medical students), each of which consisted of about 35 written cases followed by 1–4 questions regarding specific key elements of data gathering, diagnosis and/or management. Results The smallest variance component was due to subjects; the variance due to subject–item interaction was over 5 times the interaction with cases (on average, 0.1106 compared with 0.0195). Relatively little variance was due to differences between cases; about 80% of the error variance was due to variability in performance among items within cases. The D study showed that reliability varied between 0.541 and 0.579, was least with 1 item per case and highest at 2 and 3 items per case. Conclusions The main source of error variance was items within cases, not cases, and the optimal strategy in terms of enhancing reliability would use cases with 2–3 items per case.

Are some of the challenging aspects of the CanMEDS roles valid outside Canada

Abstract: CONTEXT Many countries have adopted the CanMEDS roles. However, there is limited information on how these apply in an international context and in different specialties. OBJECTIVES To survey trainee and specialist ratings of the importance of the CanMEDS roles and perceived ability to perform tasks within the roles. METHODS We surveyed 8749 doctors within a defined region (eastern Denmark) via a single-issue, mailed questionnaire. Each of the 7 roles was represented by 3 questionnaire items to be rated for perceived importance and confidence in ability to perform the role. RESULTS Responses were received from 3476 doctors (42.8%), including 190 interns, 201 doctors in the introductory year of specialist training, 529 residents and 2152 specialists. The overall mean rating of importance (on a scale of 1-5) of the aspects of competence described in the CanMEDS roles was 4.2 (standard deviation 0.6) and did not differ between trainee groups and specialists. Mean ratings of confidence were lower than ratings of importance and increased across the groups from interns to specialists. Differences between specialty groups were evident in both importance and confidence for many of the roles. For laboratory, technical and, to a lesser extent, cognitive specialties, the role of Health Advocate scored the lowest in importance. For general medicine specialties, the roles of Medical Expert, Collaborator, Manager and Scholar all scored lower for importance and confidence. CONCLUSIONS This study provides a sketch of the content and construct validity of the CanMEDS roles in a non-Canadian setting. More research is needed in how these aspects of competence can be best taught and applied across specialties in different jurisdictions.