ContextCore physician activities of lifelong learning, continuing medical education credit, relicensure, specialty recertification, and clinical competence are linked to the abilities of physicians to assess their own learning needs and choose educational activities that meet these needs.ObjectiveTo determine how accurately physicians self-assess compared with external observations of their competence.Data SourcesThe electronic databases MEDLINE (1966-July 2006), EMBASE (1980-July 2006), CINAHL (1982-July 2006), PsycINFO (1967-July 2006), the Research and Development Resource Base in CME (1978-July 2006), and proprietary search engines were searched using terms related to self-directed learning, self-assessment, and self-reflection.Study SelectionStudies were included if they compared physicians' self-rated assessments with external observations, used quantifiable and replicable measures, included a study population of at least 50% practicing physicians, residents, or similar health professionals, and were conducted in the United Kingdom, Canada, United States, Australia, or New Zealand. Studies were excluded if they were comparisons of self-reports, studies of medical students, assessed physician beliefs about patient status, described the development of self-assessment measures, or were self-assessment programs of specialty societies. Studies conducted in the context of an educational or quality improvement intervention were included only if comparative data were obtained before the intervention.Data ExtractionStudy population, content area and self-assessment domain of the study, methods used to measure the self-assessment of study participants and those used to measure their competence or performance, existence and use of statistical tests, study outcomes, and explanatory comparative data were extracted.Data SynthesisThe search yielded 725 articles, of which 17 met all inclusion criteria. The studies included a wide range of domains, comparisons, measures, and methodological rigor. Of the 20 comparisons between self- and external assessment, 13 demonstrated little, no, or an inverse relationship and 7 demonstrated positive associations. A number of studies found the worst accuracy in self-assessment among physicians who were the least skilled and those who were the most confident. These results are consistent with those found in other professions.ConclusionsWhile suboptimal in quality, the preponderance of evidence suggests that physicians have a limited ability to accurately self-assess. The processes currently used to undertake professional development and evaluate competence may need to focus more on external assessment.

https://jamanetwork.com/journals/jama/articlepdf/203258/JRV60012.pdf

Accuracy of Physician Self-assessment Compared With Observed Measures of Competence: A Systematic Review

Research from numerous corners of psychological inquiry suggests that self-assessments of skill and character are often flawed in substantive and systematic ways. We review empirical findings on the imperfect nature of self-assessment and discuss implications for three real-world domains: health, education, and the workplace. In general, people's self-views hold only a tenuous to modest relationship with their actual behavior and performance. The correlation between self-ratings of skill and actual performance in many domains is moderate to meager-indeed, at times, other people's predictions of a person's outcomes prove more accurate than that person's self-predictions. In addition, people overrate themselves. On average, people say that they are "above average" in skill (a conclusion that defies statistical possibility), overestimate the likelihood that they will engage in desirable behaviors and achieve favorable outcomes, furnish overly optimistic estimates of when they will complete future projects, and reach judgments with too much confidence. Several psychological processes conspire to produce flawed self-assessments. Research focusing on health echoes these findings. People are unrealistically optimistic about their own health risks compared with those of other people. They also overestimate how distinctive their opinions and preferences (e.g., discomfort with alcohol) are among their peers-a misperception that can have a deleterious impact on their health. Unable to anticipate how they would respond to emotion-laden situations, they mispredict the preferences of patients when asked to step in and make treatment decisions for them. Guided by mistaken but seemingly plausible theories of health and disease, people misdiagnose themselves-a phenomenon that can have severe consequences for their health and longevity. Similarly, research in education finds that students' assessments of their performance tend to agree only moderately with those of their teachers and mentors. Students seem largely unable to assess how well or poorly they have comprehended material they have just read. They also tend to be overconfident in newly learned skills, at times because the common educational practice of massed training appears to promote rapid acquisition of skill-as well as self-confidence-but not necessarily the retention of skill. Several interventions, however, can be introduced to prompt students to evaluate their skill and learning more accurately. In the workplace, flawed self-assessments arise all the way up the corporate ladder. Employees tend to overestimate their skill, making it difficult to give meaningful feedback. CEOs also display overconfidence in their judgments, particularly when stepping into new markets or novel projects-for example, proposing acquisitions that hurt, rather then help, the price of their company's stock. We discuss several interventions aimed at circumventing the consequences of such flawed assessments; these include training people to routinely make cognitive repairs correcting for biased self-assessments and requiring people to justify their decisions in front of their peers. The act of self-assessment is an intrinsically difficult task, and we enumerate several obstacles that prevent people from reaching truthful self-impressions. We also propose that researchers and practitioners should recognize self-assessment as a coherent and unified area of study spanning many subdisciplines of psychology and beyond. Finally, we suggest that policymakers and other people who makes real-world assessments should be wary of self-assessments of skill, expertise, and knowledge, and should consider ways of repairing self-assessments that may be flawed.

https://journals.sagepub.com/doi/pdf/10.1111/j.1529-1006.2004.00018.x

Flawed Self-Assessment Implications for Health, Education, and the Workplace

Many researchers and educators have identified self-assessment as a vital aspect of professional self-regulation.1,2,3 This rationale has been the expressed motivation for a large number of studies of self-assessment ability in medical education, health professional education, and professions educat

Self-assessment in the health professions: a reformulation and research agenda.

BACKGROUND Effective feedback may be defined as feedback in which information about previous performance is used to promote positive and desirable development. This can be challenging as educators must acknowledge the psychosocial needs of the recipient while ensuring that feedback is both honest and accurate. Current feedback models remain reductionist in their approach. They are embedded in the hierarchical, diagnostic endeavours of the health professions. Even when it acknowledges the importance of two-way interactions, feedback often remains an educator-driven, one-way process. LESSONS FROM THE LITERATURE An understanding of the various types of feedback and an ability to actively seek an appropriate approach may support feedback effectiveness. Facilitative rather than directive feedback enhances learning for high achievers. High-achieving recipients undertaking complex tasks may benefit from delayed feedback. It is hypothesised that such learners are supported by reducing interruptions during the task. If we accept that medical students and doctors are high achievers, we can draw on some guiding principles from a complex and rarely conclusive literature. Feedback should focus on the task rather than the individual and should be specific. It should be directly linked to personal goals. Self-assessment as a means to identify personal learning requirements has no theoretical basis. Motivated recipients benefit from challenging facilitated feedback from external sources. A NEW MODEL To achieve truly effective feedback, the health professions must nurture recipient reflection-in-action. This builds on self-monitoring informed by external feedback. An integrated approach must be developed to support a feedback culture. Early training and experience such as peer feedback may over time support the required cultural change. Opportunities to provide feedback must not be missed, including those to impart potentially powerful feedback from high-stakes assessments. Feedback must be conceptualised as a supported sequential process rather than a series of unrelated events. Only this sustained approach will maximise any effect.

State of the science in health professional education: effective feedback.

Training in surgery has historically been considered an apprenticeship. Traditionally, the early training experiences of junior surgical residents are in real operative settings, on real patients. Several factors suggest that the operating theater may no longer provide the ideal atmosphere to foster the skills of a novice surgeon. First, as pressures intensify to use operating room time and resources efficiently, less time is available for teaching and practice of fundamental technical skills.1,2 Second, ethical concerns about teaching and learning basic surgical techniques on live human patients have been voiced.3 Third, the movement toward increased specialization in academic teaching hospitals has resulted in more highly complex and challenging surgical problems, which demand greater surgical expertise. Finally, technical evolution in surgery has created new skill sets and techniques that must be mastered by both practicing surgeons and trainees prior to clinical application.

Reacting to this situation, many academic health centers have turned toward laboratory-based training programs to foster the development of technical skills in both junior and senior surgeons.4–6 While the need for laboratory-based technical skills training courses has been identified,4 there remains debate regarding the educational effectiveness of such initiatives as they currently exist. Few studies have evaluated the impact of a focused laboratory-based training program using valid and objective assessments of surgical performance. Moreover, several authors have argued that short technical skills training courses are insufficient to provide adequate knowledge and skill and should not be used as a substitute for hands-on experience.7,8

As surrogates for human anatomy, surgical skills training laboratories rely on a variety of bench model simulations. These models vary widely with respect to their level of fidelity or “realism” to living human patients. Live animal and fresh human cadaver models, considered to be of high-fidelity, are limited by availability, high costs, potential for transmission of infectious disease, and ethical concerns.6,9 Lower-fidelity synthetic bench models sacrifice realism for portability, lower costs, and the potential for repetitive use.6,9 Despite the intuitive belief that “the more realistic, the better,” few studies have directly compared the relative attributes and educational effectiveness of live animal models versus synthetic training models on the acquisition and maintenance of surgical skill. Using microsurgery as the research platform, the current initiative was designed to evaluate the significance of bench model fidelity on the acquisition of technical skill among surgical trainees using clinically relevant outcome measures.

The educational impact of bench model fidelity on the acquisition of technical skill: the use of clinically relevant outcome measures.

The competent physician pursues lifelong learning through the recognition of deficiencies and the formulation of appropriate learning goals Despite the accepted theoretical value of self-assessment, studies have consistently shown that the accuracy of self-assessment is poor This paper examines the methodological issues that plague the measurement of self-assessment ability and presents several strategies that address these methodological problems within the current paradigm In addition, the article proposes an alternative conceptualization of self-assessment and describes its associated methods The conclusions of prior research in this domain must be re-examined in light of the common pitfalls encountered in the design of the studies and the analyses of the data Future efforts to elucidate self-assessment phenomena need to consider the implications of this review

/pdf/measuring-self-assessment-current-state-of-the-art-3cnrr0kj89.pdf

Measuring self-assessment: current state of the art.

Teaching the surgical craft: From selection to certification

Background In medicine, the development of expertise requires the recognition of one's capabilities and limitations. This study aimed to verify the accuracy of self-assessment for the performance of a surgical task, and to determine whether self-assessment may be improved through self-observation or exposure to relevant standards of performance. Methods Twenty-six senior surgical residents were videotaped performing a laparoscopic Nissen fundoplication in a pig. Experts rated the videos using two scoring systems. Subjects evaluated their performances after performance of the Nissen, after self-observation of their videotaped performance, and after review of four videotaped "benchmark" performances. Results Expert interrater reliability was 0.66 (intraclass correlation coefficient). The correlation between experts' and residents' self-evaluations was initially moderate ( r = 0.50, P r = 0.63 (Δ r = 0.13, P Conclusions Self-observation of videotaped performance improved the residents' ability to self-evaluate.

Mylène Ward

Papers

Measuring self-assessment: current state of the art.

Teaching the surgical craft: From selection to certification

Resident self-assessment of operative performance.