ContextCurrent assessment formats for physicians and trainees reliably test
core knowledge and basic skills. However, they may underemphasize some important
domains of professional medical practice, including interpersonal skills,
lifelong learning, professionalism, and integration of core knowledge into
clinical practice.ObjectivesTo propose a definition of professional competence, to review current
means for assessing it, and to suggest new approaches to assessment.Data SourcesWe searched the MEDLINE database from 1966 to 2001 and reference lists
of relevant articles for English-language studies of reliability or validity
of measures of competence of physicians, medical students, and residents.Study SelectionWe excluded articles of a purely descriptive nature, duplicate reports,
reviews, and opinions and position statements, which yielded 195 relevant
citations.Data ExtractionData were abstracted by 1 of us (R.M.E.). Quality criteria for inclusion
were broad, given the heterogeneity of interventions, complexity of outcome
measures, and paucity of randomized or longitudinal study designs.Data SynthesisWe generated an inclusive definition of competence: the habitual and
judicious use of communication, knowledge, technical skills, clinical reasoning,
emotions, values, and reflection in daily practice for the benefit of the
individual and the community being served. Aside from protecting the public
and limiting access to advanced training, assessments should foster habits
of learning and self-reflection and drive institutional change. Subjective,
multiple-choice, and standardized patient assessments, although reliable,
underemphasize important domains of professional competence: integration of
knowledge and skills, context of care, information management, teamwork, health
systems, and patient-physician relationships. Few assessments observe trainees
in real-life situations, incorporate the perspectives of peers and patients,
or use measures that predict clinical outcomes.ConclusionsIn addition to assessments of basic skills, new formats that assess
clinical reasoning, expert judgment, management of ambiguity, professionalism,
time management, learning strategies, and teamwork promise a multidimensional
assessment while maintaining adequate reliability and validity. Institutional
support, reflection, and mentoring must accompany the development of assessment
programs.

/pdf/defining-and-assessing-professional-competence-1sy606vpxu.pdf

Defining and Assessing Professional Competence

Measuring patient-centered communication in patient-physician consultations: theoretical and practical issues.

In May 1999, 21 leaders and representatives from major medical education and professional organizations attended an invitational conference jointly sponsored by the Bayer Institute for Health Care Communication and the Fetzer INSTITUTE: The participants focused on delineating a coherent set of essential elements in physician-patient communication to: (1) facilitate the development, implementation, and evaluation of communication-oriented curricula in medical education and (2) inform the development of specific standards in this domain. Since the group included architects and representatives of five currently used models of doctor-patient communication, participants agreed that the goals might best be achieved through review and synthesis of the models. Presentations about the five models encompassed their research base, overarching views of the medical encounter, and current applications. All attendees participated in discussion of the models and common elements. Written proceedings generated during the conference were posted on an electronic listserv for review and comment by the entire group. A three-person writing committee synthesized suggestions, resolved questions, and posted a succession of drafts on a listserv. The current document was circulated to the entire group for final approval before it was submitted for publication. The group identified seven essential sets of communication tasks: (1) build the doctor-patient relationship; (2) open the discussion; (3) gather information; (4) understand the patient's perspective; (5) share information; (6) reach agreement on problems and plans; and (7) provide closure. These broadly supported elements provide a useful framework for communication-oriented curricula and standards.

Essential elements of communication in medical encounters: the Kalamazoo consensus statement.

PURPOSE We wanted to identify published randomized trials of interventions to alter the interaction between patients and practitioners, develop taxonomies of the interventions and outcomes, and assess the evidence that such interventions improve patients’ health and well-being.

METHODS Undertaking a systematic review of randomized trials, we sought trials in primary and secondary care with health-related outcomes, which we found by searching MEDLINE, HealthSTAR, and PsycINFO bibliographic databases through 1999. We also completed one round of manual citation searching.

RESULTS Thirty-five trials were included. Most were set in primary care in North America. Trials were heterogeneous in populations, settings, interventions, and measures. Interventions frequently combined several poorly described elements. Explicit theoretical underpinning was rare, and only one study linked intervention through process to outcome measures. Health outcomes were rarely measured objectively (6 of 35), and only 4 trials with health outcomes met predefined quality criteria.

Interventions frequently altered the process of interactions (significantly in 73%, 22 of 30 trials). Principal outcomes favored the intervention group in 74% of trials (26 of 35), reaching statistical significance in 14 (40%). Positive effects on health outcomes achieved statistical significance in 44% of trials (11 of 25); negative effects were uncommon (5 of 25, 20%). Simple approaches to increasing the participation of patients in the clinical encounter, such as providing practitioners with a note from patients about their concerns beforehand, showed promise, as did more complex programs providing specific information about disease and attention to emotion. Apparently similar interventions varied in effectiveness across studies.

CONCLUSIONS Successful interactions between patients and their practitioners lie at the heart of medicine, yet there are few rigorous trials of well-specified interventions to inform best practice. Trial evidence suggests that a range of approaches can achieve changes in this interaction, and some show promise in improving patients’ health. To advance knowledge further, we need to replicate promising studies using rigorous methods. These should include explicit theoretical frameworks designed to link effects on key communication and interaction characteristics through to effects on health outcomes.

https://www.annfammed.org/content/annalsfm/2/6/595.full.pdf

Effect on Health-Related Outcomes of Interventions to Alter the Interaction Between Patients and Practitioners: A Systematic Review of Trials

OBJECTIVE: To examine the psychometric properties of a revised scale, named 'observing patient involvement in decision making' (OPTION), by analysing its reapplication to a sample of routine primary care consultations. The OPTION instrument assesses to what degree clinicians involve patients in decision making.

DESIGN: Cross-sectional assessment of medical interaction by two calibrated raters.

SETTING: Primary care.

PARTICIPANTS: Twenty-one general practitioners provided 186 consultations for assessment.

MEASUREMENTS: Observational score using the OPTION instrument.

RESULTS: Compared with the first version of the OPTION scale, the revised scale that uses a magnitude instead of an attitude scale, when applied to the same data set, resulted in improvement in the scale's reliability and to lower scores for the levels of involvement achieved by the practitioners. Factor analysis confirms that it is acceptable to regard the scale as a single construct. Although there is moderate variability when raters are assessed on an item by item basis, the agreements between raters at the level of the overall OPTION score is high (the intraclass correlation coefficient scores for total OPTION score was 0.77), a level that is acceptable for the evaluation of a set of consultations per practitioner (e.g. between 5 and 10 consultations), where aggregate scores would be used for determining overall performance.

CONCLUSIONS: We conclude that OPTION is sufficiently reliable to be used for formal assessment at the level of the whole instrument (all 12 items).

/pdf/the-option-scale-measuring-the-extent-that-clinicians-2jbhokgc7u.pdf

The OPTION scale: measuring the extent that clinicians involve patients in decision-making tasks.

The examination for membership of the Royal College of General Practitioners (MRCGP) consists of four modules. The consulting skills module, introduced in 1996, is normally taken by submitting a videotape of seven consultations, selected by the candidate to show “competency” in each of 15 “performance criteria.” These criteria are explained in the examination literature and on the college's website.1 The module is based on a competency model of assessment, which defines outcomes, in a hierarchical framework.2 The performance criteria (see table) are grouped into five areas: discovering the reason for the patient's attendance, exploring the problem(s), tackling the problem(s), explaining the problem(s), and making effective use of the consultation.

View this table:

Success of 2094 candidates in meeting each performance criterion in none, one, two, three, four, or all of the first five consultations submitted. Values are percentages (numbers) …

https://www.bmj.com/content/bmj/325/7366/691.full.pdf

Patient centredness in the MRCGP video examination: analysis of large cohort

The Royal College of General Practitioners' Membership examination , the only postgraduate qualification in family medicine in the United Kingdom , has developed a direct assessment of candidates' interpersonal skills performance using videotaped consultations of the actual doctor - patient encounters . At present about 1 , 200 doctors are examined each year . The methodology has been developed and piloted over a period of eight years . The central tenet of the methodology is a clear definition , which is known both to the candidate and to the examiner , of the clinical and consulting competencies that are required to be demonstrated in order to pass the examination . The candidate is required to provide evidence of his or her competence usually by selecting appropriate patient encounters that demonstrate the fulfillment of the required performance criteria , effectively producing a portfolio of his or her communicative competence . The methodology is intended to encourage the learning and teaching of com...

Assessing physicians' interpersonal skills via videotaped encounters: a new approach for the Royal College of General Practitioners Membership examination.

Objective

This paper describes the development of the tests of competence used as part of the General Medical Council’s assessment of potentially seriously deficient doctors. It is illustrated by reference to tests of knowledge and clinical and practical skills created for general practice.



Subjects and tests

A notional sample of 30 volunteers in ‘good standing’ in the specialty (reference group), 27 practitioners referred to the procedures and four practitioners not referred but who were the focus of concern over their performance. Tests were constructed using available guidelines and a specially convened working group in the specialty.



Methods

Standards were set using Angoff, modified contrasting group and global judgement methods, as appropriate.



Results

Tests performed highly reliably, showed evidence of construct validity, intercorrelated at appropriate levels and, at the standards employed, demonstrated good separation of reference and referred groups. Likelihood ratios for above and below standard performance based on competence were large for each test. Seven of 27 doctors referred were shown not to be deficient in both phases of the performance assessment.

The General Medical Council's Performance Procedures: the development and implementation of tests of competence with examples from general practice.

The silent onset of ovarian cancer often leads to metastatic spread before diagnosis can be made. The cause remains unknown, but molecular studies are beginning to reveal some mechanisms. Inactivation of tumour suppressor genes is thought to be responsible for initiating many forms of cancer. Studies of allele loss indicate areas of genome where inactivation may occur. We recently examined the relation between the clinical stage of epithelial ovarian cancer and allele loss at three loci on chromosome 17: progressive loss was detected with advancing stages of disease.' The clinical stage of a tumour defines the extent of the disease and is the best determinant of prognosis for most forms of cancer. It depends on the time elapsed before diagnosis, hence the value placed on early detection. The histopathological grade, however, better describes the intrinsic nature of the tumour: well differentiated tumours are slow growing whereas undifferentiated, anaplastic tumours metastasise rapidly. Thus poorly differentiated tumours tend to be diagnosed at a more advanced stage. At the molecular level one might anticipate less damage to genetic material with well differentiated tumours.

https://www.bmj.com/content/bmj/307/6903/542.2.full.pdf

MRCGP pass rate by medical school and region of postgraduate training. Royal College of General Practitioners.

Objective

To investigate the content validity of Paper 1 of the MRCGP examination.



Method

Exploratory factor analysis was carried out on candidate responses to Paper 1 of the May and October MRCGP examination in 1998. Contribution of each test question across factors was assessed using a pattern matrix of the oblique rotation. Common dimensions and variations between factor sets were identified. Key testing areas were then matched against the ‘domains of competence’ intended to be assessed by Paper 1 (as defined within the examination blueprint matrix).



Results

Whilst critical appraisal, disease prevention/evidence-based medicine and clinical management emerged as areas tested consistently, content variation was observed between factor sets extracted from both sittings. In addition, some overlap, in terms of domains tested, was seen among other assessment instruments used within the examination.



Conclusion

Paper 1 conforms to the majority of its stated intentions. However, further development of techniques for investigating validity will be required in order to minimize content variation between both sittings of the examination as well as to help more closely define areas of competence to be tested by Paper 1 of the MRCGP examination.

Assessing validity in written tests of general practice – exploration by factor analysis of candidate response patterns to Paper 1 of the MRCGP examination

John Foulkes

Papers

Patient centredness in the MRCGP video examination: analysis of large cohort

Assessing physicians' interpersonal skills via videotaped encounters: a new approach for the Royal College of General Practitioners Membership examination.

The General Medical Council's Performance Procedures: the development and implementation of tests of competence with examples from general practice.

MRCGP pass rate by medical school and region of postgraduate training. Royal College of General Practitioners.

Assessing validity in written tests of general practice – exploration by factor analysis of candidate response patterns to Paper 1 of the MRCGP examination