The assessment of professional competence: Developments, research and practical implications.

Background Research in clinical reasoning has been conducted for over 30 years. Throughout this time there have been a number of identifiable trends in methodology and theory.

Purpose  This paper identifies three broad research traditions, ordered chronologically, are: (a) attempts to understand reasoning as a general skill − the ‘clinical reasoning' process; (b) research based on probes of memory − reasoning related to the amount of knowledge and memory; and (c) research related to different kinds of mental representations − semantic qualifiers, scripts, schemas and exemplars.

Results and Conclusions  Several broad themes emerge from this review. First, there is little evidence that reasoning can be characterised in terms of general process variables. Secondly, it is evident that expertise is associated, not with a single basic representation but with multiple coordinated representations in memory, from causal mechanisms to prior examples. Different representations may be utilised in different circumstances, but little is known about the characteristics of a particular situation that led to a change in strategy.

Implications  It becomes evident that expertise lies in the availability of multiple representations of knowledge. Perhaps the most critical aspect of learning is not the acquisition of a particular strategy or skill, nor is it the availability of a particular kind of knowledge. Rather, the critical element may be deliberate practice with multiple examples which, on the hand, facilitates the availability of concepts and conceptual knowledge (i.e. transfer) and, on the other hand, adds to a storehouse of already solved problems.

/pdf/research-in-clinical-reasoning-past-history-and-current-n8pgre7mvk.pdf

Research in clinical reasoning: past history and current trends.

Credit scoring is the term used to describe formal statistical methods used for classifying applicants for credit into "good" and "bad" risk classes. Such methods have become increasingly important with the dramatic growth in consumer credit in recent years. A wide range of statistical methods has been applied, though the literature available to the public is limited for reasons of commercial confidentiality. Particular problems arising in the credit scoring context are examined and the statistical methods which have been applied are reviewed.

Statistical Classification Methods in Consumer Credit Scoring: A Review

SUMMARY Credit scoring is the term used to describe formal statistical methods used for classifying applicants for credit into 'good' and 'bad' risk classes. Such methods have become increasingly important with the dramatic growth in consumer credit in recent years. A wide range of statistical methods has been applied, though the literature available to the public is limited for reasons of commercial confidentiality. Particular problems arising in the credit scoring context are examined and the statistical methods which have been applied are reviewed.

Statistical Classification Methods in Consumer Credit Scoring: a Review

The real test bias: Influences of testing on teaching and learning.

Employing simulation technique analogous to that utilized in management games and military exercises, the authors have directed the development of a series of branched problems in patient management requiring sequential analysis and decision. These problems are designed to measure aspects of behavior defined by a criterion group as essential components of clinical competence. They are suitable for either individual or group administration and are amenable to either clerical or computer scoring and analysis (Crawford & Lewy, 1965). This paper is one section of a four-part report dealing with the techniques of constructing, scoring (Williamson, 1964), estimating the reliability (Lewy & McGuire, 1966) and assessing the validity of such problems. Primary attention is devoted here to the rationale and design of simulated clinical problems with only a brief summary of the scoring methods and statistical studies of presently available exercises. CONSIDERATION IN THE DEVELOPMENT OF SIMULATION EXERCISES IN MEDICINE A clinical problem which purports to simulate the physician-patient encounter must have the following characteristics: First, it must be initiated by information of the type a patient gives a physician, not by a predigested summary of the salient features of the case, and if it is to be realistic it must be couched in terms the patient or a referring physician would use. Second, the exercise must require a series of sequential, interdependent decisions representing the various stages in the diagnostic work-up and management of the patient. Third, the examinee must be able to obtain in realistic form information about the results of each decision, as a basis for subsequent action. Fourth, once these data are obtained it must be impossible for the examinee to retract a decision that is revealed to be ineffectual or harmful. Fifth, the problem must be constructed so as to allow both for different medical approaches and for variation in patient responses appropriate to these several approaches. Hence, provision must be made for modifications in the problem as the patient responds to the specific interventions chosen by each

Simulation technique in the measurement of problem‐solving skills1

Descriptive studies of clinical reasoning have been based primarily on an information-processing paradigm that has entailed microscopic analyses of the verbal behavior of small numbers of persons confronted with the task of making a diagnosis of a patient with symptoms of acute illness. From these studies, researchers have generally concluded: that medical problem-solving is an hypothetico-deductive process and that performance is not generalizable across problems. The author of the present paper argues that this narrow approach casts serious doubt on the validity of the conclusions and leaves unilluminated a large portion of the cognitive process involved in decisions about patient care. On the other hand, prescriptive, decision-analysis studies of medical problem-solving have generally been based on sophisticated decision theory that involves calculation and manipulation of complex probability and utility values in order to arrive at optimal decisions that will maximize patient benefits. While these prescriptive studies offer a methodology for improving clinical judgment, that methodology may not be one which decision-makers find attractive or feasible to employ.

Medical problem-solving: a critique of the literature.

Although the multiple-choice technique has been widely used in achievement testing for many years, these instruments have recently been subjected to mounting criticism Most of this criticism is based on the assertion that multiple-choice exercises focus on the measurement of the recall of isolated bits of information which have little relevance to any meaningful behavioral objectives (Hoffman, 1962Lange, 1967) In 1964 the American Board of Orthopaedic Surgery in collaboration with the Center for the Study of Medical Education at the University of Illinois College of Medicine, initiated a joint research project to investigate the reliability and validity of alternative techniques for assessing professional competence This report is designed to summarize the findings of that study relevant to the multiple-choice

The Validity of Multiple Choice Achievement Tests as Measures of Competence in Medicine

https://files.eric.ed.gov/fulltext/ED091152.pdf

Simulation Technique in the Teaching and Testing of Problem-Solving Skills.

In order to attempt to assess aspects of clinical competence, not adequately assessed by other means, the Center for the Study of Medical Education, University of Illinois College of Medicine together with the American Board of Orthopaedic Surgery developed oral examinations in formats specifically designed to yield information on high level cognitive functioning. The examinations were administered to 784 candidates for certification in January 1968. Reliability of the oral problem-solving component score pooled from four examiners was approximately .50. Assessment of content, construct, and concurrent validity made by questionnaire and factor analytic studies indicated that the oral tests identified factors not measured by multiple-choice tests and, therefore, significantly improved the relationship between supervisory evaluations and test scores.

Christine H. McGuire

Papers

Simulation technique in the measurement of problem‐solving skills1

Medical problem-solving: a critique of the literature.

The Validity of Multiple Choice Achievement Tests as Measures of Competence in Medicine

Simulation Technique in the Teaching and Testing of Problem-Solving Skills.

The Validity and Reliability of Oral Examinations in Assessing Cognitive Skills in Medicine.