Showing papers in "Medical Education in 1979"

A comparison of multiple‐choice tests and free‐response tests in examinations of clinical competence

Abstract: Summary This paper reports a study which compared the performance of different groups of students and doctors on identical and equivalent tests set in an objective-type format and in a free-response format. The tests were designed to ensure that the content was relevant to clinical practice at the hospital intern level. In all test situations candidates’ scores were significantly higher in the objective tests than in the free-response tests. This difference was greater for the more junior and less competent students than for the more competent doctors. The cueing effect of the options was thought to be the main factor responsible for the difference in performance. The results of a questionnaire survey demonstrated that students were aware of the deficiencies in multiple-choice tests. A large majority of the students believed that the free-response tests gave a more accurate assessment of their clinical ability. It was found that in these tests, aimed at measuring aspects of clinical competence, multiple-choice questions appeared to overestimate the candidate's ability to an extent that made them less suitable than free-response questions for this purpose. It was also found that free-response tests, of the type used in this study, provide a suitable alternative to multiple-choice tests for use in the written section of clinical examinations. It was concluded that the written component of the final examination in the medical course should have a preponderance of free-response items over multiple-choice items.

Medical education: a student perspective.

Abstract: Discussions with a sample of final year students generated a questionnaire designed to elicit a phenomenological account of their experiences at medical school. Sources of dissatisfaction and stress were identified and the students also offered recommendations for improving the system.

The art and science of clinical teaching.

Abstract: A review of the literature on clinical teaching indicates that medical schools emphasize didactic teaching of facts. Interpersonal skills, problem solving skills and cultivation of attitudes are neglected. However, the literature also described the skills of 'good' clinical teachers, and enumerates them. Programmes attempting to develop students' interpersonal skills and problem solving skills are reported. Finally, areas for further practice and study in clinical teaching are recommended.

A preliminary investigation of three types of multiple choice questions.

Abstract: Summary The present study reports the results of an evaluation of three types of multiple choice questions—the five choice completion, multiple completion, and assertion-reason. Fifty-four questions, eighteen of each type and measuring the candidate on the same scientific principle and classified as either factual or comprehension, were developed and included in the General Surgery certifying examination of the Royal College of Physicians and Surgeons of Canada. In addition to using descriptive statistics, the multitrait-multi-method technique was used to investigate whether the item types measured different aspects of examinee capabilities. Results indicated that performance on the five choice completion and the multiple completion type questions was roughly the same, whereas performance on the assertion-reason type was lower. The results of the multitrait-multimethod validation revealed that the three item types were unable to discriminate between the two traits of factual and comprehension.

Teaching communication skills to pre-clinical medical students: a general practice based approach.

Abstract: Summary This paper describes a method of teaching communication skills to pre-clinical medical students in the setting of general practice. By focusing on the nature of the patient's problem this teaching tries to place interviewing and diagnostic procedures in their proper context in the doctor-patient relationship

The influence of a lecture course in loss and grief on medical students: an empirical study of attitude formation

Abstract: Summary In this study the results of a questionnaire administered to medical students attending a lecture course on loss and grief are analysed. The focus of the inquiry was to determine whether attitudes towards death, dying and loss could be influenced by confrontation with factual information on bereavement. The results showed that the original idealized attitudes underwent a reality correction due to increased factual knowledge of bereavement processes. The defensive attitudes of students who experienced death in their family during the course are discussed. Intensity and temporality of one's involvement with death appear to be a determining factor in the formation of attitudes towards death. Typical male and female attitudes are also presented. It is concluded that the institutionalization of education in bereavement processes should receive priority in medical programmes and that the classical lecture deserves its place therein.

An investigation of change in medical students' conceptualizations of psychiatric patients due to a short training course in psychiatry.

Abstract: The paper describes an investigation of medical students' conceptualizations of different psychiatric patients categories, using the Katz-Braly stereotype measuring technique and the semantic differential technique. The conceptual content of sixteen fourth year medical students' stereotypes of psychiatric patient categories was elicited and their evaluations of the categories were measured. These measures were taken before and after a two-month training course in psychiatry and special attention was paid to changes that appeared to result from the educational process. The Katz-Braly stereotype measure produces a work picture of each patients category involving four traits; for instance for the category neurotic the following traits were derived: 'temperamental', 'hypochondriacal', 'worried' and 'tense'. The important triats for each category were analysed separately to determine if any statistically significant changes in the stereotype had occurred over the two-month training course. Several significant additions and omissions of traits were found. These changes may be explained as the result of teaching, the students' direct experience of psychiatric patients and their acquisition of a new technical vocabulary. Despite the changes in the students' stereotypes of patients, their evaluations of the patients, as measured by the semantic differential, remained substantially unaltered by the training course.

Interviewing skill: the effects of ultra-brief training for general practitioners. A preliminary report.

Abstract: Summary Concepts of interviewing skill and of appropriate training and evaluation are briefly reviewed. It is hypothesized that if goals and skills relevant to a specific clinical interview can be identified and modelled in advance then even very brief training can improve outcome. An experiment is described in which three randomly selected general practitioners conducted, separately, test interviews with the same role played ‘patient’. These interviews were repeated after a ‘goal focusing’ exercise and again after a modelling and instruction exercise. All interviews were tele-recorded and subsequently blindly rated for impact on the ‘patient’. The results suggest powerful training effects. The drawbacks of the study, such as lack of controls, are discussed. The similarity of evaluation raters who were ‘behaviour’ orientated and those who were ‘psycho-dynamics’ orientated is noted and it is suggested that agreed concepts of effective interviewing may be within reach.

Curriculum content from critical incidents

Abstract: Summary The aim of this study was the creation of a medical school curriculum which meets the needs of students, teachers, and community. The method for collecting data was the Critical Incident Technique (CIT). Using appropriate modifications in questions and format, responses were sought from the medical school staff and students, all alumni of the medical school, members of the state medical society, nursing and technical personnel of the school's three teaching hospitals, and patients at each of these hospitals. Each respondent was asked to recall as many incidents as possible in which he observed a doctor do (or fail to do) something which was critical to the care of a patient. From 2,739 individuals there were 942 responses. These yielded 2,181 critical incidents which, in turn, were dissected into 5,657 critical behaviours. These behaviours were catalogued in rank order under the following headings: (1) wisdom in deciding on appropriate care; (2) diagnostic acumen; (3) responsibility as a doctor; (4) effectiveness of doctor-patient relationship; (5) physical examination skill; (6) medical education; (7) test technique and understanding; (8) judgment and skill in implementing care; (9) competence of continuing care; (10) history-taking competence. Each behaviour was reviewed by a committee of teachers which gave an opinion as to whether it was an essential competence for a medical school graduate. The resulting list constitutes a core curriculum for the medical school.

A factor analytic study of branching patient management problems.

Abstract: Summary This study was undertaken to determine the factor analytic structure of patient management problems (PMPs) and to determine whether such factors are stable for different groups taking the same examination and for the same group over time. Two examinations were administered to a group of medical students, the first during their junior year and the second during their senior year. The second examination was also administered to a second class of students during their junior year. Factor analyses results indicated there are two components to medical problem-solving as measured by PMPs-data gathering and management. Both factors were stable over groups and over time.

Videotapes in psychiatry: their use in teaching observation techniques

Abstract: Summary In order to assess the effectiveness of videotapes in teaching mental state examination, two groups of medical students, one who had received videotape teaching and one not exposed to videotapes, are compared in their ability to observe accurately the mental state during a live interview. Accuracy of observation was assessed by obtaining a measure of the students’ ability to shift from an incorrect opinion about a patient to a more accurate assessment following the live interview. Students who had received videotape teaching were able to make more accurate judgements than those who had not. This ability was not related to certain personality attitudes measured, nor to an ability to detect subjective elements in the interview, nor is it reflected in their examination results. The only significant variable found to influence this ability was whether or not the student had received videotape teaching.

Communication skills and undergraduate medical education.

Abstract: This paper reports some aspects of teaching communication skills based on experience with over 100 pre-clinical students in each of two years. Students interviewed patients selected by general practitioner tutors from their practices and 214 interviews, recorded on videotape, were subsequently analysed. Without having been given detailed instruction, approximately two-thirds of each class of second year students (mean age 19 1/2 years) sustained a consultation for 5 minutes or more. Over one half of the students elicited the salient features of the patients' problems within this short time. Anxiety exhibited by the majority of students could be seen to influence the interviews in several ways. These manifestations and the ways in which the interview was affected are briefly described. When some of the patients were subsequently interviewed by postgraduate trainees in general practice, the consultations differed in a number of respects. These differences are described, and the possible significance of the observations is discussed in relation to medical education.

A human sexuality course for Oxford University medical students

Abstract: Summary A human sexuality course for clinical students in Oxford was held based on the format of similar courses held in the U.S.A. with one-and-a-half days of films and lectures followed by discussion in small groups. The course was largely successful and evaluation of attitudes and knowledge revealed that significant preliminary changes resulted. Students who failed to complete the course had more inhibited attitudes towards sexuality and less sexual information compared with those who attended the whole course. Advice is given to those intending to hold such a course elsewhere, including how more inhibited students, who probably are most in need of such teaching, might be encouraged to participate.

Attitudes of Ugandan medical students toward ‘straight’ and ‘modified’ ECT

Abstract: In Third World countries, limited resources of anaesthetists and resuscitation facilities mean that electroconvulsive therapy (ECT) is usually given 'straight' (i.e. without anaesthetic and muscle relaxant). This study compared the attitudes of Ugandan medical students towards ECT given straight and ECT administered with muscle relaxant and general anaesthesia (i.e. 'modified' ECT). Results confirmed the clinical impression that students regarded straight ECT as an inhumane method of treatment. However, whilst this was a major factor influencing students' preference for modified ECT as a treatment for themselves or for a close friend, it was relatively unimportant in determining their choice of ECT technique for a patient. Convenience of administration was the main criterion applied by students when deciding whether they would prefer to administer straight or modified ECT to a patient. Implications for medical education and the development of professional attitudes are discussed.

A course on collaboration between social workers and general practitioners during their vocational training.

Abstract: Summary The educational programme reported was an experiment in the vocational training scheme of the Department of General Practice, Erasmus University, Rotterdam, Holland, and is now part of the course. The programme focused on the training in team function (co-operation) given to trainee GPs and social workers. It became clear that both groups during their professional training develop markedly different attitudes and views about patient (client) care. These differences form a fundamental handicap in any discussion about teamwork. During the programme the students were made aware of this divergence of viewpoint and were taught how to handle these resulting handicaps and, if possible, to eliminate them.

The evolution of a course in human sexuality University of Edinburgh, 1972-1978.

Abstract: Summary A short undergraduate course in human sexuality has been developed over a period of 7 years. The objectives, initially concerned with cognitive learning, shifted to affective aspects of the topic, introduced through a format of sexually explicit films and small-group discussions. Continuing evaluation of successive courses by students has been particularly valuable in identifying helpful and unhelpful behaviour in group leaders, and has been used in their training. The evaluations also demonstrated that students saw a need to develop their interviewing skills. The course, therefore, now seeks to combine affective learning with other topics of direct relevance to clinical practice.

A pilot experiment on the inter‐examiner reliability of short essay questions

Abstract: Although long essay questions have been shown generally to have a low inter-examiner reliability, they are still much used in undergraduate medical examinations, evidently because examiners feel that essays have greater inherent validity than some of the modern 'objective' techniques such as multiple choice questions (MCQs) Evidence exists in the literature to suggest that the reliability of short (c 10 minutes) essay questions may be higher A pilot experiment is reported in which students answered six questions requiring a short essay or extended notes (10 minutes per question): these 'SEQs' were each marked by two examiners With the exception of one question, agreement between examiners was good The implications of the experiment are considered to be: (a) that SEQs may well have a useful role in a broadly-based examination system; (b) there is a need for more experimentation with this type of question, which can arguably have greater validity than the (highly reliable) MCQ; and (c) that dual marking of all essay scripts is most desirable

Clinical problem solving and clinical knowledge.

Abstract: Videotape and simulated patients were used for evaluating third year medical students' performances in solving familiar and unfamiliar clinical problems. To assist the doctor to rate the students' performances by the review of videotapes, criteria were set up for students' behaviours that could be construed as evidence that the students recognized cues obtained by interviewing the patient and the information content of such cues. Criteria for data use by the students were also set. A mean of 90% of the pertinent medical history data was collected by the students. The amount of pertinent historical data collected was found to correlate very closely with student-patient interaction analysis score for the familiar and unfamiliar problems (r = 0.89 and r = 0.92, respectively). Significant differences in mean scores for data generation (P less than 0.001) and for data use (P less than 0.01) were found between those students who solved and those who failed to solve the familiar problem. With the unfamiliar problem a significant difference was found between the mean scores for data generation (P less than 0.01), but not for data use. The implications of these findings in medical education are discussed.

Effectiveness of instructional computers in teaching basic medical sciences.

Abstract: Summary The present study was designed to investigate the effectiveness of computer-based PLATO IV basic medical science lessons. Effectiveness was operationalized in terms of increased performance on basic medical science examinations for those medical students who had used the lessons when compared to those students who had not. Usage of the PLATO lessons was quantified as‘minutes of use’ of the relevant lessons. Data were gathered in 1976–77 from first-year medical students at two sites, both under the auspices of one college of medicine. Usage of PLATO lessons and subsequent performance on three subtests from three different examinations were analysed. The findings from the current study offer encouragement that use of PLATO basic medical science materials contribute to increased performance on subsequent examinations.

The effect of continuing medical education programmes on clinical practice: fact or fantasy.

Abstract: Summary This paper reports an evaluation study of a continuing medical education programme on the haemagglutination-inhibition (H.I.) test for rubella antibodies in terms of its effect on the practice of the doctors in attendance. The study employed a pre-post control group design. The data on the doctors’ use of the H.I. test were extracted from laboratory records. Through an analysis of co-variance of these data, the lecture was shown to have had an apparently significant impact on the practice of the doctors in attendance. However, available data describing the number of H.I. tests requested by a population of doctors place serious doubt on this conclusion. This paper describes the design of this study as a prototype, identifies constraints which impede the implementation of such studies, and justifies the need to pursue studies of this nature in spite of these constraints.

Development and exploratory trials of a cognitive preference inventory for medical students.

Abstract: Summary A medical cognitive preference inventory was developed and tested with two samples, one in Israel and the other in the United States. Acceptable levels of internal consistency of the whole test and of its three subtests were demonstrated. Direct and indirect evidence for the validity of the test was provided. The potential uses of the test for student selection and evaluation as well as for programme evaluation were discussed. Two forms, E and F, each consisting of eighteen items, are recommended for use with medical students. A combination of these two forms is designated as form G. Administration of one form to half of a sample and the other form to the other half, followed by pooling the individual scores, thereby obtaining results comparable to those of form G, is recommended when time to administer the inventory is limited.

Attitudes of health science students towards teaching practices, examinations, and other related issues.

Abstract: A brief multiple choice questionnaire explored the attitudes of students enrolled in several health science faculties (medicine, pharmacy, nursing, and physical and health education) A substantial proportion indicated a wish for more challenging lectures On the other hand, the majority of students devoted less than one hour of personal study per week to required 'courses', interest in health-related 'courses' being particularly weak Small textbooks with minimal references were desired, and relatively little use was made of free lecture handouts Financial constraints had little influence on personal study, but affected textbook purchase The demand for 'handouts' would have been reduced by a charge Frequent term examinations as a stimulus to learning were favoured by all groups, as were full or challenging presentations by instructors Team teaching, although favoured generally, was not equally appealing to all groups

Inversions in true/false and in multiple choice questions – a new form of item analysis

Abstract: Summary Total ignorance is expressed not by consistently selecting the wrong answers in a true/false or multiple choice examination, but rather by making a random choice from all the alternatives available, resulting in a predictable proportion of correct responses. When a significant majority of students answers incorrectly in a true/false question, it must therefore be concluded that this is not the result of ignorance, but of some factor which has led to their deliberately choosing the wrong answer. This situation has been termed an ‘inversion’. Of 2057 true/false examination questions set in this department over the past 3 years, 140 (6.8%) were found to be ‘inverted’. In about one third of these cases the ‘inversion’ was found to be due to a wrong model answer, an ambiguous or misleading question, or to incorrect teaching. The remaining two-thirds of the ‘inversions’ could be ascribed to attempts to work out the answer from inadequate or inappropriate first principles. It has been our policy to delete all ‘inverted’ questions from the examination (unless the model answer needed to be changed), because the score obtained when the students deliberately chose the wrong answer is less than the score they would have obtained had they been totally ignorant and had ventured a guess or had chosen not to answer the question. This concept can also be applied to multiple choice questions.

Large 'small' groups: their place in higher education.

Abstract: Summary Small group teaching in higher education appears to be an ideal worth aiming for. The review of the literature in this paper shows that factural information about the effectiveness of such teaching is far from prolific. In addition, what may be the most common form of small group, viz. the ‘seminar’, has received far less attention than the‘lecture’ or‘tutorial’. Special attention is drawn to the relevance of the studies for medical education but the overall finding is that need for well designed evaluative research is urgent, particularly with regard to ‘seminar’ learning groups.

The beginning doctor: a problem-based course for first year medical and dental students.

Abstract: Summary A problem-based course designated as ‘Beginning Doctor’ was given to 136 first year medical and dental students. The students were divided into small groups and under guidance of a tutor studied largely unfamiliar clinical problems presented to them as written resumes. They determined what further information they required, obtained the information on their own and described it to the others in plenary sessions. The tutors acted as facilitators of student learning rather than as sources of information. Contrary to the widely held assumption that students in these parts of the world are passive learners, incapable of taking direct responsibility for their own learning, the staff was pleasantly surprised to see the enthusiasm and the competence with which the students tackled the problems and easily adopted the self-learning mode.

Survey of the career experience and postgraduate training of the 1965 and 1970 graduates of the Scottish university medical schools.

Abstract: Summary A survey of the career experience and postgraduate training of the 1965 and 1970 graduates of the Scottish university medical schools was carried out in 1975. The duration of training for the specialties of medicine and surgery and for obstetrics and gynaecology was in general longer than for anaesthetics, psychiatry and radiology; the implications of varying periods of training for postgraduate education are discussed. Attention is drawn to a number of factors which influenced career choice. The 1970 graduates who became principals spent longer in training for general practice than those of 1965. Some married women with children had difficulty in finding suitable part-time work; this may be a serious problem for those seeking accreditation of higher specialist training and for training for general practice. It was estimated that, of the British nationals, about 11% of the 1965 and 8% of the 1970 graduates had emigrated. General practice was the discipline most commonly chosen by doctors working overseas.

Independent learning packages (audio tape/pamphlet) for a course in cell excitation: evidence that they can be at least as effective as conventional lectures

Abstract: Summary Three tape/pamphlet independent learning packages have been constructed to serve the same learning objectives as three conventional lectures on the physiology of the nerve axon. The learning efficacy of each package was compared with that of its companion lecture with the co-operation of fifty first-year medical students in a controlled experiment. The packages were at least as effective as the lectures in generating both short- and long-term attainment of their intrinsic learning objectives. The findings are discussed in relation to learning efficacy, student study time, student acceptability, package preparation time and costing.