Showing papers by "Brian Jolly published in 2015"

Evaluation of assessment in the context of work‐based learning: Qualitative perspectives of new graduates

Abstract: AimEvaluation or assessment of competence is an important step to ensure the safety and efficacy of health professionals, including dietitians. Most competency-based assessment studies are focused on valid and reliable methods of assessment for the preparation of entry-level dietitians, few papers have explored student dietitians' perceptions of these evaluations. The present study aimed to explore the perceptions of recent graduates from accredited nutrition and dietetics training programs in Australia. It also aimed to establish the relevance of competency-based assessment to adequately prepare them for entry-level work roles.

Medical school benchmarking – From tools to programmes

Abstract: Background: Benchmarking among medical schools is essential, but may result in unwanted effects.Aim: To apply a conceptual framework to selected benchmarking activities of medical schools.Methods: We present an analogy between the effects of assessment on student learning and the effects of benchmarking on medical school educational activities. A framework by which benchmarking can be evaluated was developed and applied to key current benchmarking activities in Australia and New Zealand.Results: The analogy generated a conceptual framework that tested five questions to be considered in relation to benchmarking: what is the purpose? what are the attributes of value? what are the best tools to assess the attributes of value? what happens to the results? and, what is the likely “institutional impact” of the results? If the activities were compared against a blueprint of desirable medical graduate outcomes, notable omissions would emerge.Conclusion: Medical schools should benchmark their performance o...

Can I Leave the Theatre? A Key to More Reliable Workplace-Based Assessment

Abstract: Background The value of workplace-based assessments such as the mini-clinical evaluation exercise (mini-CEX), and clinicians' confidence and engagement in the process, has been constrained by low reliability and limited capacity to identify underperforming trainees. We proposed that changing the way supervisors make judgements about trainees would improve score reliability and identification of underperformers. Anaesthetists regularly make decisions about the level of trainee independence with a case, based on how closely they need to supervise them. We therefore used this as the basis for a new scoring system. Methods We analysed 338 mini-CEXs where supervisors scored trainees using the conventional system, and also scored trainee independence, based on the need for direct, or more distant, supervision. As supervisory requirements depend on case difficulty, we then compared the actual trainee independence score and the expected trainee independence score obtained externally. Results Compared with the conventional scoring system used in previous studies, reliability was very substantially improved using a system based on a trainee's level of independence with a case. Reliability improved further when this score was corrected for case difficulty. Furthermore, the new scoring system overcame the previously identified problem of assessor leniency and identified a number of trainees performing below expectations. Conclusions Supervisors' judgements on trainee independence with a case, based on the need for direct or more distant supervision, can generate reliable scores of trainee ability without the need for an onerous number of assessments, identify trainees performing below expectations, and track trainee progress towards independent specialist practice.