Acknowledgements the authors thank the UK General Medical Council (GMC) for commissioning an initial rapid review of the literature on selection and widening access to medicine in 2013. Further thanks are due to the UK Medical Schools Council (MSC) for commissioning a significantly updated review of selection methods in 2014, funding for which was provided by Health Education England (HEE) and the Office for Fair Access (OFFA). Working together on both of these projects encouraged us to further develop our ideas and produce an updated systematic review for publication in 2015. We also thank those who contributed to the original project funded by the GMC, notably John McLachlan, Member of the Centre for Medical Education Research, Durham University, and Emma Dunlop, Medical Admissions, University of Aberdeen. Funding funding was provided by HEE and the OFFA.

/pdf/how-effective-are-selection-methods-in-medical-education-a-1qnthmo28f.pdf

How effective are selection methods in medical education? A systematic review

The present study examines whether video-based situational judgement test (SJT) formats provide benefits over “traditional” text-based SJTs. Focusing on three SJT conditions – two video-based condi...

https://www.tandfonline.com/doi/pdf/10.1080/1359432X.2020.1736619

Using Video-Based Situational Judgment Tests for Teacher Selection: A Quasi-Experiment Exploring the Relations between Test Format, Subgroup Differences, and Applicant Reactions

Erfolgreiche schulische Bildungsprozesse sind – neben den Lernvoraussetzungen von Schuler/innen – insbesondere auch vom Handeln der Lehrer/innen und damit auch von deren kognitiven und nicht-kognitiven Handlungsdispositionen abhangig. Damit kommt der Auswahl von Personen fur Lehramtsstudien eine zentrale Rolle zu. Neben geeigneten Attrahierungsstrategien und einer wirkungsvollen Lehrer/innenausbildung lassen sich fur den Lehrer/innenberuf geeignete Personen vor allem durch Masnahmen der Selbst- und der Fremdselektion rekrutieren. Der Beitrag stellt ein neues dreistufiges, theoretisch und empirisch fundiertes Verfahren zur Auswahl von Lehramtsstudierenden vor, welches fur den Einsatz an osterreichischen Universitaten und Padagogischen Hochschulen konzipiert wird, wobei grundlegende Konzepte und Befunde der Lehrer/innenbildungsforschung und der psychologischen Eignungsdiagnostik, im Speziellen der Personalauswahl, Berucksichtigung finden. Das dreistufige Verfahren besteht aus (1.) einem nicht-selektiven Self-Assessment Verfahren, welches eine Selbsterkundung auf Basis von motivationalen, personlichkeitsstrukturellen und biografischen Merkmalen ermoglicht und umfangreiche Informationen zum Lehrer/innenberuf bereitstellt. (2.) werden durch standardisierte psychologische Tests allgemeine kognitive Funktionen, sprachliche, emotionale und kreativitatsbezogene Kompetenzen sowie ausgewahlte Personlichkeitsmerkmale erfasst. (3.) In einem Face-to-Face-Assessment werden auf standardisierte Weise verbale und nonverbale Kommunikationsfahigkeit, kognitive Entscheidungsfahigkeit und Selbstreflexionskompetenz ermittelt. Kunftige Powered by Editorial Manager® and ProduXion Manager® from Aries Systems Corporation Forschungsdesiderata und Aufgaben fur die Evaluation der Verfahren werden insbesondere im Aufbau einer umfassenden Langsschnittstudie gesehen, die neben ausbildungsbezogenen Kriterien aus Studium und Praktikum auch Erfolgskriterien der Tatigkeit im Lehrer/innenberuf erfassen soll. Erste empirische Befunde zur prognostischen Validitat in der ersten Studienphase werden berichtet.

/pdf/tesat-ein-neues-verfahren-zur-eignungsfeststellung-und-4w4kjec773.pdf

TESAT – Ein neues Verfahren zur Eignungsfeststellung und Bewerberauswahl für das Lehramtsstudium

Progress tests provide students feedback on their level of proficiency over the course of their medical studies. Peer-assisted learning and competency-based education have become increasingly important in medical education. Although progress tests have been proven to be useful as a longitudinal feedback instrument, there are currently no progress tests that have been created in cooperation with students or that focus on competency in medical education. In this study, we investigated the extent to which students can be included in the development of a progress test and demonstrated that aspects of knowledge related to competency can be represented on a competency-based progress test.A two-dimensional blueprint for 144 multiple-choice questions (MCQs) covering groups of medical subjects and groups of competency areas was generated by three expert groups for developing the competency-based progress test. A total of 31 students from seven medical schools in Germany actively participated in this exercise. After completing an intensive and comprehensive training programme, the students generated and reviewed the test questions for the competency-based progress test using a separate platform of the ItemManagementSystem (IMS). This test was administered as a formative test to 469 students in a pilot study in November 2013 at eight medical schools in Germany. The scores were analysed for the overall test and differentiated according to the subject groups and competency areas.A pool of more than 200 MCQs was compiled by the students for pilot use, of which 118 student-generated MCQs were used in the progress test. University instructors supplemented this pool with 26 MCQs, which primarily addressed the area of scientific skills. The post-review showed that student-generated MCQs were of high quality with regard to test statistic criteria and content. Overall, the progress test displayed a very high reliability. When the academic years were compared, the progress test mapped out over the course of study not only by the overall test but also in terms of the subject groups and competency areas.Further development in cooperation with students will be continued. Focus will be on compiling additional questions and test formats that can represent competency at a higher skill level, such as key feature questions, situational judgement test questions and OSCE. In addition, the feedback formats will be successively expanded. The intention is also to offer the formative competency-based progress test online.

Development of a competency-based formative progress test with student-generated MCQs: Results from a multi-centre pilot study.

Most psychological questionnaires face issues of response bias in respondent-reported scales, inadequacy for criterion-reference testing, or difficulty in estimating a large number of latent traits. Situational tests together with the general nominal diagnosis model framework provide a viable alternative to alleviate these concerns. Under this framework, there are different ways to design situationally nominal items, which can offer more flexibility for test development. Any response bias remaining with respondent-reported questionnaires may be addressed with appropriate test designs. The saturated model subsumes different reduced forms that can help inform whether the test is designed as expected. Two simulation studies are presented to demonstrate the effectiveness of the models and designs.

/pdf/test-designs-and-modeling-under-the-general-nominal-3jrt42fcaf.pdf

D Metcalfe

Papers

Situational Judgement Test