Objectives

The purpose of this study was to use meta-analysis to establish which of the information available to the resident selection committee is associated with resident or doctor performance.



Methods

Multiple electronic databases were searched to 4 September 2012. Two reviewers independently selected studies that met the present inclusion criteria and extracted data in duplicate; disagreement was resolved by consensus. Risk for bias was assessed using a customised bias assessment tool. Measures of association were converted to a common effect size (Hedges' g). Meta-analysis was performed using the random-effects model for each selection strategy and all outcomes without pooling. Sensitivity analysis for each selection strategy–outcome pair was performed with pooling of effect size.



Results

Eighty studies involving a total of 41 704 participants were included in the meta-analysis. Seventeen different selection strategies and 17 outcomes were assessed across these studies. The strongest positive associations referred to examination-based selection strategies, such as the US Medical Licensing Examination (USMLE) Step 1, and examination-based outcomes, such as scores on in-training examinations. Moderate positive associations were present for medical school marks and both examination-based and subjective outcomes. Minimal or no associations were seen for the selection tools represented by interviews, reference letters and deans' letters.



Conclusions

Standardised examination performance and medical school grades show the strongest associations with current measures of doctor performance. Deans' letters, reference letters and interviews all show a lower than expected strength of association given the relative value often assigned to them during resident doctor selection. Objective selection strategies are potentially the most useful to residency selection committees based on current evaluative methods. However, reports in the literature of validated long-term doctor performance outcomes are scant.

https://onlinelibrary.wiley.com/doi/pdf/10.1111/medu.12234

Associations between residency selection strategies and doctor performance: a meta-analysis

The Predictive Value of General Surgery Application Data for Future Resident Performance

The Change of USMLE Step 1 to Pass/Fail: Perspectives of the Surgery Program Director.

Internal medicine residents are required to participate in scholarly activity, but conducting original research during residency is challenging. Following a poor Match at Baystate Medical Center, the authors implemented a resident research program to overcome known barriers to resident research. The multifaceted program addressed the following barriers: lack of interest, lack of time, insufficient technical support, and paucity of mentors. The program consisted of evidence-based medicine training to stimulate residents' interest in research and structural changes to support their conduct of research, including protected time for research during ambulatory blocks, a research assistant to help with tasks such as institutional review board applications and data entry, a research nurse to help with data collection, easily accessible biostatistical support, and a resident research director to provide mentorship. Following implementation in the fall of 2005, there was a steady rise in the number of resident presentations at national meetings, then in the number of resident publications. From 2001 to 2006, the department saw 3 resident publications. From 2006 to 2012, that number increased to 39 (P< .001). The department also saw more original research (29 publications) and resident first authors (12 publications) after program implementation. The percentage of residents accepted into fellowships rose from 33% before program implementation to 49% after (P = .04). This comprehensive resident research program, which focused on evidence-based medicine and was tailored to overcome specific barriers, led to a significant increase in the number of resident Medline publications and improved the reputation of the residency program.

/pdf/implementing-a-resident-research-program-to-overcome-7w3ambq9fs.pdf

Implementing a resident research program to overcome barriers to resident research.

Background
It is believed that the type of educational environment in teaching hospitals may affect the performance of medical knowledge base among residents, but this has not yet been proven.

/pdf/the-hospital-educational-environment-and-performance-of-19qee5l992.pdf

The hospital educational environment and performance of residents in the General Medicine In-Training Examination: a multicenter study in Japan.

IMGs constitute about a third of the United States (US) internal medicine graduates. US residency training programs face challenges in selection of IMGs with varied background features. However data on this topic is limited. We analyzed whether any pre-selection characteristics of IMG residents in our internal medicine program are associated with selected outcomes, namely competency based evaluation, examination performance and success in acquiring fellowship positions after graduation. We conducted a retrospective study of 51 IMGs at our ACGME accredited teaching institution between 2004 and 2007. Background resident features namely age, gender, self-reported ethnicity, time between medical school graduation to residency (pre-hire time), USMLE step I & II clinical skills scores, pre-GME clinical experience, US externship and interest in pursuing fellowship after graduation expressed in their personal statements were noted. Data on competency-based evaluations, in-service exam scores, research presentation and publications, fellowship pursuance were collected. There were no fellowships offered in our hospital in this study period. Background features were compared between resident groups according to following outcomes: (a) annual aggregate graduate PGY-level specific competency-based evaluation (CBE) score above versus below the median score within our program (scoring scale of 1 – 10), (b) US graduate PGY-level specific resident in-training exam (ITE) score higher versus lower than the median score, and (c) those who succeeded to secure a fellowship within the study period. Using appropriate statistical tests & adjusted regression analysis, odds ratio with 95% confidence intervals were calculated. 94% of the study sample were IMGs; median age was 35 years (Inter-Quartile range 25th – 75th percentile (IQR): 33–37 years); 43% women and 59% were Asian physicians. The median pre-hire time was 5 years (IQR: 4–7 years) and USMLE step I & step II clinical skills scores were 85 (IQR: 80–88) & 82 (IQR: 79–87) respectively. The median aggregate CBE scores during training were: PG1 5.8 (IQR: 5.6–6.3); PG2 6.3 (IQR 6–6.8) & PG3 6.7 (IQR: 6.7 – 7.1). 25% of our residents scored consistently above US national median ITE scores in all 3 years of training and 16% pursued a fellowship. Younger residents had higher aggregate annual CBE score than the program median (p < 0.05). Higher USMLE scores were associated with higher than US median ITE scores, reflecting exam-taking skills. Success in acquiring a fellowship was associated with consistent fellowship interest (p < 0.05) and research publications or presentations (p <0.05). None of the other characteristics including visa status were associated with the outcomes. Background IMG features namely, age and USMLE scores predict performance evaluation and in-training examination scores during residency training. In addition enhanced research activities during residency training could facilitate fellowship goals among interested IMGs.

/pdf/predicting-performance-using-background-characteristics-of-2ellpd198m.pdf

Predicting performance using background characteristics of international medical graduates in an inner-city university-affiliated Internal Medicine residency training program

Jane Akhuetie

Papers

Predicting performance using background characteristics of international medical graduates in an inner-city university-affiliated Internal Medicine residency training program