Evaluating the validity of the selection measures used for the UK’s foundation medical training programme: a national cohort study
Open Access
Abstract
Objectives Currently relative performance at medical school (educational performance measure (EPM) decile), additional educational achievements and the score on a situational judgement test (SJT) are used to rank applicants to the UK Foundation Years postgraduate medical training programme. We sought to evaluate whether these three measures were predictive of subsequent successful completion of the programme, and thus were valid selection criteria. Methods Data were obtained from the UK Medical Education Database (UKMED) on 14 131 UK applicants to the foundation programme starting in 2013 and 2014. These data included training outcomes in the form of Annual Reviews of Competency Progression (ARCPs), which indicated whether the programme was successfully completed. The relationship between applicants’ performance on the three selection measures to the odds of successful programme completion were modelled. Results On univariable analyses, all three measures were associated with the odds of successful completion of the programme. Converting the SJT score to deciles to compare the effect sizes suggested that one decile increase in the EPM increased the odds of completing the programme by approximately 15%, whereas the equivalent value was 8% for the SJT scores. On multivariable analyses (with all three measures included in the model), these effects were only independently and statistically significant for EPM decile (OR 1.14, 95% CI 1.10 to 1.18, pConclusions The EPM decile and SJT scores may be effective selection measures for the foundation programme. However, educational achievements does not add value to the other two measures when predicting programme completion. Thus, its usefulness in this context is less clear. Moreover, our findings suggest that the weighting for the EPM decile score, relative to SJT performance, should be increased.Keywords
This publication has 16 references indexed in Scilit:
- The situational judgement test: a student’s worst nightmareAdvances in Medical Education and Practice, 2015
- Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100Medical Teacher, 2015
- Older doctors and progression through specialty training in the UK: a cohort analysis of General Medical Council dataBMJ Open, 2015
- Does a high ranking mean success in the Situational Judgement Test?The Clinical Teacher, 2015
- The predictive validity of selection for entry into postgraduate training in general practice: evidence from three longitudinal studiesBritish Journal of General Practice, 2013
- New machine-marked tests for selection into core medical training: evidence from two validation studiesClinical Medicine, 2009
- Subgroup Differences in Situational Judgment Test Performance: A Meta-AnalysisHuman Performance, 2008
- SITUATIONAL JUDGMENT TESTS, RESPONSE INSTRUCTIONS, AND VALIDITY: A META‐ANALYSISPersonnel Psychology, 2007
- The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings.Psychological Bulletin, 1998
- The criterion problem: 1917-1992.Journal of Applied Psychology, 1992