Evaluating the complementary roles of an SJT and academic assessment for entry into clinical practice
Open Access
- 8 February 2017
- journal article
- research article
- Published by Springer Nature in Advances in Health Sciences Education
- Vol. 22 (2) , 401-413
Abstract
Although there is extensive evidence confirming the predictive validity of situational judgement tests (SJTs) in medical education, there remains a shortage of evidence for their predictive validity for performance of postgraduate trainees in their first role in clinical practice. Moreover, to date few researchers have empirically examined the complementary roles of academic and non-academic selection methods in predicting in-role performance. This is an important area of enquiry as despite it being common practice to use both types of methods within a selection system, there is currently no evidence that this approach translates into increased predictive validity of the selection system as a whole, over that achieved by the use of a single selection method. In this preliminary study, the majority of the range of scores achieved by successful applicants to the UK Foundation Programme provided a unique opportunity to address both of these areas of enquiry. Sampling targeted high (>80th percentile) and low (r = .31 and .28, respectively). The relationship was stronger between the SJT and in-role performance for the low scoring group (r = .33, high scoring group r = .11), and between academic performance and in-role performance for the high scoring group (r = .29, low scoring group r = .11). Trainees with low SJT scores were almost five times more likely to receive remedial action. Results indicate that an SJT for entry into trainee physicians’ first role in clinical practice has good predictive validity of supervisor-rated performance and incidence of remedial action. In addition, an SJT and a measure of academic performance appeared to be complementary to each other. These initial findings suggest that SJTs may be more predictive at the lower end of a scoring distribution, and academic attainment more predictive at the higher end.Keywords
This publication has 27 references indexed in Scilit:
- The ‘Dark Side’ and ‘Bright Side’ of Personality: When Too Much Conscientiousness and Too Little Anxiety Are Detrimental with Respect to the Acquisition of Medical Knowledge and SkillPLOS ONE, 2014
- Test Equating, Scaling, and Linking2014
- The remediation challenge: theoretical and methodological insights from a systematic reviewMedical Education, 2013
- Adjusting medical school admission: assessing interpersonal skills using situational judgement testsMedical Education, 2013
- The Effects of Coaching on Situational Judgment Tests in High‐stakes SelectionInternational Journal of Selection and Assessment, 2012
- The validity and incremental validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations for predicting job performance in advanced-level high-stakes selection.Journal of Applied Psychology, 2011
- Situational judgment tests: a review of recent researchPersonnel Review, 2008
- The Operational Validity of a Video-Based Situational Judgment Test for Medical College Admissions: Illustrating the Importance of Matching Predictor and Criterion Construct Domains.Journal of Applied Psychology, 2005
- Factors associated with success in medical school: systematic review of the literatureBMJ, 2002