Improving Medical Student Leadership Evaluation Through Standard Setting
- 21 September 2026
- journal article
- research article
- Published by Wolters Kluwer Health in Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare
Abstract
Introduction: Physician trainees are increasingly expected to demonstrate leadership in complex, team-based clinical settings. However, assessing leadership performance in dynamic environments like simulation remains challenging, with limited consensus on valid, reliable standards. While some observational frameworks exist, few offer defensible methods for determining minimum passing thresholds, particularly in high-stakes contexts. Traditional test-centered approaches may not capture the behavioral complexity of leadership, whereas examinee-centered methods offer more context-sensitive evaluations. This study compares test-centered and examinee-centered standard-setting approaches to identify a fair, reproducible method for assessing leadership in simulation, offering a model that can be applied across medical education programs. Methods: This study had 3 phases: first, the extended Angoff method (a test-centered method) was used to identify characteristics of borderline performers to set a cut-score; then we used the borderline group method (an examinee-centered method) to identify the cut-score; and finally, the cut-score was used to identify failing performance in a new data set. Results: The expert panel in phase 1 determined 10.34 as the cut-score; therefore, a score of ≥11 would be required to pass the evaluation. Analyses conducted in phase 2 suggested that 13 should be the cut-score. The mean and SD for phase 3 were similar to those of phase 2, lending support for the cut score of 13. Conclusions: Setting a cut-score is essential to accurately evaluating student performance when observations and ratings are more subjective. Ultimately, we sought to compare the test-centered versus examinee-centered methods and conclude that the examinee-centered method was a more holistic, fair, and reliable approach to student assessment.Keywords
Funding Information
- None
This publication has 18 references indexed in Scilit:
- Gathering Validity Evidence for the Bushmaster Assessment ToolMilitary Medicine, 2024
- Students’ Leadership Development During a High-fidelity Military Medical Field PracticumMilitary Medicine, 2023
- Operation Bushmaster: The Impact of Simulation-Based Education on Preparing the Next Generation of Military Medical OfficersMilitary Medicine, 2023
- Undergraduate Medical Education Leader Performance Predicts Postgraduate Military Leader PerformanceMilitary Medicine, 2023
- Faculty Assessments in a Military Medical Field Practicum: Rater Experience and Gender Do Not Appear to Influence ScoringMilitary Medicine, 2020
- Medical Student Leader Performance in an Applied Medical Field PracticumMilitary Medicine, 2019
- A Comparison of Approaches for Mastery Learning Standard SettingAcademic Medicine, 2018
- Teamwork in healthcare: Key discoveries enabling safer, high-quality care.American Psychologist, 2018
- What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessmentAdvances in Health Sciences Education, 2013
- Undesired variance due to examiner stringency/leniency effect in communication skill scores assessed in OSCEsAdvances in Health Sciences Education, 2007