Using borderline methods to compare passing standards for OSCEs at graduation across three medical schools
- 25 October 2007
- journal article
- research article
- Published by Wiley in Medical Education
- Vol. 41 (11) , 1024-1031
Abstract
Context Medical schools in the UK set their own graduating examinations and pass marks. In a previous study we examined the equivalence of passing standards using the Angoff standard‐setting method. To address the limitation this imposed on that work, we undertook further research using a standard‐setting method specifically designed for objective structured clinical examinations (OSCEs). Methods Six OSCE stations were incorporated into the graduating examinations of 3 of the medical schools that took part in the previous study. The borderline group method (BGM) or borderline regression method (BRM) was used to derive the pass marks for all stations in the OSCE. We compared passing standards at the 3 schools. We also compared the results within the schools with their previously generated Angoff pass marks. Results The pass marks derived using the BGM or BRM were consistent across 2 of the 3 schools, whereas the third school generated pass marks which were (with a single exception) much lower. Within‐school comparisons of pass marks revealed that in 2 schools the pass marks generally did not significantly differ using either method, but for 1 school the Angoff mark was consistently and significantly lower than the BRM. Discussion The pass marks set using the BGM or BRM were more consistent across 2 of the 3 medical schools than pass marks set using the Angoff method. However, 1 medical school set significantly different pass marks from the other 2 schools. Although this study is small, we conclude that passing standards at different medical schools cannot be guaranteed to be equivalent.Keywords
This publication has 14 references indexed in Scilit:
- Standard Setting for Clinical Competence at Graduation from Medical School: A Comparison of Passing Scores Across Five Medical SchoolsAdvances in Health Sciences Education, 2006
- Standard Setting in a Small Scale OSCE: A Comparison of the Modified Borderline-Group Method and the Borderline Regression MethodAdvances in Health Sciences Education, 2006
- Techniques for measuring clinical competence: objective structured clinical examinationsMedical Education, 2004
- Setting standards on educational testsMedical Education, 2003
- Comparison of a rational and an empirical standard setting procedure for an OSCEMedical Education, 2003
- Setting standards for an objective structured clinical examination: the borderline group method gains ground on AngoffMedical Education, 2001
- Using the Judgments of Physician Examiners in Setting the Standards for a National Multi-center High Stakes OSCEAdvances in Health Sciences Education, 1997
- The Consistency and Uncertainty in Examiners' Definitions of Pass/Fail Performance on Osce StationsEvaluation & the Health Professions, 1996
- Large-scale high-stakes testing with an OSCEAcademic Medicine, 1996
- Validating an objective structured clinical examination (OSCE) as a method for selecting foreign medical graduates for a pre-internship programAcademic Medicine, 1991