Critical Deficiency Ratings in Milestone Assessment: A Review and Case Study
Authors:
- 1 June 2017
- journal article
- review article
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 92 (6) , 820-826
Abstract
Purpose: The Accreditation Council for Graduate Medical Education (ACGME) requires programs to report learner progress using specialty-specific milestones. It is unclear how milestones can best identify critical deficiencies (CDs) in trainee performance. Specialties developed milestones independently of one another; not every specialty included CDs within milestones ratings. This study examined the proportion of ACGME milestone sets that include CD ratings, and describes one residency program’s experiences using CD ratings in assessment. Method: The authors reviewed ACGME milestones for all 99 specialties in November 2015, determining which rating scales contained CDs. The authors also reviewed three years of data (July 2012–June 2015) from the University of Cincinnati Medical Center (UCMC) internal medicine residency assessment system based on observable practice activities mapped to ACGME milestones. Data were analyzed by postgraduate year, assessor type, rotation, academic year, and core competency. The Mantel–Haenszel chi-square test was used to test for changes over time. Results: Specialties demonstrated heterogeneity in accounting for CDs in ACGME milestones, with 22% (22/99) of specialties having no language describing CDs in milestones assessment. Thirty-three percent (63/189) of UCMC internal medicine residents received at least one CD rating, with CDs accounting for 0.18% (668/364,728) of all assessment ratings. The authors identified CDs across multiple core competencies and rotations. Conclusions: Despite some specialties not accounting for CDs in milestone assessment, UCMC’s experience demonstrates that a significant proportion of residents may be rated as having a CD during training. Identification of CDs may allow programs to develop remediation and improvement plans.Keywords
This publication has 32 references indexed in Scilit:
- Remediation as a Road to CompetencyJournal of Craniofacial Surgery, 2016
- Remediation of problematic residents—A national surveyThe Laryngoscope, 2015
- Reflections on the First 2 Years of Milestone ImplementationJournal of Graduate Medical Education, 2015
- Can This Resident Be Saved? Identification and Early Intervention for Struggling ResidentsJournal of the American College of Surgeons, 2014
- The Next Accreditation System: Stakeholder Expectations and Dialogue with the CommunityJournal of Graduate Medical Education, 2012
- The Next GME Accreditation System — Rationale and BenefitsNew England Journal of Medicine, 2012
- Competency-Based Education and Training in Internal MedicineAnnals of Internal Medicine, 2010
- Guiding Principles for Resident Remediation: Recommendations of the CORD Remediation Task ForceAcademic Emergency Medicine, 2010
- Assessment in Medical EducationNew England Journal of Medicine, 2007
- National Survey of Internal Medicine Residency Program Directors Regarding Problem ResidentsJAMA, 2000