Numerical versus narrative: A comparison between methods to measure medical student performance during clinical clerkships
Authors:
- 28 August 2017
- journal article
- research article
- Published by Taylor & Francis in Medical Teacher
- Vol. 39 (11) , 1154-1158
Abstract
Background: Medical school evaluations typically rely on both language-based narrative descriptions and psychometrically converted numeric scores to convey performance to the grading committee. We evaluated inter-rater reliability and correlation of numeric versus narrative evaluations for students on their Neurology Clerkship. Design/Methods: 50 Neurology Clerkship in-training evaluation reports completed by their residents and faculty members at the University of Rochester School of Medicine were dissected into narrative and numeric components. 5 Clerkship grading committee members retrospectively gave new narrative scores (NNS) while blinded to original numeric scores (ONS). We calculated intra-class correlation coefficients (ICC) and their associated confidence intervals for the ONS and the NNS. In addition, we calculated the correlation between ONS and NNS. Results: The ICC was greater for the NNS (ICC = .88 (95% CI = .70–.94)) than the ONS (ICC = .62 (95% CI = .40–.77)) Pearson correlation coefficient showed that the ONS and NNS were highly correlated (r = .81). Conclusions: Narrative evaluations converted by a small group of experienced graders are at least as reliable as numeric scoring by individual evaluators. We could allow evaluators to focus their efforts on creating richer narrative of greater value to trainees.Keywords
This publication has 33 references indexed in Scilit:
- The Next GME Accreditation System — Rationale and BenefitsNew England Journal of Medicine, 2012
- Determining Need for Remediation Through Postrotation EvaluationsJournal of Graduate Medical Education, 2012
- The assessment of professional competence: building blocks for theory developmentBest Practice & Research Clinical Obstetrics & Gynaecology, 2010
- Toward Authentic Clinical Evaluation: Pitfalls in the Pursuit of CompetencyAcademic Medicine, 2010
- Assessment in Medical EducationNew England Journal of Medicine, 2007
- Merging views on assessmentMedical Education, 2004
- The Feasibility and Acceptability of Implementing Formal Evaluation Sessions and Using Descriptive Vocabulary to Assess Student Performance on a Clinical ClerkshipTeaching and Learning in Medicine, 2002
- Global Descriptive Evaluations Are More Responsive than Global Numeric Ratings in Detecting Studentsʼ Progress during the Inpatient Portion of an Internal Medicine ClerkshipAcademic Medicine, 2001
- Investing in descriptive evaluation: a vision for the future of assessmentMedical Teacher, 2000
- Do final grades reflect written qualitative evaluations of student performance?Teaching and Learning in Medicine, 1993