Using Standardized Patients to Assess Professionalism: A Generalizability Study
Authors:
- 8 October 2010
- journal article
- research article
- Published by Taylor & Francis in Teaching and Learning in Medicine
- Vol. 22 (4) , 274-279
Abstract
Background: Assessment of professionalism in undergraduate medical education is challenging. One approach that has not been well studied in this context is performance-based examinations. Purpose: This study sought to investigate the reliability of standardized patients’ scores of students’ professionalism in performance-based examinations. Methods: Twenty students were observed on 4 simulated cases involving professional challenges; 9 raters evaluated each encounter on 21 professionalism items. Correlational and multivariate generalizability (G) analyses were conducted. Results: G coefficients were .75, .53, and .68 for physicians, standardized patients (SPs), and lay raters, respectively. Composite G coefficient for all raters reached acceptable level of .86. Results indicated SP raters were more variable than other rater types in severity with which they rated students, although rank ordering of students was consistent among SPs. Conclusions: SPs’ ratings were less reliable and consistent than physician or lay ratings, although the SPs rank ordered students more consistently than the other rater types.Keywords
This publication has 11 references indexed in Scilit:
- APPLIED RESEARCH: Reliability and Validity of Checklists and Global Ratings by Standardized Students, Trained Raters, and Faculty Raters in an Objective Structured Teaching EnvironmentTeaching and Learning in Medicine, 2005
- Using a standardised patient assessment to measure professional attributesMedical Education, 2004
- Basing the Evaluation of Professionalism on Observable Behaviors: A Cautionary TaleAcademic Medicine, 2004
- An analysis of standardised patient checklist errors and their effect on student scoresMedical Education, 2003
- Assessing Professional BehaviorAcademic Medicine, 2002
- Using Standardized Patients to Assess Medical Studentsʼ ProfessionalismAcademic Medicine, 2001
- A Longitudinal Approach to Teaching and Assessing Professional Attitudes and Behaviors in Medical SchoolAcademic Medicine, 2001
- A Year-long Course for Third-year Students on Ethics, Professionalism, and CommunicationAcademic Medicine, 2001
- Generalizability Theory2001
- Impact of inaccuracies in standardized patient portrayal and reporting on physician performance during blinded clinic visitsTeaching and Learning in Medicine, 1997