Reliability of the Interprofessional Collaborator Assessment Rubric (ICAR) in Multi Source Feedback (MSF) with post-graduate medical residents
Open Access
- 31 December 2014
- journal article
- research article
- Published by Springer Nature in BMC Medical Education
- Vol. 14 (1) , 1-9
Abstract
Increased attention on collaboration and teamwork competency development in medical education has raised the need for valid and reliable approaches to the assessment of collaboration competencies in post-graduate medical education. The purpose of this study was to evaluate the reliability of a modified Interprofessional Collaborator Assessment Rubric (ICAR) in a multi-source feedback (MSF) process for assessing post-graduate medical residents’ collaborator competencies. Post-graduate medical residents (n = 16) received ICAR assessments from three different rater groups (physicians, nurses and allied health professionals) over a four-week rotation. Internal consistency, inter-rater reliability, inter-group differences and relationship between rater characteristics and ICAR scores were analyzed using Cronbach’s alpha, one-way and two-way repeated measures ANOVA, and logistic regression. Missing data decreased from 13.1% using daily assessments to 8.8% utilizing an MSF process, p = .032. High internal consistency measures were demonstrated for overall ICAR scores (α = .981) and individual assessment domains within the ICAR (α = .881 to .963). There were no significant differences between scores of physician, nurse, and allied health raters on collaborator competencies (F2,5 = 1.225, p = .297, η2 = .016). Rater gender was the only significant factor influencing scores with female raters scoring residents significantly lower than male raters (6.12 v. 6.82; F1,5 = 7.184, p = .008, η 2 = .045). The study findings suggest that the use of the modified ICAR in a MSF assessment process could be a feasible and reliable assessment approach to providing formative feedback to post-graduate medical residents on collaborator competencies.Keywords
This publication has 35 references indexed in Scilit:
- Multisource feedback questionnaires in appraisal and for revalidation: a qualitative study in UK general practiceBritish Journal of General Practice, 2012
- Feasibility and Reliability of a Multisource Feedback Tool for Emergency Medicine ResidentsJournal of Graduate Medical Education, 2011
- Development and validation of the interprofessional collaborator assessment rubric ((ICAR))Journal of Interprofessional Care, 2011
- The Quality of Written Comments on Professional Behaviors in a Developmental Multisource Feedback ProgramAcademic Medicine, 2010
- Multisource Feedback in the Ambulatory SettingJournal of Graduate Medical Education, 2010
- From the eye of the nurses: 360-degree evaluation of residentsJournal of Continuing Education in the Health Professions, 2009
- Attitudes of health sciences students towards interprofessional teamwork and educationLearning in Health and Social Care, 2008
- Impact of a 360-degree Professionalism Assessment on Faculty Comfort and Skills in Feedback DeliveryJournal of General Internal Medicine, 2008
- Workplace-based assessment as an educational tool: AMEE Guide No. 31Medical Teacher, 2007
- Gender bias of Ohio physicians in the evaluation of the personal statements of residency applicantsAcademic Medicine, 1991