Use of the Quality Improvement (QI) Knowledge Application Tool in Assessing Pediatric Resident QI Education
Authors:
- Eric W. GlissmeyerEGEric W. Glissmeyer
- Sonja I. ZinielSZSonja I. Ziniel
- James MosesJMJames Moses
- 1 June 2014
- journal article
- Published by Journal of Graduate Medical Education in Journal of Graduate Medical Education
- Vol. 6 (2) , 284-291
Abstract
Background: Assessing the effectiveness of quality improvement curricula is important to improving this area of resident education. Objective: To assess the ability of the Quality Improvement Knowledge Application Tool (QIKAT) to differentiate between residents who were provided instruction in QI and those who were not, when scored by individuals not involved in designing the QIKAT, its scoring rubric, or QI curriculum instruction. Methods: The QIKAT and a 9-item self-assessment of QI proficiency were administered to an intervention and a control group. The intervention was a longitudinal curriculum consisting of 8 hours of didactic QI training and 6 workshops providing just-in-time training for resident QI projects. Two uninvolved faculty scored the QIKAT. Results: A total of 33 residents in the intervention group and 27 in the control group completed the baseline and postcurriculum QIKAT and self-assessment. QIKAT mean intervention group scores were significantly higher than mean control group scores postcurriculum ( P < .001). Absolute QIKAT differences were small (of 15 points, intervention group improved from a mean score of 12.8 to 13.2). Interrater agreement as measured by kappa test was low (0.09). Baseline self-assessment showed no differences, and after instruction, the intervention group felt more proficient in QI knowledge than controls in 4 of 9 domains tested. Conclusions: The QIKAT detected a statistically significant improvement postintervention, but the absolute differences were small. Self-reported gain in QI knowledge and proficiency agreed with the results of the QIKAT. However, QIKAT limitations include poor interrater agreement and a scoring rubric that lacks specificity. Programs considering using QIKAT to assess curricula should understand these limitations.Keywords
This publication has 11 references indexed in Scilit:
- Quality improvement in medical education: current state and future directionsMedical Education, 2011
- Development and Preliminary Evaluation of a Practice-Based Learning and Improvement Tool for Assessing Resident Competence and Guiding Curriculum DevelopmentJournal of Graduate Medical Education, 2011
- Effect of a quality improvement curriculum on resident knowledge and skills in improvementQuality and Safety in Health Care, 2010
- Validation of a method to measure resident doctors’ reflections on quality improvementMedical Education, 2010
- Residentsʼ Engagement in Quality Improvement: A Systematic Review of the LiteratureAcademic Medicine, 2009
- Validity Evidence for an OSCE to Assess Competency in Systems-Based Practice and Practice-Based Learning and Improvement: A Preliminary InvestigationAcademic Medicine, 2008
- Effectiveness of Teaching Quality Improvement to CliniciansJAMA, 2007
- Validation of a Method for Assessing Resident Physicians’ Quality Improvement ProposalsJournal of General Internal Medicine, 2007
- An Experiential Interdisciplinary Quality Improvement Education InitiativeAmerican Journal of Medical Quality, 2006
- Teaching and assessing resident competence in practice-based learning and improvementJournal of General Internal Medicine, 2004