Evaluating Competence by Design as a Large System Change Initiative: Readiness, Fidelity, and Outcomes
Authors:
- Andrew K. HallAHAndrew K. Hall
- Anna OswaldAOAnna Oswald
- Jason R. FrankJFJason R. Frank
- Tim DalsegTDTim Dalseg
- Warren J. CheungWCWarren J. Cheung
- Lara CookeLCLara Cooke
- Lisa GormanLGLisa Gorman
- Stacey BrzezinaSBStacey Brzezina
- Sinthiya SelvaratnamSSSinthiya Selvaratnam
- Natalie WagnerNWNatalie Wagner
- Stanley J. HamstraSHStanley J. Hamstra
- Elaine Van MelleEMElaine Van Melle
Open Access
- 6 February 2024
- journal article
- research article
- Published by Ubiquity Press, Ltd. in Perspectives on Medical Education
- Vol. 13 (1) , 95-107
Abstract
Program evaluation is an essential, but often neglected, activity in any transformational educational change. Competence by Design was a large-scale change initiative to implement a competency-based time-variable educational system in Canadian postgraduate medical education. A program evaluation strategy was an integral part of the build and implementation plan for CBD from the beginning, providing insights into implementation progress, challenges, unexpected outcomes, and impact. The Competence by Design program evaluation strategy was built upon a logic model and three pillars of evaluation: readiness to implement, fidelity and integrity of implementation, and outcomes of implementation. The program evaluation strategy harvested from both internally driven studies and those performed by partners and invested others. A dashboard for the program evaluation strategy was created to transparently display a real-time view of Competence by Design implementation and facilitate continuous adaptation and improvement. The findings of the program evaluation for Competence by Design drove changes to all aspects of the Competence by Design implementation, aided engagement of partners, supported change management, and deepened our understanding of the journey required for transformational educational change in a complex national postgraduate medical education system. The program evaluation strategy for Competence by Design provides a framework for program evaluation for any large-scale change in health professions education.Keywords
This publication has 54 references indexed in Scilit:
- PerspectiveAcademic Medicine, 2013
- The Next GME Accreditation System — Rationale and BenefitsNew England Journal of Medicine, 2012
- Ready for a Change? A Longitudinal Study of Antecedents, Consequences and Contingencies of Readiness for ChangeJournal of Change Management, 2012
- Health professionals for a new century: transforming education to strengthen health systems in an interdependent worldThe Lancet, 2010
- Competency-based medical education: theory to practiceMedical Teacher, 2010
- A Framework for Measuring Fidelity of Implementation: A Foundation for Shared Language and Accumulation of KnowledgeAmerican Journal of Evaluation, 2010
- Evaluating Obstetrical Residency Programs Using Patient OutcomesJAMA, 2009
- Operative Experience of Residents in US General Surgery ProgramsAnnals of Surgery, 2009
- Review: Conceptualization and Measurement of Organizational Readiness for ChangeMedical Care Research and Review, 2008
- Curricular Change in Medical SchoolsAcademic Medicine, 2000