Simulation-based evaluation of anaesthesia residents: optimising resource use in a competency-based assessment framework
- 3 December 2019
- journal article
- research article
- Published by BMJ in BMJ Simulation and Technology Enhanced Learning
- Vol. 6 (6) , 339-343
Abstract
Introduction Simulation training in anaesthesiology bridges the gap between theory and practice by allowing trainees to engage in high-stakes clinical training without jeopardising patient safety. However, implementing simulation-based assessments within an academic programme is highly resource intensive, and the optimal number of scenarios and faculty required for accurate competency-based assessment remains to be determined. Using a generalisability study methodology, we examine the structure of simulation-based assessment in regard to the minimal number of scenarios and faculty assessors required for optimal competency-based assessments. Methods Seventeen anaesthesiology residents each performed four simulations which were assessed by two expert raters. Generalisability analysis (G-analysis) was used to estimate the extent of variance attributable to (1) the scenarios, (2) the assessors and (3) the participants. The D-coefficient and the G-coefficient were used to determine accuracy targets and to predict the impact of adjusting the number of scenarios or faculty assessors. Results We showed that multivariate G-analysis can be used to estimate the number of simulations and raters required to optimise assessment. In this study, the optimal balance was obtained when four scenarios were assessed by two simulation experts. Conclusion Simulation-based assessment is becoming an increasingly important tool for assessing the competency of medical residents in conjunction with other assessment methods. G-analysis can be used to assist in planning for optimal resource use and cost-efficacy.Keywords
Funding Information
- Faculty of Health Sciences, Queen’s University (Post Graduate Medical Education-Special Purpose Gr)
This publication has 29 references indexed in Scilit:
- Generalizability Theory and Classical Test TheoryApplied Measurement in Education, 2010
- A Randomized Controlled Trial of the Impact of Simulation-Based Training on Resident Performance During a Simulated Obstetric Anesthesia EmergencySimulation in Healthcare: The Journal of the Society for Simulation in Healthcare, 2010
- Simulation Education in Anesthesia Training: A Case Report of Successful Resuscitation of Bupivacaine-Induced Cardiac Arrest Linked to Recent Simulation TrainingAnesthesia & Analgesia, 2008
- Performance of Residents and Anesthesiologists in a Simulation-based Skill AssessmentAnesthesiology, 2007
- The future vision of simulation in health careQuality and Safety in Health Care, 2004
- Simulation-Based Training in Anesthesia Crisis Resource Management (ACRM): A Decade of ExperienceSimulation & Gaming, 2001
- Anesthesia crisis resource management: Real-life simulation training in operating room crisesJournal of Clinical Anesthesia, 1995
- Maximizing Power in Generalizability Studies Under Budget ConstraintsJournal of Educational Statistics, 1993
- Generalizability TheoryEducational Measurement: Issues and Practice, 1992
- ANESTHESIA SIMULATION IN AN ACTUAL OPERATING ROOM ENVIRONMENTAnesthesiology, 1987