Cognitive and Psychomotor Entrustable Professional Activities: Can Simulators Help Assess Competency in Trainees?
- 1 April 2016
- journal article
- research article
- Published by Wolters Kluwer Health in Clinical Orthopaedics and Related Research
- Vol. 474 (4) , 926-934
Abstract
Ion-based assessment of resident performance of entrustable professional activities reliable? (2) Is there evidence of important differences between Postgraduate Year (PGY)-1 and PGY-4 residents when performing simulated entrustable professional activities? Methods Three entrustable professional activities were chosen from a list of competencies: management of the patient for total knee arthroplasty (TKA); management of the patient with an intertrochanteric hip fracture; and management of the patient with an ankle fracture. Each assessment of entrustable professional activity was 40 minutes long with three components: preoperative management of a patient (history-taking, examination, image interpretation); performance of a technical procedure on a sawbones model; and postoperative management of a patient (postoperative orders, management of complications). Residents were assessed by six faculty members who used checklists based on a modified Delphi technique, an overall global rating scale as well as a previously validated global rating scale for the technical procedure component of each activity. Nine PGY-1 and nine PGY-4 residents participated in our simulated assessment. We assessed reliability by calculating the internal consistency of the mean global rating for each activity as well as the interrater reliability between the faculty assessment and blinded review of videotaped encounters. We sought evidence of a difference in performance between PGY-1 and PGY-4 residents on the overall global rating scale for each station of each entrustable professional activity. Results The reliability (Cronbach's α) for the hip fracture activity was 0.88, it was 0.89 for the ankle fracture activity, and it was 0.84 for the TKA activity. A strong correlation was seen between blinded observer video review and faculty scores (mean 0.87 [0.07], p < 0.001). For the hip fracture entrustable professional activity, the PGY-4 group had a higher mean global rating scale than the PGY-1 group for preoperative management (3.56 [0.5] versus 2.33 [0.5], p < 0.001), postoperative management (3.67 [0.5] versus 2.22 [0.7], p < 0.001), and technical procedures (3.11 [0.3] versus 3.67 [0.5], p = 0.015). For the TKA activity, the PGY-4 group scored higher for postoperative management (3.5 [0.8] versus 2.67 [0.5], p = 0.016) and technical procedures (3.22 [0.9] versus 2.22 [0.9], p = 0.04) than the PGY-1 group, but no difference for preoperative management with the numbers available (PGY-4, 3.44 [0.7] versus PGY-1 2.89 [0.8], p = 0.14). For the ankle fracture activity, the PGY-4 group scored higher for postoperative management (3.22 [0.8] versus 2.33 [0.7], p = 0.18) and technical procedures (3.22 [1.2] versus 2.0 [0.7], p = 0.018) than the PGY-1 groups, but no difference for preoperative management with the numbers available (PGY-4, 3.22 [0.8] versus PGY-1, 2.78 [0.7], p = 0.23). Conclusions The results of our study show that simulated assessment of entrustable professional activities may be used to determine the ability of a resident to perform professional tasks that are critical components of medical training. In this manner, educators can ensure that competent performance of these skills in the simulated setting occurs before actual practice with patients in the clinical setting....Keywords
Funding Information
- OMeGA (yhlg9369YH)
This publication has 38 references indexed in Scilit:
- The Development of Entrustable Professional Activities for Internal Medicine Residency TrainingAcademic Medicine, 2015
- Time to TrustAcademic Medicine, 2014
- Applying the Milestones in an Internal Medicine Residency Program CurriculumAcademic Medicine, 2013
- The Arthroscopic Surgical Skill Evaluation Tool (ASSET)The American Journal of Sports Medicine, 2013
- Transforming Primary Care Training—Patient-Centered Medical Home Entrustable Professional Activities for Internal Medicine ResidentsJournal of General Internal Medicine, 2012
- Innate Arthroscopic Skills in Medical Students and Variation in Learning CurvesJournal of Bone and Joint Surgery, 2011
- Fiberoptic Oral IntubationAnesthesiology, 2008
- From the Educational Bench to the Clinical Bedside: Translating the Dreyfus Developmental Model to the Learning of Clinical SkillsAcademic Medicine, 2008
- Introducing competency-based postgraduate medical education in the NetherlandsMedical Teacher, 2008
- A comparison of operative times in arthroscopic ACL reconstruction between orthopaedic faculty and residents: the financial impact of orthopaedic surgical training in the operating room.2001