Proficiency-based training for robotic surgery: construct validity, workload, and expert levels for nine inanimate exercises
- 21 February 2012
- journal article
- research article
- Published by Springer Nature in Surgical Endoscopy
- Vol. 26 (6) , 1516-1521
Abstract
We previously developed nine inanimate training exercises as part of a comprehensive, proficiency-based robotic training curriculum that addressed 23 unique skills identified via task deconstruction of robotic operations. The purpose of this study was to evaluate construct validity, workload, and expert levels for the nine exercises. Expert robotic surgeons (n = 8, fellows and faculty) and novice trainees (n = 4, medical students) each performed three to five consecutive repetitions of nine previously reported exercises (five FLS models with or without modifications and four custom-made models). Each task was scored for time and accuracy using modified FLS metrics; task scores were normalized to a previously established (preliminary) proficiency level and a composite score equaled the sum of the nine normalized task scores. Questionnaires were administered regarding prior experience. After each exercise, participants completed a validated NASA-TLX Workload Scale to rate the mental, physical, temporal, performance, effort, and frustration levels of each task. Experts had performed 119 (range = 15–600) robotic operations; novices had observed ≤1 robotic operation. For all nine tasks and the composite score, experts achieved significantly better performance than novices (932 ± 67 vs. 618 ± 111, respectively; P < 0.001). No significant differences in workload between experts and novices were detected (32.9 ± 3.5 vs. 32.0 ± 9.1, respectively; n.s.). Importantly, frustration ratings were relatively low for both groups (4.0 ± 0.7 vs. 3.8 ± 1.6, n.s.). The mean performance of the eight experts was deemed suitable as a revised proficiency level for each task. Using objective performance metrics, all nine exercises demonstrated construct validity. Workload was similar between experts and novices and frustration was low for both groups. These data suggest that the nine structured exercises are suitable for proficiency-based robotic training.Keywords
This publication has 14 references indexed in Scilit:
- Two-year skill retention and certification exam performance after fundamentals of laparoscopic skills training and proficiency maintenanceSurgery, 2010
- Fundamentals of Laparoscopic Surgery simulator training to proficiency improves laparoscopic performance in the operating room—a randomized controlled trialThe American Journal of Surgery, 2010
- Robotic assistance improves intracorporeal suturing performance and safety in the operating room while decreasing operator workloadSurgical Endoscopy, 2009
- Implementation, Construct Validity, and Benefit of a Proficiency-Based Knot-Tying and Suturing CurriculumJournal of Surgical Education, 2008
- Proficiency-Based Training for Surgical SkillsSeminars in Colon and Rectal Surgery, 2008
- FLS simulator performance predicts intraoperative laparoscopic skillSurgical Endoscopy, 2007
- Design of a Proficiency-Based Skills Training Curriculum for the Fundamentals of Laparoscopic SurgerySurgical Innovation, 2007
- Psychomotor testing predicts rate of skill acquisition for proficiency-based laparoscopic skills trainingSurgery, 2006
- Proving the Value of Simulation in Laparoscopic SurgeryAnnals of Surgery, 2004
- Fundamental principles of validation, and reliability: rigorous science for the assessment of surgical education and trainingSurgical Endoscopy, 2003