Deep neural networks are effective tools for assessing performance during surgical training
- 15 July 2021
- journal article
- research article
- Published by Springer Nature in Journal of Robotic Surgery
- Vol. 16 (3) , 559-562
Abstract
Surgical education courses and certification tests require human evaluators to assess performance. Deep neural network (DNN) methods include techniques for classifying the content of videos which may enable automated scoring of video performance. Researchers collected 254 videos of two simulation-based exercises performed by attending surgeons. The performance in each video was scored by experienced instructors and converted into three class labels—expert, intermediate, and novice. The videos were cut into 2227 10 s clips for training DNNs in the Google Video Intelligence AutoML service. The DNN models matched the classifications applied by human evaluators with 83.1% accuracy for the Ring & Rail exercise and 80.8% for the Suture Sponge exercise. DNN models trained on individual exercises delivered very good results (80 + % accuracy) in matching the classifications assigned by human instructors and may eventually be able to supplement or replace human evaluators.Keywords
This publication has 10 references indexed in Scilit:
- Motion2Vec: Semi-Supervised Representation Learning from Surgical Videos2020
- A model for predicting the GEARS score from virtual reality surgical simulator metricsSurgical Endoscopy, 2018
- A Comparison of Robotic Simulation Performance on Basic Virtual Reality Skills: Simulator Subjective Versus Objective Assessment ToolsJournal of Minimally Invasive Gynecology, 2017
- Recognizing Surgical Activities with Recurrent Neural Networks2016
- A Review of Training Research and Virtual Reality Simulators for the da Vinci Surgical SystemTeaching and Learning in Medicine, 2015
- A case study of trajectory transfer through non-rigid registration for a simplified suturing scenario2013
- Discriminative Segment Annotation in Weakly Labeled Video2013
- Global Evaluative Assessment of Robotic Skills: Validation of a Clinical Assessment Tool to Measure Robotic Surgical SkillsJournal of Urology, 2012
- Objective assessment of technical surgical skillsBritish Journal of Surgery, 2010
- Objective structured assessment of technical skill (OSATS) for surgical residentsBritish Journal of Surgery, 1997