Multimodal deep learning for objective skill assessment in robot-assisted vesico-urethral anastomosis
Open Access
- 10 March 2026
- journal article
- research article
- Published by Springer Nature in Journal of Robotic Surgery
- Vol. 20 (1) , 337
Abstract
This study aimed to classify robot-assisted surgery (RAS) skill levels (inexperienced, competent, and experienced) during performance of vesico-urethral anastomosis (VUA) using multimodal physiological signals. We trained Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) models on data collected from 23 RAS (RAS trainees and experienced surgeons) performing two VUAs on animal tissue. The dataset included 116-channel electroencephalogram (EEG) and 20 eye-tracking signals recorded during two VUA subtasks: (1) needle grasping, positioning, and entry; and (2) needle driving with wrist rotation and suture pull-out. Skill levels were rated by three raters using the Robotic Anastomosis Competency Evaluation (RACE) tool. Hyperparameters of the models were tuned using grid search with group 4-fold cross-validation on 16 participants and final model performance was evaluated on data from 7 unseen (held-out test) participants, repeated over 10 iterations. Weighted F-scores for classifying skill level using EEG and eye-tracking data were 0.84 for subtask 1 and 0.89 for subtask 2. Using paired t-tests, high-density EEG (116 channels) significantly outperformed low-density EEG (32 channels) for subtask 1 ( p = 0.001), with no difference for subtask 2 ( p = 0.15). Adding eye-tracking data significantly improved classification for subtask 2 ( p = 0.001), but not for subtask 1 ( p = 0.5). Multimodal deep learning using EEG and eye-tracking data enabled objective classification of surgical skills during VUA. The benefits of high-density EEG and multimodal integration were task-dependent, underscoring the need to tailor assessment tools to the cognitive and sensorimotor demands of specific surgical subtasks.Keywords
Funding Information
- National Institute of Biomedical Imaging and Bioengineering (R01EB029398)
This publication has 32 references indexed in Scilit:
- Surgeon Performance Predicts Early Continence After Robot-Assisted Radical ProstatectomyJournal of Endourology, 2017
- On the benefits of using surface Laplacian (current source density) methodology in electrophysiologyInternational Journal of Psychophysiology, 2015
- Surgical Competency for Urethrovesical Anastomosis During Robot-assisted Radical Prostatectomy: Development and Validation of the Robotic Anastomosis Competency EvaluationUrology, 2015
- Eye tracking for skills assessment and training: a systematic reviewJournal of Surgical Research, 2014
- Global Evaluative Assessment of Robotic Skills: Validation of a Clinical Assessment Tool to Measure Robotic Surgical SkillsJournal of Urology, 2012
- Surgical Task and Skill Classification from Eye Tracking and Tool Motion in Minimally Invasive Surgery2010
- EEG and MEG coherence: Measures of functional connectivity at distinct spatial scales of neocortical dynamicsJournal of Neuroscience Methods, 2007
- Frontal midline theta and the error-related negativity: neurophysiological mechanisms of action regulationClinical Neurophysiology, 2004
- Objective laparoscopic skills assessments of surgical residents using Hidden Markov Models based on haptic information and tool/tissue interactions.2001
- The Measurement of Observer Agreement for Categorical DataBiometrics, 1977