Artificial intelligence classifies surgical technical skills in simulated laparoscopy: a pilot study
- 21 April 2025
- journal article
- research article
- Published by Springer Nature in Surgical Endoscopy
- Vol. 39 (6) , 3592-3599
Abstract
Objective To develop a computer algorithm for the automatic classification of basic surgical skills in laparoscopy. Summary Background Data The ability to objectively assess the operative skills of trainees would be invaluable for the success of competency-based medical education. Although technical advancements in computer vision have resulted in promising clinical applications, they have not yet been utilized in surgical education. Methods A single-institution, prospective study involving faculty and trainee surgeons recruited to use a bench-top simulator in order to complete the “precision cutting” task from the Fundamentals of Laparoscopic Surgery. An artificial intelligence (AI) computer algorithm was developed based on a transformer neural network model to classify videos of laparoscopic tasks as either executed by an expert or a novice surgeon. Performance metrics were reported in line with the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis guidelines. The model was trained using fivefold cross-validation. The model’s performance was evaluated using sensitivity, specificity, positive predictive value, negative predictive value, accuracy, F1 score, and area under the curve (AUC). The results were averaged across the folds, and 95% confidence intervals were computed for each metric. ROC curves were plotted to visualize the model’s performance. Results The internal dataset comprised 135 videos from 46 participants recruited between 2022 and 2023. Among these, 30 participants (65.2%) were junior surgical residents or medical students, and 16 (34.8%) were board-certified surgeons with prior laparoscopic experience. Following cross-validation, the AI model achieved an accuracy of 0.867 in classifying between novice and expert groups based on video analysis, independent of task completion time. For single-image classification, the model achieved an accuracy of 0.57. Conclusion This proof-of-concept study serves as a pilot investigation into the application of AI for classifying surgical skills, demonstrating the utility of computer vision in automatically and objectively classifying surgical expertise. While the results show promise, further validation is necessary to establish its utility in routine surgical training and certification. By providing objective evaluations, this technology could support and enhance the role of human evaluators in surgical education.Keywords
This publication has 24 references indexed in Scilit:
- Artificial intelligence in surgeryNature Medicine, 2024
- Technical skill assessment in minimally invasive surgery using artificial intelligence: a systematic reviewSurgical Endoscopy, 2023
- Automatic assessment of performance in the FLS trainer using computer visionSurgical Endoscopy, 2023
- Flexible endoscopic incisional therapy for Zenker’s diverticulum (FEIT-Z) is an effective treatment for surgical failures or non-operative patientsSurgical Endoscopy, 2022
- Artificial Intelligence and Surgical Education: A Systematic Scoping Review of InterventionsJournal of Surgical Education, 2022
- The treatment-travel tradeoff of colorectal cancer careSurgery, 2021
- Gender differences in the acquisition of suturing skills with the da Vinci surgical systemJournal of the Formosan Medical Association, 2020
- Assessing gender bias in qualitative evaluations of surgical residentsThe American Journal of Surgery, 2018
- Artificial Intelligence in Surgery: Promises and PerilsAnnals of Surgery, 2018
- Development of a Model for Training and Evaluation of Laparoscopic Skills 11This work was supported by an educational grant from United States Surgical Corporation (Auto Suture Canada).The American Journal of Surgery, 1998