AI-driven qualitative skill assessment in laparoscopic training: a prospective observational study
Open Access
- 20 May 2026
- journal article
- Published by Springer Nature in npj Digital Medicine
- Vol. 9 (1)
Abstract
Qualitative assessment of surgical skills is central to proficiency-based training but remains difficult to scale due to its reliance on expert evaluation. While simulator-derived metrics offer objective data, they largely fail to capture qualitative aspects of performance. Artificial intelligence (AI)-based video analysis may help bridge this gap by enabling standardized and reproducible assessment. In this prospective observational study conducted during the 41st Annual Davos Surgical Course (2024), 50 junior surgical residents performed laparoscopic cholecystectomy on porcine simulation models. Procedures were video recorded, anonymized, segmented, and independently evaluated by expert raters and an AI-based model using the Global Operative Assessment of Laparoscopic Skills (GOALS). The AI demonstrated excellent test-retest reliability for the overall GOALS score (ICC 0.91) and good reliability across most domains. Agreement between AI and expert ratings was also excellent for the total score (ICC 0.92), with good to excellent concordance across individual domains and minimal bias. These findings suggest that AI-based video analysis provides reliable and reproducible qualitative assessment of laparoscopic skills and may facilitate scalable integration of qualitative evaluation into surgical training and other video-based clinical assessment settings.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Concordance of ChatGPT artificial intelligence decision-making in colorectal cancer multidisciplinary meetings: retrospective studyBJS Open, 2025
- Surgical Mentorship in the Era of Artificial IntelligenceIndian Journal of Surgery, 2024
- Be prepared! Impact of structured video-assisted coaching on performance in a simulated bleeding exercise during laparoscopic surgerySurgical Endoscopy, 2024
- The Impact of Unconscious Bias in Healthcare: How to Recognize and Mitigate ItThe Journal of Infectious Diseases, 2019
- A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability ResearchJournal of Chiropractic Medicine, 2016
- Exploring the impact of mental workload on rater-based assessmentsAdvances in Health Sciences Education, 2012
- A global assessment tool for evaluation of intraoperative laparoscopic skillsThe American Journal of Surgery, 2005
- Developing senior doctors as mentors: a form of continuing professional development. Report of an initiative to develop a network of senior doctors as mentors: 1994–99Medical Education, 2000
- Sample size and optimal designs for reliability studiesStatistics in Medicine, 1998
- THE SIGNED-RANK (WILCOXON) TESTThe Lancet, 1969