Distinguishing the Uterine Artery, the Ureter, and Nerves in Laparoscopic Surgical Images Using Ensembles of Binary Semantic Segmentation Networks
Open Access
Abstract
Performing a minimally invasive surgery comes with a significant advantage regarding rehabilitating the patient after the operation. But it also causes difficulties, mainly for the surgeon or expert who performs the surgical intervention, since only visual information is available and they cannot use their tactile senses during keyhole surgeries. This is the case with laparoscopic hysterectomy since some organs are also difficult to distinguish based on visual information, making laparoscope-based hysterectomy challenging. In this paper, we propose a solution based on semantic segmentation, which can create pixel-accurate predictions of surgical images and differentiate the uterine arteries, ureters, and nerves. We trained three binary semantic segmentation models based on the U-Net architecture with the EfficientNet-b3 encoder; then, we developed two ensemble techniques that enhanced the segmentation performance. Our pixel-wise ensemble examines the segmentation map of the binary networks on the lowest level of pixels. The other algorithm developed is a region-based ensemble technique that takes this examination to a higher level and makes the ensemble based on every connected component detected by the binary segmentation networks. We also introduced and trained a classic multi-class semantic segmentation model as a reference and compared it to the ensemble-based approaches. We used 586 manually annotated images from 38 surgical videos for this research and published this dataset.Keywords
Funding Information
- National Research, Development, and Innovation Fund of Hungary (TKP2021-NKTA-3)
This publication has 34 references indexed in Scilit:
- Automated Cardiovascular Pathology Assessment Using Semantic Segmentation and Ensemble LearningJournal of Digital Imaging, 2020
- Ensemble learning: A surveyWIREs Data Mining and Knowledge Discovery, 2018
- Surgical approach to hysterectomy for benign gynaecological diseaseCochrane Database of Systematic Reviews, 2015
- Causes and prevention of laparoscopic ureter injuries: an analysis of 31 cases during laparoscopic hysterectomy in the NetherlandsSurgical Endoscopy, 2012
- An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemesPattern Recognition, 2011
- Comparison of laparoscopic and abdominal hysterectomy in terms of quality of life: A systematic reviewEuropean Journal of Obstetrics & Gynecology and Reproductive Biology, 2008
- Real-Time Intraoperative Ureteral Guidance Using Invisible Near-Infrared FluorescenceJournal of Urology, 2007
- HysterectomyBest Practice & Research Clinical Obstetrics & Gynaecology, 2006
- Assessments of laparoscopic-assisted vaginal hysterectomyThe Journal of the American Association of Gynecologic Laparoscopists, 1994
- Ureter injury during laparoscopy-assisted vaginal hysterectomy with the endoscopic linear staplerAmerican Journal of Obstetrics and Gynecology, 1992