Abstract
This study addresses the challenge of achieving accurate and real-time surgical instrument instance segmentation under strict computational constraints in minimally invasive surgery by proposing EfficientEndoSeg, a compact and high-throughput segmentation framework coupled with tool-tip analysis. Building on the You Only Look Once (YOLO) version 5 instance segmentation baseline, EfficientEndoSeg incorporates four targeted architectural refinements: the C3SG2 module combining GhostNetV2 and Swin Transformer components to enhance local–global representation, Ghost Convolution for parameter-efficient feature extraction, a Double-Attention (A2) block for contextual refinement, and Complete Intersection-over-Union–guided Soft Non-Maximum Suppression to improve robustness in crowded surgical scenes. The framework is systematically evaluated on an internally annotated extension of the m2cai16-tool dataset, comprising 5,249 endoscopic frames and 8,785 polygon-level annotations across seven instrument categories, which serves as a controlled benchmark for experimental validation. These architectural modifications reduce model parameters by 25% and increase inference throughput by 52.7%, while consistently improving recall and mean Average Precision across multiple Intersection-over-Union thresholds. The proposed model matches or surpasses recent YOLO version 7 to version 12 variants and remains competitive with transformer-based and two-stage frameworks, including SegFormer, ISINet, LETNet, Detectron2, and Mask R-CNN. In addition, a training-free tool-tip tracking module based on modified K3M skeletonization enables trajectory, heatmap, and usage analyses, facilitating qualitative skill-related motion assessment without additional learning overhead. Overall, EfficientEndoSeg delivers a reproducible, lightweight, and deployment-ready solution for segmentation-driven surgical scene understanding and instrument motion analysis.