Point detection through multi-instance deep heatmap regression for sutures in endoscopy
Open Access
- 8 November 2021
- journal article
- research article
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 16 (12) , 2107-2117
Abstract
Purpose: Mitral valve repair is a complex minimally invasive surgery of the heart valve. In this context, suture detection from endoscopic images is a highly relevant task that provides quantitative information to analyse suturing patterns, assess prosthetic configurations and produce augmented reality visualisations. Facial or anatomical landmark detection tasks typically contain a fixed number of landmarks, and use regression or fixed heatmap-based approaches to localize the landmarks. However in endoscopy, there are a varying number of sutures in every image, and the sutures may occur at any location in the annulus, as they are not semantically unique. Method: In this work, we formulate the suture detection task as a multi-instance deep heatmap regression problem, to identify entry and exit points of sutures. We extend our previous work, and introduce the novel use of a 2D Gaussian layer followed by a differentiable 2D spatial Soft-Argmax layer to function as a local non-maximum suppression. Results: We present extensive experiments with multiple heatmap distribution functions and two variants of the proposed model. In the intra-operative domain, Variant 1 showed a mean $$F_1$$ of $$+ 0.0422$$ over the baseline. Similarly, in the simulator domain, Variant 1 showed a mean $$F_1$$ of $$+ 0.0865$$ over the baseline. Conclusion: The proposed model shows an improvement over the baseline in the intra-operative and the simulator domains. The data is made publicly available within the scope of the MICCAI AdaptOR2021 Challenge https://adaptor2021.github.io/, and the code at https://github.com/Cardio-AI/suture-detection-pytorch/.
Keywords
All Related Versions
Funding Information
- Deutsche Forschungsgemeinschaft (398787259)
- Klaus Tschira Stiftung (Informatics 4 Life)
This publication has 27 references indexed in Scilit:
- Detecting Anatomical Landmarks From Limited Medical Imaging Data Using Two-Stage Task-Oriented Deep Neural NetworksIEEE Transactions on Image Processing, 2017
- Human Pose Estimation via Convolutional Part Heatmap Regression2016
- Endoscopic feature tracking for augmented-reality assisted prosthesis selection in mitral valve repair2016
- Approaching human level facial landmark localization by deep learningImage and Vision Computing, 2016
- Deep Convolutional Encoder Networks for Multiple Sclerosis Lesion Segmentation2015
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Augmented Reality-Enhanced Endoscopic Images for Annuloplasty Ring Sizing2014
- A connexionist approach for robust and precise facial feature detection in complex scenes2005
- Mitral Valve Surgery Can Now Routinely Be Performed EndoscopicallyCirculation, 2003
- A new reconstructive operation for correction of mitral and tricuspid insufficiencyThe Journal of Thoracic and Cardiovascular Surgery, 1971