Robust fetoscopic mosaicking from deep learned flow fields
Open Access
- 3 May 2022
- journal article
- research article
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 17 (6) , 1125-1134
Abstract
Purpose: Fetoscopic laser photocoagulation is a minimally invasive procedure to treat twin-to-twin transfusion syndrome during pregnancy by stopping irregular blood flow in the placenta. Building an image mosaic of the placenta and its network of vessels could assist surgeons to navigate in the challenging fetoscopic environment during the procedure. Methodology: We propose a fetoscopic mosaicking approach by combining deep learning-based optical flow with robust estimation for filtering inconsistent motions that occurs due to floating particles and specularities. While the current state of the art for fetoscopic mosaicking relies on clearly visible vessels for registration, our approach overcomes this limitation by considering the motion of all consistent pixels within consecutive frames. We also overcome the challenges in applying off-the-shelf optical flow to fetoscopic mosaicking through the use of robust estimation and local refinement. Results: We compare our proposed method against the state-of-the-art vessel-based and optical flow-based image registration methods, and robust estimation alternatives. We also compare our proposed pipeline using different optical flow and robust estimation alternatives. Conclusions: Through analysis of our results, we show that our method outperforms both the vessel-based state of the art and LK, noticeably when vessels are either poorly visible or too thin to be reliably identified. Our approach is thus able to build consistent placental vessel mosaics in challenging cases where currently available alternatives fail.Keywords
Funding Information
- Wellcome Trust (203145Z/16/Z)
- Engineering and Physical Sciences Research Council (NS/A000027/1, WT101957)
- Petroleum Technology Development Fund
- Engineering and Physical Sciences Research Council (EP/R004080/1)
This publication has 26 references indexed in Scilit:
- FlowNet: Learning Optical Flow with Convolutional Networks2015
- Object scene flow for autonomous vehicles2015
- Optical flow modeling and computation: A surveyComputer Vision and Image Understanding, 2015
- Seeing 3D Chairs: Exemplar Part-Based 2D-3D Alignment Using a Large Dataset of CAD Models2014
- A Naturalistic Open Source Movie for Optical Flow Evaluation2012
- Twin-to-twin transfusion syndrome (TTTS)*Journal of Perinatal Medicine, 2010
- Minimal Solutions for Panoramic Stitching2007
- Residual Anastomoses After Fetoscopic Laser Surgery in Twin-to-Twin Transfusion Syndrome: Frequency, Associated Risks and OutcomePlacenta, 2007
- Bundle Adjustment — A Modern Synthesis2000
- Random sample consensusCommunications of the ACM, 1981