$\text{MR}^{2}$-Net: Retinal OCTA Image Stitching via Multi-Scale Representation Learning and Dynamic Location Guidance
- 25 September 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 29 (1) , 482-494
Abstract
Optical coherence tomography angiography (OCTA) plays a crucial role in quantifying and analyzing retinal vascular diseases. However, the limited field of view (FOV) inherent in most commercial OCTA imaging systems poses a significant challenge for clinicians, restricting the possibility to analyze larger retinal regions of high resolution. Automatic stitching of OCTA scans in adjacent regions may provide a promising solution to extend the region of interest. However, commonly-used stitching algorithms face difficulties in achieving effective alignment due to noise, artifacts and dense vasculature present in OCTA images. To address these challenges, we propose a novel retinal OCTA image stitching network, named $\rm {MR}^{2}$ -Net, which integrates multi-scale representation learning and dynamic location guidance. In the first stage, an image registration network with a progressive multi-resolution feature fusion is proposed to derive deep semantic information effectively. Additionally, we introduce a dynamic guidance strategy to locate the foveal avascular zone (FAZ) and constrain registration errors in overlapping vascular regions. In the second stage, an image fusion network based on multiple mask constraints and adjacent image aggregation (AIA) strategies is developed to further eliminate the artifacts in the overlapping areas of stitched images, thereby achieving precise vessel alignment. To validate the effectiveness of our method, we conduct a series of experiments on two delicately constructed datasets, i.e., OPTOVUE-OCTA and SVision-OCTA. Experimental results demonstrate that our method outperforms other image stitching methods and effectively generates high-quality wide-field OCTA images, achieving a structural similarity index (SSIM) score of 0.8264 and 0.8014 on the two datasets, respectively.Keywords
Funding Information
- National Natural Science Foundation of China (62371442, 62103398, 62272444)
- Natural Science Foundation of Zhejiang Province (LZ23F010002, LR24F010002, LR22F020008, LQ23F010002)
- Natural Science Foundation of Ningbo Municipality (2022J143)
This publication has 38 references indexed in Scilit:
- NerveStitcher: Corneal confocal microscope images stitching with neural networksComputers in Biology and Medicine, 2022
- Learning edge-preserved image stitching from multi-scale deep homographyNeurocomputing, 2022
- Image stitching via deep homography estimationNeurocomputing, 2021
- A view-free image stitching network based on global homographyJournal of Visual Communication and Image Representation, 2020
- An Improved Image Stitching Method Based on Seed Region Growth and Poisson Fusion2020
- U2-Net: Going deeper with nested U-structure for salient object detectionPattern Recognition, 2020
- Using Deep Learning Models to Characterize Major Retinal Features on Color Fundus PhotographsOphthalmology, 2020
- Optical coherence tomography angiography: A comprehensive review of current methods and clinical applicationsProgress in Retinal and Eye Research, 2017
- Automatic Panoramic Image Stitching using Invariant FeaturesInternational Journal of Computer Vision, 2006
- SURF: Speeded Up Robust Features2006