Convolutional Network With Twofold Feature Augmentation for Diabetic Retinopathy Recognition From Multi-Modal Images
- 2 December 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 25 (7) , 2686-2697
Abstract
Objective: With the scenario of limited labeled dataset, this paper introduces a deep learning-based approach that leverages Diabetic Retinopathy (DR) severity recognition performance using fundus images combined with wide-field swept-source optical coherence tomography angiography (SS-OCTA). Methods: The proposed architecture comprises a backbone convolutional network associated with a Twofold Feature Augmentation mechanism, namely TFA-Net. The former includes multiple convolution blocks extracting representational features at various scales. The latter is constructed in a two-stage manner, i.e., the utilization of weight-sharing convolution kernels and the deployment of a Reverse Cross-Attention (RCA) stream. Results: The proposed model achieves a Quadratic Weighted Kappa rate of 90.2% on the small-sized internal KHUMC dataset. The robustness of the RCA stream is also evaluated by the single-modal Messidor dataset, of which the obtained mean Accuracy (94.8%) and Area Under Receiver Operating Characteristic (99.4%) outperform those of the state-of-the-arts significantly. Conclusion: Utilizing a network strongly regularized at feature space to learn the amalgamation of different modalities is of proven effectiveness. Thanks to the widespread availability of multi-modal retinal imaging for each diabetes patient nowadays, such approach can reduce the heavy reliance on large quantity of labeled visual data. Significance: Our TFA-Net is able to coordinate hybrid information of fundus photos and wide-field SS-OCTA for exhaustively exploiting DR-oriented biomarkers. Moreover, the embedded feature-wise augmentation scheme can enrich generalization ability efficiently despite learning from small-scale labeled data.Keywords
Funding Information
- Ministry of Science and ICT
- Information Technology Research Center (IITP-2017-0-01 629)
- Institute for Information & Communications Technology Promotion (No. 2017-0-00 655)
- Iran Telecommunication Research Center (IITP-2020-0-01 489)
- Institute for Information & Communications Technology Planning & Evaluation (NRF-2016K1A3A7A03951968, NRF-2019R1A2C2090504)
This publication has 33 references indexed in Scilit:
- A deep convolutional neural network module that promotes competition of multiple-size filtersPattern Recognition, 2017
- Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus PhotographsJAMA, 2016
- Deep Residual Learning for Image Recognition2016
- Rethinking the Inception Architecture for Computer Vision2016
- Automated Quantification of Capillary Nonperfusion Using Optical Coherence Tomography Angiography in Diabetic RetinopathyJAMA Ophthalmology, 2016
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015
- Going deeper with convolutions2015
- Swept-Source OCT Angiography of the Retinal Vasculature Using Intensity Differentiation-based Optical Microangiography AlgorithmsOphthalmic Surgery, Lasers and Imaging Retina, 2014
- FEEDBACK ON A PUBLICLY DISTRIBUTED IMAGE DATABASE: THE MESSIDOR DATABASEImage Analysis and Stereology, 2014
- Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scalesOphthalmology, 2003