The quest for early detection of retinal disease: 3D CycleGAN-based translation of optical coherence tomography into confocal microscopy
Open Access
- 16 December 2024
- journal article
- research article
- Published by Cambridge University Press (CUP) in Biological Imaging
- Vol. 4, e15
Abstract
Optical coherence tomography (OCT) and confocal microscopy are pivotal in retinal imaging, offering distinct advantages and limitations. In vivo OCT offers rapid, noninvasive imaging but can suffer from clarity issues and motion artifacts, while ex vivo confocal microscopy, providing high-resolution, cellular-detailed color images, is invasive and raises ethical concerns. To bridge the benefits of both modalities, we propose a novel framework based on unsupervised 3D CycleGAN for translating unpaired in vivo OCT to ex vivo confocal microscopy images. This marks the first attempt to exploit the inherent 3D information of OCT and translate it into the rich, detailed color domain of confocal microscopy. We also introduce a unique dataset, OCT2Confocal, comprising mouse OCT and confocal retinal images, facilitating the development of and establishing a benchmark for cross-modal image translation research. Our model has been evaluated both quantitatively and qualitatively, achieving Fréchet inception distance (FID) scores of 0.766 and Kernel Inception Distance (KID) scores as low as 0.153, and leading subjective mean opinion scores (MOS). Our model demonstrated superior image fidelity and quality with limited data over existing methods. Our approach effectively synthesizes color information from 3D confocal images, closely approximating target outcomes and suggesting enhanced potential for diagnostic and monitoring applications in ophthalmology.Keywords
All Related Versions
Funding Information
- China Scholarship Council
This publication has 60 references indexed in Scilit:
- Multimodality Fusion Strategies in Eye Disease DiagnosisJournal of Imaging Informatics in Medicine, 2024
- Eye diseases diagnosis using deep learning and multimodal medical eye imagingMultimedia Tools and Applications, 2023
- Medical image data augmentation: techniques, comparisons and interpretationsArtificial Intelligence Review, 2023
- Image synthesis: a review of methods, datasets, evaluation metrics, and future outlookArtificial Intelligence Review, 2023
- Conditional Generative Adversarial Networks for Metal Artifact Reduction in CT Images of the Ear2018
- Towards MR-Only Radiotherapy Treatment Planning: Synthetic CT Generation Using Multi-view Deep Convolutional Neural Networks2018
- Adversarial Sparse-View CBCT Artifact Reduction2018
- Efficient Active Learning for Image Classification and Segmentation Using a Sample Selection and Conditional Generative Adversarial Network2018
- Image Super Resolution Using Generative Adversarial Networks and Local Saliency Maps for Retinal Image Analysis2017
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015