The Quest for Early Detection of Retinal Disease: 3D CycleGAN-based Translation of Optical Coherence Tomography into Confocal Microscopy
Preprint
Abstract
Optical coherence tomography (OCT) and confocal microscopy are pivotal in retinal imaging, offering distinct advantages and limitations. In vivo OCT offers rapid, non-invasive imaging but can suffer from clarity issues and motion artifacts, while ex vivo confocal microscopy, providing high-resolution, cellular-detailed color images, is invasive and raises ethical concerns. To bridge the benefits of both modalities, we propose a novel framework based on unsupervised 3D CycleGAN for translating unpaired in vivo OCT to ex vivo confocal microscopy images. This marks the first attempt to exploit the inherent 3D information of OCT and translate it into the rich, detailed color domain of confocal microscopy. We also introduce a unique dataset, OCT2Confocal, comprising mouse OCT and confocal retinal images, facilitating the development of and establishing a benchmark for cross-modal image translation research. Our model has been evaluated both quantitatively and qualitatively, achieving Fréchet Inception Distance (FID) scores of 0.766 and Kernel Inception Distance (KID) scores as low as 0.153, and leading subjective Mean Opinion Scores (MOS). Our model demonstrated superior image fidelity and quality with limited data over existing methods. Our approach effectively synthesizes color information from 3D confocal images, closely approximating target outcomes and suggesting enhanced potential for diagnostic and monitoring applications in ophthalmology.Keywords
All Related Versions
This publication has 40 references indexed in Scilit:
- Objective quality assessment of medical images and videos: review and challengesMultimedia Tools and Applications, 2024
- Multimodality Fusion Strategies in Eye Disease DiagnosisJournal of Imaging Informatics in Medicine, 2024
- Cross-modality image translation: CT image synthesis of MR brain images using multi generative network with perceptual supervisionComputer Methods and Programs in Biomedicine, 2023
- Medical image data augmentation: techniques, comparisons and interpretationsArtificial Intelligence Review, 2023
- Image synthesis: a review of methods, datasets, evaluation metrics, and future outlookArtificial Intelligence Review, 2023
- Image-to-image translation with Generative Adversarial Networks via retinal masks for realistic Optical Coherence Tomography imaging of Diabetic Macular Edema disordersBiomedical Signal Processing and Control, 2022
- Segmentation methods and morphometry of confocal microscopy imaged corneal epithelial cellsContact Lens and Anterior Eye, 2022
- Deep learning methods to generate synthetic CT from MRI in radiotherapy: A literature reviewPhysica Medica, 2021
- MedGAN: Medical image translation using GANsComputerized Medical Imaging and Graphics, 2020
- Adaptive-weighted bilateral filtering and other pre-processing techniques for optical coherence tomographyComputerized Medical Imaging and Graphics, 2014