Metadata-enhanced contrastive learning from retinal optical coherence tomography images
Authors:
Preprint
Abstract
Deep learning has potential to automate screening, monitoring and grading of disease in medical images. Pretraining with contrastive learning enables models to extract robust and generalisable features from natural image datasets, facilitating label-efficient downstream image analysis. However, the direct application of conventional contrastive methods to medical datasets introduces two domain-specific issues. Firstly, several image transformations which have been shown to be crucial for effective contrastive learning do not translate from the natural image to the medical image domain. Secondly, the assumption made by conventional methods, that any two images are dissimilar, is systematically misleading in medical datasets depicting the same anatomy and disease. This is exacerbated in longitudinal image datasets that repeatedly image the same patient cohort to monitor their disease progression over time. In this paper we tackle these issues by extending conventional contrastive frameworks with a novel metadata-enhanced strategy. Our approach employs widely available patient metadata to approximate the true set of inter-image contrastive relationships. To this end we employ records for patient identity, eye position (i.e. left or right) and time series information. In experiments using two large longitudinal datasets containing 170,427 retinal OCT images of 7,912 patients with age-related macular degeneration (AMD), we evaluate the utility of using metadata to incorporate the temporal dynamics of disease progression into pretraining. Our metadata-enhanced approach outperforms both standard contrastive methods and a retinal image foundation model in five out of six image-level downstream tasks related to AMD. Due to its modularity, our method can be quickly and cost-effectively tested to establish the potential benefits of including available metadata in contrastive pretraining.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- A foundation model for generalizable disease detection from retinal imagesNature, 2023
- Self supervised contrastive learning for digital histopathologyMachine Learning with Applications, 2022
- Self-Supervised Feature Learning and Phenotyping for Assessing Age-Related Macular Degeneration Using Retinal Fundus ImagesOphthalmology Retina, 2021
- Structured layer surface segmentation for retina OCT using fully convolutional regression networksMedical Image Analysis, 2021
- Self-supervised learning for medical image analysis using image context restorationMedical Image Analysis, 2019
- Clinically applicable deep learning for diagnosis and referral in retinal diseaseNature Medicine, 2018
- Deep Learning Is Effective for Classifying Normal versus Age-Related Macular Degeneration OCT ImagesOphthalmology Retina, 2017
- Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles2016
- Colorful Image Colorization2016
- Global prevalence of age-related macular degeneration and disease burden projection for 2020 and 2040: a systematic review and meta-analysisThe Lancet Global Health, 2014