Comparative analysis of generic vision-language models in detecting and diagnosing inherited retinal diseases using fundus photographs
- 7 October 2025
- journal article
- research article
- Published by Springer Nature in Eye
- Vol. 39 (17) , 3187-3194
Abstract
Background To evaluate the clinical applicability of three generic Vision-Large-Language Models (VLLMs) — OpenAI’s GPT-4omni, GPT-4V(ision) and Google’s Gemini in detecting and diagnosing inherited retinal diseases (IRDs), using fundus photographs. Methods The head-to-head comparative study curated 60 ultra-widefield (UWF) fundus images of 30 IRD patients from the National University Hospital, Singapore. Additionally, ten normal, open-sourced UWF fundus images were included for comparison. The 70 fundus images were analysed by the three VLLMs using standardised prompts to generate descriptions of 10 specified retinal features and provide clinical insights. Each VLLM received 2100 scores for descriptions across ten features, rated by three blinded consultant-level graders using three-point scale (0 = poor, 1 = borderline, 2 = good). Clinical insights including disease detection, diagnosis and pathological gene inference evaluated against clinical ground-truth. Results GPT-4o achieved the highest mean quality score in feature description (1.64 [0.697], mean [SEM]), outperforming GPT-4V (1.57 [0.738]) and Gemini (1.46 [0.800]; both p < 0.001). All models demonstrated high detection accuracy (\(\ge\)81.4%), but Gemini incorrectly classified all normal fundus images as IRD. GPT-4omni (65.7%) outperformed GPT-4V (50%) and Gemini (60%) in diagnosis accuracy. Gene inference precision remained low (\(\le\)20.3%) across all models. High concordance was observed across all models between feature descriptions and diagnoses (\(\ge\)97.1%), between diagnoses and clinical recommendations (100%). Conclusions GPT-4omni and GPT-4V demonstrated promising potential in detecting IRDs from fundus photographs, with good feature extraction capabilities and high detection accuracy. Gemini struggled with misidentifying normal fundus images. All three VLLMs require further refinement to improve diagnostic accuracy and gene inference.
Keywords
Funding Information
- MOH | National Medical Research Council (MOH-CSASI22jul-0001)
This publication has 37 references indexed in Scilit:
- Inherited retinal diseases are the most common cause of blindness in the working-age population in AustraliaOphthalmic Genetics, 2021
- The Impact of Inherited Retinal Diseases in the Republic of Ireland (ROI) and the United Kingdom (UK) from a Cost-of-Illness PerspectiveClinical Ophthalmology, 2020
- Worldwide carrier frequency and genetic prevalence of autosomal recessive inherited retinal diseasesProceedings of the National Academy of Sciences, 2020
- Next-generation sequencing and its application in diagnosis of retinitis pigmentosaOphthalmic Genetics, 2019
- Accuracy of a deep convolutional neural network in detection of retinitis pigmentosa on ultrawide-field imagesPeerJ, 2019
- Peripheral Visual Fields in ABCA4 Stargardt Disease and Correlation With Disease Extent on Ultra-widefield Fundus AutofluorescenceAmerican Journal of Ophthalmology, 2017
- A comparison of the causes of blindness certifications in England and Wales in working age adults (16–64 years), 1999–2000 with 2009–2010BMJ Open, 2014
- The molecular basis of human retinal and vitreoretinal diseasesProgress in Retinal and Eye Research, 2010
- Retinitis pigmentosa and allied conditions today: a paradigm of translational researchGenome Medicine, 2010
- Disease expression in X-linked retinitis pigmentosa caused by a putative null mutation in the RPGR gene.1997