Comparative Analysis of LLMs in Dry Eye Syndrome Healthcare Information
Open Access
- 30 July 2025
- journal article
- research article
- Published by MDPI AG in Diagnostics
- Vol. 15 (15) , 1913
Abstract
Background/Objective: Dry eye syndrome affects 16 million Americans with USD 52 billion in annual healthcare costs. With large language models (LLMs) increasingly used for healthcare information, understanding their performance in delivering equitable dry eye guidance across diverse populations is critical. This study aims to evaluate and compare five major LLMs (Grok, ChatGPT, Gemini, Claude.ai, and Meta AI) regarding dry eye syndrome information delivery across different demographic groups. Methods: LLMs were queried using standardized prompts simulating a 62-year-old patient with dry eye symptoms across four demographic categories (White, Black, East Asian, and Hispanic males and females). Responses were analyzed for word count, readability, cultural sensitivity scores (0–3 scale), keyword coverage, and response times. Results: Significant variations existed across LLMs. Word counts ranged from 32 to 346 words, with Gemini being the most comprehensive (653.8 ± 96.2 words) and Claude.ai being the most concise (207.6 ± 10.8 words). Cultural sensitivity scores revealed Grok demonstrated highest awareness for minority populations (scoring 3 for Black and Hispanic demographics), while Meta AI showed minimal cultural tailoring (0.5 ± 0.5). All models recommended specialist consultation, but medical term coverage varied significantly. Response times ranged from 7.41 s (Meta AI) to 25.32 s (Gemini). Conclusions: While all LLMs provided appropriate referral recommendations, substantial disparities exist in cultural sensitivity, content depth, and information delivery across demographic groups. No LLM consistently addressed the full spectrum of dry eye causes across all demographics. These findings underscore the importance for physician oversight and standardization in AI-generated healthcare information to ensure equitable access and prevent care delays.Keywords
This publication has 24 references indexed in Scilit:
- Dry Eye Disease Management Via Technological Methods: A Systematic Review and Network Meta-analysisOphthalmology and Therapy, 2025
- Areas for improvement in the current care and treatment of dry eye diseaseExpert Review of Ophthalmology, 2025
- Availability and transparency of artificial intelligence models in radiology: a meta-research studyEuropean Radiology, 2025
- Large language models in medicineNature Medicine, 2023
- The CLEAR path: A framework for enhancing information literacy through prompt engineeringThe Journal of Academic Librarianship, 2023
- Dry eye disease: identification and therapeutic strategies for primary care clinicians and clinical specialistsAnnals of Medicine, 2022
- Assessing the Quality, Reliability, and Readability of Online Information on Dry Eye DiseaseCornea, 2022
- In-office thermal systems for the treatment of dry eye diseaseSurvey of Ophthalmology, 2022
- TFOS DEWS II Epidemiology ReportThe Ocular Surface, 2017
- Dry Eye Medication Use and ExpendituresCornea, 2012