Readability and Appropriateness of Responses Generated by ChatGPT 3.5, Gemini 2.5, and Microsoft Copilot to Frequently Asked Questions About Central Serous Chorioretinopathy

Authors:
Publisher website
Google Scholar
Abstract
Background: This study aims to compare the readability levels and informational quality of responses generated by three different artificial intelligence (AI)-based text generation models in relation to Central Serous Chorioretinopathy (CSCR).Materials and Methods: A total of 40 questions pertaining to CSCR were formulated based on articles indexed in PubMed over the past ten years. These questions were submitted to three AI-based text generation tools ChatGPT 3.5, Gemini 2.5, and Microsoft Copilot. The resulting responses were analyzed for readability using five standard indices: Flesch Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG), Gunning Fog Index (GFOG), and Automated Readability Index (ARI). Sentence lengths and structural complexity were also assessed. Contentquality was independently evaluated and scored by two researchers using a standardized rubric.Results: Nor adequate readability. A statistically significant difference in FRES scores was observed among the models (p = 0.01). Similarly, none of the models met the acceptable readability standards across the other four indices, with all scores exceeding recommended limits indicating generally poor readability. Among the tools, Gemini yielded significantly higher quality scores compared to the others (p < .001), suggesting superior informational content. Conversely, Microsoft Copilot produced more concise outputs characterized by shorter and fewer sentencesone of the models achieved theoptimal FRES threshold (≥ 60) required f.Conclusions: The findings suggest that AI-generated responses regarding CSCR are often overly technical and may not be easily comprehensible to individuals without a medical background. Moreover, the study highlights variability among different AI models in terms of both readability and content quality. These results underscore the importance of critically evaluating AI-generated medical content prior to its dissemination for public or clinical use.
Funding Information
  • The authors received no financial support for the research, authorship, and/or publication of this article.

This publication has 25 references indexed in Scilit: