Artificial intelligence in patient education: evaluating large language models for understanding rheumatology literature
Open Access
- 15 October 2025
- journal article
- digital health
- Published by Frontiers Media SA in Frontiers in Digital Health
- Vol. 7, 1623399
Abstract
Background: Inadequate health literacy hinders positive health outcomes, yet medical literature often exceeds the general population's comprehension level. While health authorities recommend patient materials be at a sixth-grade reading level, scientific articles typically require college-level proficiency. Large language models (LLMs) like ChatGPT show potential for simplifying complex text, possibly bridging this gap.Objective: This study evaluated the effectiveness of ChatGPT 4.0 in enhancing the readability of peer-reviewed rheumatology articles for layperson comprehension.Methods: Twelve open-access rheumatology articles authored by the senior investigators were included. Baseline readability was evaluated utilizing Flesch-Kincaid Grade Level (FKGL) and Simple Measure of Gobbledygook (SMOG) indices. Each article was processed by ChatGPT 4.0 with a prompt requesting simplification to a sixth-grade level. Two expert rheumatologists evaluated the generated summaries’ appropriateness (accuracy, absence of errors/omissions). Readability changes were analyzed using paired t-tests.Results: ChatGPT significantly improved readability (P < .0001), reducing the average reading level from approximately 15th grade (FKGL: 15.06, SMOG: 14.08) to 10th grade (FKGL: 10.52, SMOG: 9.48). The expert reviewers deemed the generated summaries appropriate and accurate. The average word count was significantly reduced from 3,517 to 446 words (P = 0.047).Conclusions: ChatGPT effectively lowered the reading complexity of specialized rheumatology literature, making it more accessible than the original publications. However, the achieved 10th-grade reading level still exceeds the recommended sixth-grade level for patient education materials. While LLMs are a promising tool, their output may require further refinement or expert review to meet optimal health literacy standards and ensure equitable patient understanding in rheumatology.Keywords
This publication has 26 references indexed in Scilit:
- Evaluation of Patient Information Provided by ChatGPT on Antimalarial Use in Systemic Lupus Erythematosus: Spanish Language TranslationInternational Journal of Rheumatic Diseases, 2025
- ChatGPT in education: global reactions to AI innovationsScientific Reports, 2023
- ChatGPT: these are not hallucinations – they’re fabrications and falsificationsSchizophrenia, 2023
- The impact of health literacy: associations with disease activity and medication prescription in patients with rheumatoid arthritisRheumatology, 2023
- Exploring discordance between Health Literacy Questionnaire scores of people with RMDs and assessment by treating health professionalsRheumatology, 2022
- Assessment of the Quality, Content, and Readability of Freely Available Online Information for Patients Regarding Diabetic RetinopathyJAMA Ophthalmology, 2019
- Health Literacy and Use and Trust in Health InformationJournal of Health Communication, 2018
- Is health literacy associated with greater medical care trust?International Journal for Quality in Health Care, 2018
- Health literacy and adherence to medical treatment in chronic and acute illness: A meta-analysisPatient Education and Counseling, 2016
- Do patients need to read research?: Knowledge is powerBMJ, 2003