Leveraging Large Language Models to Enhance Patient Educational Resources in Rhinology
- 28 May 2025
- journal article
- Published by SAGE Publications in Annals of Otology, Rhinology & Laryngology
- Vol. 134 (9) , 683-691
Abstract
Background: To compare the readability of patient education materials (PEMs) on rhinologic conditions and procedures from the American Rhinologic Society (ARS) with those generated by large language models (LLMs). Methods: Forty-one PEMs from the ARS were retrieved. Readability was assessed through the Flesch Kincaid Reading Ease (FKRE) and Flesch Kincaid Grade Level (FKGL), in which higher FKRE and lower FKGL scores indicate better readability. Three LLMs—ChatGPT 4.o, Google Gemini, and Microsoft Copilot—were then used to translate each ARS PEM to the recommended sixth-grade reading level. Readability scores were calculated and compared for each translated PEM. Results: A total of 164 PEMs were evaluated, including 123 generated by LLMs. The original ARS PEMs had a mean FKGL of 10.28, while AI-generated PEMs demonstrated significantly better readability, with a mean FKGL of 8.6 ( P < .0001). Among the AI platforms, Gemini was the most easily readable, reaching a mean FKGL of 7.5 and FKRE of 65.5. Conclusion: LLMs improved the readability of PEMs, potentially enhancing accessibility to medical information for diverse populations. Despite these findings, healthcare providers and patients should cautiously appraise LLM-generated content, particularly for rhinology conditions and procedures. Level of Evidence: N/A.Keywords
This publication has 31 references indexed in Scilit:
- Enhancing Health Literacy: Evaluating the Readability of Patient Handouts Revised by ChatGPT's Large Language ModelOtolaryngology–Head and Neck Surgery, 2024
- Can artificial intelligence ( AI ) educate your patient? A study to assess overall readability and pharmacists' perception of AI ‐generated patient education materialsJACCP: Journal of the American College of Clinical Pharmacy, 2024
- Exploring the Role of Artificial Intelligence Chatbots in Preoperative Counseling for Head and Neck Cancer SurgeryThe Laryngoscope, 2023
- CanChatGPThelp patients answer their otolaryngology questions?Laryngoscope Investigative Otolaryngology, 2023
- The utility of the ChatGPT artificial intelligence tool for patient education and enquiry in robotic radical prostatectomyInternational Urology and Nephrology, 2023
- Assessing health literacy in rhinologic patientsInternational Forum of Allergy & Rhinology, 2020
- Association Between Functional Health Literacy and Postoperative Recovery, Health Care Contacts, and Health-Related Quality of Life Among Patients Undergoing Day SurgeryJAMA Surgery, 2018
- Readability assessment of the American Rhinologic Society patient education materialsInternational Forum of Allergy & Rhinology, 2012
- Readability assessment of internet‐based patient education materials related to endoscopic sinus surgeryThe Laryngoscope, 2012
- Assessing Readability of Patient Education Materials: Current Role in OrthopaedicsClinical Orthopaedics and Related Research, 2010