Evaluating AI-generated patient education materials for endometrial cancer surgery: a comparative analysis of response quality, reliability, and readability between ChatGPT and DeepSeek models
Top Cited Papers
Open Access
- 26 August 2026
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Public Health
- Vol. 14, 1831988
Abstract
Purpose: This study aimed to evaluate and compare the quality, reliability, and readability of patient education materials on endometrial cancer surgery generated by ChatGPT (GPT-5) and DeepSeek (R1). Materials and methods: This cross-sectional study analyzed the responses generated by ChatGPT and DeepSeek to totally 41 questions covering four domains: surgical planning, preoperative evaluation, postoperative care, and long-term follow-up. Reliability was assessed through the DISCERN and EQIP instruments, quality was evaluated by the Global Quality Score (GQS), and readability was analyzed by the Flesch Reading Ease Score (FRES), Gunning Fog Index (GFI), and Flesch-Kincaid Grade Level (FKGL). Statistical comparisons were performed by using paired t -tests and Wilcoxon signed-rank tests. Results: The two large language models (LLMs) generated education materials of comparable quality, as reflected in GQS scores (median: DeepSeek vs. ChatGPT 5.00 vs. 4.67, p = 0.077). DeepSeek demonstrated statistically significantly higher reliability scores on both DISCERN and EQIP instruments (both p < 0.001). Readability scores (FRES, GFI) were similar between groups, while DeepSeek exhibited a higher FKGL (10.28 vs. 8.84, p < 0.001), indicating the greater text complexity. Subgroup analysis showed that DeepSeek performed better in terms of reliability in the postoperative care and long-term follow-up domains, while ChatGPT exhibited better readability in the surgical planning domain. Conclusion: Both DeepSeek and ChatGPT can generate patient education text drafts that are commendable in their structural coherence and linguistic clarity. DeepSeek demonstrates a significant advantage in information reliability, particularly excelling in postoperative and follow-up management content. ChatGPT shows a slight edge in the readability of surgical planning sections. However, the text readability of both models exceeds the general public's health literacy level. This indicates that large language models can only serve as auxiliary tools for generating patient education materials. Their outputs must undergo review by clinical experts and readability optimization to ensure both accuracy and comprehensibility of the information.Keywords
This publication has 41 references indexed in Scilit:
- Exploring Women's Experiences of Information Across Their Endometrial Cancer Diagnosis and Treatment: A Qualitative AnalysisPsycho-Oncology, 2026
- SUN‐SHINE Sarcoma Systematic Environmental Scan: Evaluation of the Readability, Understandability, and Actionability of Websites Supporting Patients Diagnosed With Sarcoma and Their CaregiversPsycho-Oncology, 2025
- Improving communication between healthcare professionals and patients living with advanced breast cancer: a global expert review and call-to-action for 2025–2035 (Goal 5)The Breast, 2025
- Large Language Model–Based Responses to Patients’ In-Basket MessagesJAMA Network Open, 2024
- ChatGPT and most frequent urological diseases: analysing the quality of information and potential risks for patientsWorld Journal of Urology, 2023
- Evaluation High‐Quality of Information from ChatGPT (Artificial Intelligence—Large Language Model) Artificial Intelligence on Shoulder Stabilization SurgeryArthroscopy, 2023
- Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media ForumJAMA Internal Medicine, 2023
- AI-Generated Medical Advice—GPT and BeyondJAMA, 2023
- Readability of influenza information online: Implications for consumer healthAmerican Journal of Infection Control, 2019
- Cancer-Related Information Seeking Among Cancer Survivors: Trends Over a Decade (2003–2013)Journal of Cancer Education, 2015