Quality of patient information on interstitial cystitis from artificial intelligence chatbots
- 13 October 2025
- journal article
- Published by Wiley in BJU International
- Vol. 138 (S2)
Abstract
Objective: To evaluate the quality (DISCERN), understandability and actionability (Patient Education Materials Assessment Tool for Printable Materials [PEMAT‐P]), readability (Flesch–Kincaid), and misinformation of patient‐facing information on interstitial cystitis generated by four publicly available artificial intelligence (AI) chatbots: ChatGPT‐4.0, Perplexity, ChatSonic, and Bing AI. Methods: A total of 10 queries derived from Google Trends and Hopkins Medicine content were submitted to each chatbot. Responses were evaluated by two blinded reviewers using validated tools: the DISCERN instrument (reliability/quality), PEMAT‐P (understandability/actionability), and Flesch–Kincaid Grade Level (readability). Word count and citation inclusion were also recorded. Results: Across chatbots, information quality was moderate with a median (interquartile range [IQR]) DISCERN score of 3/5 (2–3), with Perplexity performing best and Bing AI worst. Understandability was moderate (median [IQR] PEMAT‐P score 75% [66.7–83.3%]), highest for ChatSonic with Hopkins Medicine‐derived prompts and lowest for ChatGPT with Google Trends inputs. Actionability was consistently poor (median [IQR] score 40% [20–60%]), with ChatSonic performing best and Bing AI lowest. Responses averaged 256 words and college‐level readability (median [IQR] Flesch–Kincaid score 25.4 [20.89–28.50]) across all platforms, limiting accessibility. Misinformation was minimal across all platforms. Chatbots referencing clinically curated prompts (Hopkins Medicine) scored higher in understandability and completeness than those responding to public search trends. Conclusion: Artificial intelligence chatbots offer generally accurate and understandable information about interstitial cystitis but lack actionable guidance and generate content at reading levels above typical patient comprehension. Enhancing readability, actionability, and personalisation may increase their utility as adjunct tools for patient education in functional urology.Keywords
This publication has 28 references indexed in Scilit:
- Quality of Chatbot Information Related to Benign Prostatic HyperplasiaThe Prostate, 2024
- Effectiveness of the Medical Chatbot PROSCA to Inform Patients About Prostate Cancer: Results of a Randomized Controlled TrialEuropean Urology Open Science, 2024
- Readability Assessment of Patient Education Materials on Uro-oncological Diseases Using Automated MeasuresEuropean Urology Focus, 2024
- Urological Cancers and ChatGPT: Assessing the Quality of Information and Possible Risks for PatientsClinical Genitourinary Cancer, 2024
- Assessment of Artificial Intelligence Chatbot Responses to Top Searched Queries About CancerJAMA Oncology, 2023
- Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media ForumJAMA Internal Medicine, 2023
- Assessment of Patient Retention of Inpatient Care Information Post-HospitalizationThe Joint Commission Journal on Quality and Patient Safety, 2022
- The misdiagnosis of interstitial cystitis/bladder pain syndrome in a VA populationNeurourology and Urodynamics, 2019
- Development of the Patient Education Materials Assessment Tool (PEMAT): A new measure of understandability and actionability for print and audiovisual patient informationPatient Education and Counseling, 2014
- Interstitial Cystitis and Painful Bladder SyndromeJournal of Urology, 2007