Frequently asked questions on erectile dysfunction: evaluating artificial intelligence answers with expert mentorship
Authors:
- 7 May 2024
- journal article
- research article
- Published by Springer Nature in International Journal Of Impotence Research
- Vol. 37 (4) , 310-314
Abstract
The present study assessed the accuracy of artificiaI intelligence-generated responses to frequently asked questions on erectile dysfunction. A cross-sectional analysis involved 56 erectile dysfunction-related questions searched on Google, categorized into nine sections: causes, diagnosis, treatment options, treatment complications, protective measures, relationship with other illnesses, treatment costs, treatment with herbal agents, and appointments. Responses from ChatGPT 3.5, ChatGPT 4, and BARD were evaluated by two experienced urology experts using the F1 and global quality scores (GQS) for accuracy, relevance, and comprehensibility. ChatGPT 3.5 and ChatGPT 4 achieved higher GQS than BARD in categories such as causes (4.5 ± 0.54, 4.5 ± 0.51, 3.15 ± 1.01, respectively, p < 0.001), treatment options (4.35 ± 0.6, 4.5 ± 0.43, 2.71 ± 1.38, respectively, p < 0.001), protective measures (5.0 ± 0, 5.0 ± 0, 4 ± 0.5, respectively, p = 0.013), relationships with other illnesses (4.58 ± 0.58, 4.83 ± 0.25, 3.58 ± 0.8, respectively, p = 0.006), and treatment with herbal agents (3 ± 0.61, 3.33 ± 0.83, 1.8 ± 1.09, respectively, p = 0.043). F1 scores in categories: causes (1), diagnosis (0.857), treatment options (0.726), and protective measures (1), indicated their alignment with the guidelines. There was no significant difference between ChatGPT 3.5 and ChatGPT 4 regarding answer quality, but both outperformed BARD in the GQS. These results emphasize the need to continually enhance and validate AI-generated medical information, underscoring the importance of artificiaI intelligence systems in delivering reliable information on erectile dysfunction.Keywords
This publication has 17 references indexed in Scilit:
- Application of Artificial Intelligence to Patient-Targeted Health Information on Kidney Stone DiseaseJournal of Renal Nutrition, 2023
- Can ChatGPT, an Artificial Intelligence Language Model, Provide Accurate and High-quality Patient Information on Prostate Cancer?Urology, 2023
- Radiology artificial intelligence: a systematic review and evaluation of methods (RAISE)European Radiology, 2022
- On evaluation metrics for medical applications of artificial intelligenceScientific Reports, 2022
- Applications of artificial intelligence in the diagnosis and prediction of erectile dysfunction: a narrative reviewInternational Journal Of Impotence Research, 2022
- European Association of Urology Guidelines on Sexual and Reproductive Health—2021 Update: Male Sexual DysfunctionEuropean Urology, 2021
- Current and future applications of machine and deep learning in urology: a review of the literature on urolithiasis, renal cell carcinoma, and bladder and prostate cancerWorld Journal of Urology, 2019
- Intraoperative Registered Transrectal Ultrasound Guidance for Robot-Assisted Laparoscopic Radical ProstatectomyJournal of Urology, 2015
- A systematic analysis of performance measures for classification tasksInformation Processing & Management, 2009
- Lower Urinary Tract Symptoms and Erectile Dysfunction: Co-Morbidity or Typical “Aging Male” Symptoms? Results of the “Cologne Male Survey”European Urology, 2003