Can ChatGPT Aid Clinicians in Educating Patients on the Surgical Management of Glaucoma?
Authors:
- 24 November 2023
- journal article
- research article
- Published by Wolters Kluwer Health in Journal of Glaucoma
- Vol. 33 (2) , 94-100
Abstract
Précis: ChatGPT can help health care providers automate the quality assessment of online health information, but it does not produce easier-to-understand responses compared with existing online health information. Purpose: To compare the readability of ChatGPT-generated health information about glaucoma surgery to existing material online and to evaluate ChatGPT’s ability to analyze the quality of information found online about glaucoma surgery. Methods: ChatGPT was asked to create patient handouts on glaucoma surgery using 7 independent prompts, aiming to generate sixth grade level reading material. Existing patient-targeted online health information about glaucoma surgery was selected from the top 50 search results of 3 search engines, excluding advertisements, blog posts, information intended for health professionals, irrelevant content, and duplicate links. Four validated tools were used to assess readability, and the readability of the ChatGPT-generated material was compared with the readability of existing online information. The DISCERN instrument was used for the quality assessment of online materials. The DISCERN instrument was also programmed to use ChatGPT to evaluate its ability to analyze quality. R software and descriptive statistics were used for data analysis. Results: Thirty-five webpages were included. There was no difference between the reading level of online webpages (12th grade) and the reading level of ChatGPT-generated responses (11th grade), despite the ChatGPT prompts asking for simple language and a sixth grade reading level. The quality of health content was “fair,” with only 5 resources receiving an “excellent” score. ChatGPT scored the quality of health resources with high precision (r=0.725). Conclusions: Patient-targeted information on glaucoma surgery is beyond the reading level of the average patient, therefore at risk of not being understood, and is of subpar quality, per DISCERN tool scoring. ChatGPT did not generate documents at a lower reading level as prompted, but this tool can aid in automating the time-consuming and subjective process of quality assessment.Keywords
This publication has 28 references indexed in Scilit:
- Orthopaedic Patient Information on the World Wide WebJournal of Bone and Joint Surgery, 2016
- Access to Care and Use of the Internet to Search for Health Information: Results From the US National Health Interview SurveyJournal of Medical Internet Research, 2015
- Global Prevalence of Glaucoma and Projections of Glaucoma Burden through 2040Ophthalmology, 2014
- Readability of Patient Education Materials on the American Association for Surgery of Trauma WebsiteArchives of Trauma Research, 2014
- Low Health Literacy, Limited English Proficiency, and Health Status in Asians, Latinos, and Other Racial/Ethnic Groups in CaliforniaJournal of Health Communication, 2012
- Low Health Literacy and Health Outcomes: An Updated Systematic ReviewAnnals of Internal Medicine, 2011
- Trends in Annual Medicare Expenditures for Glaucoma Surgical Procedures From 1997 to 2006Archives of Ophthalmology (1950), 2009
- Literacy and health outcomesJournal of General Internal Medicine, 2004
- Assessing, Controlling, and Assuring the Quality of Medical Information on the InternetJAMA, 1997
- Literacy skills and communication methods of low-income older personsPatient Education and Counseling, 1995