Validation of ChatGPT 3.5 as a Tool to Optimize Readability of Patient-facing Craniofacial Education Materials
Authors:
Open Access
- 1 February 2024
- journal article
- research article
- Published by Wolters Kluwer Health in Plastic and Reconstructive Surgery - Global Open
- Vol. 12 (2) , e5575
Abstract
Background: To address patient health literacy, the American Medical Association recommends that readability of patient education materials should not exceed a sixth grade reading level; the National Institutes of Health recommend no greater than an eigth-grade reading level. However, patient-facing materials in plastic surgery often remain at an above-recommended average reading level. The purpose of this study was to evaluate ChatGPT 3.5 as a tool for optimizing patient-facing craniofacial education materials.Methods: Eighteen patient-facing craniofacial education materials were evaluated for readability by a traditional calculator and ChatGPT 3.5. The resulting scores were compared. The original excerpts were then inputted to ChatGPT 3.5 and simplified by the artificial intelligence tool. The simplified excerpts were scored by the calculators.Results: The difference in scores for the original excerpts between the online calculator and ChatGPT 3.5 were not significant (P= 0.441). Additionally, the simplified excerpts’ scores were significantly lower than the originals (P < 0.001), and the mean of the simplified excerpts was 7.78, less than the maximum recommended 8.Conclusions: The use of ChatGPT 3.5 for simplification and readability analysis of patient-facing craniofacial materials is efficient and may help facilitate the conveyance of important health information. ChatGPT 3.5 rendered readability scores comparable to traditional readability calculators, in addition to excerpt-specific feedback. It was also able to simplify materials to the recommended grade levels. With human oversight, we validate this tool for readability analysis and simplification.Keywords
This publication has 22 references indexed in Scilit:
- Social Media: Is the Message Reaching the Plastic Surgery Audience?Plastic and Reconstructive Surgery, 2019
- Low health literacyThe Nurse Practitioner, 2018
- Online Patient Resources for Gender Affirmation SurgeryAnnals of Plastic Surgery, 2017
- The accessibility, readability, and quality of online resources for gender affirming surgeryJournal of Surgical Research, 2017
- Readability, complexity, and suitability of online resources for mastectomy and lumpectomyJournal of Surgical Research, 2017
- Effect of Health Literacy on Decision-Making Preferences among Medically Underserved PatientsMedical Decision Making, 2016
- Readability Assessment of Patient Information about Lymphedema and Its TreatmentPlastic and Reconstructive Surgery, 2016
- Assessment of online patient materials for breast reconstructionJournal of Surgical Research, 2015
- Readability Assessment of Online Patient Resources for Breast Augmentation SurgeryPlastic and Reconstructive Surgery, 2015
- Readability of Patient Education Materials on the American Association for Surgery of Trauma WebsiteArchives of Trauma Research, 2014