ChatGPT Provides Unsatisfactory Responses to Frequently Asked Questions Regarding Anterior Cruciate Ligament Reconstruction
Top Cited Papers
- 2 February 2024
- journal article
- research article
- Published by Wiley in Arthroscopy
- Vol. 40 (7) , 2067
Abstract
Purpose: To determine whether the free online artificial intelligence platform ChatGPT could accurately, adequately, and appropriately answer questions regarding anterior cruciate ligament (ACL) reconstruction surgery. Methods: A list of 10 questions about ACL surgery was created based on a review of frequently asked questions that appeared on websites of various orthopaedic institutions. Each question was separately entered into ChatGPT (version 3.5), and responses were recorded, scored, and graded independently by 3 authors. The reading level of the ChatGPT response was calculated using the WordCalc software package, and readability was assessed using the Flesch‐Kincaid grade level, Simple Measure of Gobbledygook index, Coleman‐Liau index, Gunning fog index, and automated readability index. Results: Of the 10 frequently asked questions entered into ChatGPT, 6 were deemed as unsatisfactory and requiring substantial clarification; 1, as adequate and requiring moderate clarification; 1, as adequate and requiring minor clarification; and 2, as satisfactory and requiring minimal clarification. The mean DISCERN score was 41 (inter‐rater reliability, 0.721), indicating the responses to the questions were average. According to the readability assessments, a full understanding of the ChatGPT responses required 13.4 years of education, which corresponds to the reading level of a college sophomore. Conclusions: Most of the ChatGPT‐generated responses were outdated and failed to provide an adequate foundation for patients’ understanding regarding their injury and treatment options. The reading level required to understand the responses was too advanced for some patients, leading to potential misunderstanding and misinterpretation of information. ChatGPT lacks the ability to differentiate and prioritize information that is presented to patients. Clinical Relevance: Recognizing the shortcomings in artificial intelligence platforms may equip surgeons to better set expectations and provide support for patients considering and preparing for ACL reconstruction.Keywords
Funding Information
- Arthrex
- Administration for Community Living
This publication has 88 references indexed in Scilit:
- Evaluation of the Quality of Information on the Internet Available to Patients Undergoing Cervical Spine SurgeryWorld Neurosurgery, 2014
- Graft Size and Patient Age Are Predictors of Early Revision After Anterior Cruciate Ligament Reconstruction With Hamstring AutograftArthroscopy, 2012
- Low Health Literacy and Health Outcomes: An Updated Systematic ReviewAnnals of Internal Medicine, 2011
- Characteristics of Opioid Prescriptions in 2009JAMA, 2011
- Assessing Readability of Patient Education Materials: Current Role in OrthopaedicsClinical Orthopaedics and Related Research, 2010
- Anterior Cruciate Ligament Reconstruction Improves Activity-Induced Pain in Comparison With Pain at Rest in Middle-Aged Patients With Significant Cartilage DegenerationThe American Journal of Sports Medicine, 2010
- Patient Satisfaction after Total Knee Arthroplasty: Who is Satisfied and Who is Not?Clinical Orthopaedics and Related Research, 2010
- The meaning and the measure of health literacyJournal of General Internal Medicine, 2006
- The long-term followup of primary anterior cruciate ligament repairThe American Journal of Sports Medicine, 1991
- Primary surgical treatment of anterior cruciate ligament rupturesThe American Journal of Sports Medicine, 1990