Assessing the clinical support capabilities of ChatGPT 4o and ChatGPT 4o mini in managing lumbar disc herniation
Top Cited Papers
Open Access
- 22 January 2025
- journal article
- research article
- Published by Springer Nature in European Journal of Medical Research
- Vol. 30 (1) , 1-9
Abstract
This study evaluated and compared the clinical support capabilities of ChatGPT 4o and ChatGPT 4o mini in diagnosing and treating lumbar disc herniation (LDH) with radiculopathy. Twenty-one questions (across 5 categories) from NASS Clinical Guidelines were input into ChatGPT 4o and ChatGPT 4o mini. Five orthopedic surgeons assessed their responses using a 5-point Likert scale for accuracy and completeness, and a 7-point scale for reliability. Flesch Reading Ease scores were calculated to assess readability. Additionally, ChatGPT 4o analyzed lumbar images from 53 patients, comparing its recognizable agreement with orthopedic surgeons using Kappa values. Both models demonstrated strong clinical support capabilities with no significant differences in accuracy or reliability. However, ChatGPT 4o provided more comprehensive and consistent responses. The Flesch Reading Ease scores for both models indicated that their generated content was “very difficult to read,” potentially limiting patient accessibility. In evaluating lumbar disc herniation images, ChatGPT 4o achieved an overall accuracy of 0.81, with LDH recognition precision, recall, and F1 scores exceeding 0.80. The AUC was 0.80, and the Kappa value was 0.61, indicating moderate agreement between the model’s predictions and actual diagnoses, though with room for improvement. While both models are effective, ChatGPT 4o offers more comprehensive clinical responses, making it more suitable for high-integrity medical tasks. However, the difficulty in reading AI-generated content and occasional use of misleading terms, such as “tumor,” indicate a need for further improvements to reduce patient anxiety.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- Assessing ChatGPT’s theoretical knowledge and prescriptive accuracy in bacterial infections: a comparative study with infectious diseases residents and specialistsInfection, 2024
- Assessing ChatGPT’s Potential in HIV Prevention Communication: A Comprehensive Evaluation of Accuracy, Completeness, and InclusivityAIDS and Behavior, 2024
- Lumbar disc herniation with radiculopathy: a comparison of NASS guidelines and ChatGPTNorth American Spine Society Journal (NASSJ), 2024
- Performance and Consistency of ChatGPT‐4 Versus Otolaryngologists: A Clinical Case SeriesOtolaryngology–Head and Neck Surgery, 2024
- Lumbar disc herniation: Epidemiology, clinical and radiologic diagnosis WFNS spine committee recommendationsWorld Neurosurgery: X, 2024
- Thromboembolic prophylaxis in spine surgery: an analysis of ChatGPT recommendationsThe Spine Journal, 2023
- GPT-4: a new era of artificial intelligence in medicineIrish Journal of Medical Science (1971 -), 2023
- Lumbar Disc Herniation: Diagnosis and ManagementThe American Journal of Medicine, 2023
- An evidence-based clinical guideline for the diagnosis and treatment of lumbar disc herniation with radiculopathyThe Spine Journal, 2014
- Expenditures and Health Status Among Adults With Back and Neck ProblemsJAMA, 2008