ChatGPT can yield valuable responses in the context of orthopaedic trauma surgery
Authors:
Open Access
- 17 June 2024
- journal article
- research article
- Published by Wiley in Journal of Experimental Orthopaedics
- Vol. 11 (3) , e12047
Abstract
Purpose: To assess the possibility of using Generative Pretrained Transformer (ChatGPT) specifically in the context of orthopaedic trauma surgery by questions posed to ChatGPT and to evaluate responses (correctness, completeness and adaptiveness) by orthopaedic trauma surgeons.Methods: ChatGPT (GPT‐4 of 12 May 2023) was asked to address 34 common orthopaedic trauma surgery‐related questions and generate responses suited to three target groups: patient, nonorthopaedic medical doctor and expert orthopaedic surgeon. Three orthopaedic trauma surgeons independently assessed ChatGPT's responses by using a three‐point response scale with a response range between 0 and 2, where a higher number indicates better performance (correctness, completeness and adaptiveness).Results: A total of 18 (52.9%) of all responses were assessed to be correct (2.0) for the patient target group, while 22 (64.7%) and 24 (70.5%) of the responses were determined to be correct for nonorthopaedic medical doctors and expert orthopaedic surgeons, respectively. Moreover, a total of 18 (52.9%), 25 (73.5%) and 28 (82.4%) of the responses were assessed to be complete (2.0) for patients, nonorthopaedic medical doctors and expert orthopaedic surgeons, respectively. The average adaptiveness was 1.93, 1.95 and 1.97 for patients, nonorthopaedic medical doctors and expert orthopaedic surgeons, respectively.Conclusion: The study results indicate that ChatGPT can yield valuable and overall correct responses in the context of orthopaedic trauma surgery across different target groups, which encompassed patients, nonorthopaedic medical surgeons and expert orthopaedic surgeons. The average correctness scores, completeness levels and adaptiveness values indicated the ability of ChatGPT to generate overall correct and complete responses adapted to the target group.Level of Evidence: Not applicable.Keywords
This publication has 10 references indexed in Scilit:
- Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media ForumJAMA Internal Medicine, 2023
- Chat generative pre-trained transformer (ChatGPT): potential implications for rheumatology practiceRheumatology International, 2023
- The Potential of GPT-4 as an AI-Powered Virtual Assistant for Surgeons Specialized in Joint ArthroplastyAnnals of Biomedical Engineering, 2023
- Sparks of Artificial General Intelligence: Early experiments with GPT-4arXiv, 2023
- Capabilities of GPT-4 on Medical Challenge ProblemsarXiv, 2023
- Using a Google Web Search Analysis to Assess the Utility of ChatGPT in Total Joint ArthroplastyThe Journal of Arthroplasty, 2023
- A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTarXiv, 2023
- Living evidence: a new approach to the appraisal of rapidly evolving musculoskeletal researchBritish Journal of Sports Medicine, 2022
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language ProcessingarXiv, 2021
- Integration of Advanced Health Technology Within the Healthcare System to Fight the Global Pandemic: Current Challenges and Future Opportunities2021