Return to sport decision support with large language models after lower limb ligament injury
Open Access
- 16 November 2026
- journal article
- Published by SAGE Publications in International Journal of Sports Science & Coaching
Abstract
Patients and clinicians increasingly use large language models (LLMs) for return-to-sport decisions, but their reliability and the effect of structured clinical context remain uncertain. In this pre-registered, blinded, multi-model study, GPT-5, Gemini 2.5 Pro, Claude Sonnet 4.5, and DeepSeek-V3.2-Exp answered 12 fictional anterior cruciate ligament reconstruction or lateral ankle sprain vignettes under instruction-only (Prompt+) and guideline-derived context (Context+) conditions across two runs, generating 192 outputs. Three blinded domain experts scored six binary criteria. The primary outcome was a correct clinical claim supported by a valid DOI or PMID. Context + did not improve the primary outcome compared with Prompt + (OR 0.41, 95% CI 0.16–1.05; p = 0.065). GPT-5 was the only model whose point estimate favoured Context+, whereas the other three models favoured Prompt+. Claude Sonnet 4.5 showed a higher majority-vote Hallucination rate under Context + (22.7% vs 62.5%; Holm-adjusted p = 0.014); however, inter-rater agreement for Hallucination was poor (Fleiss’ κ < 0 in both domains), so this finding should be interpreted cautiously. Citation validity declined under Context + in three of the four models. Structured guideline-derived context did not reliably improve return-to-sport decision support in the models and conditions tested, and its effects varied between models. LLMs intended for clinical decision support should therefore undergo task- and model-specific evaluation before deployment.This publication has 28 references indexed in Scilit:
- The TRIPOD-LLM reporting guideline for studies using large language modelsNature Medicine, 2025
- Diagnosis, treatment, and prevention of ankle sprains: Comparing free chatbot recommendations with clinical guidelinesFoot and Ankle Surgery, 2024
- Artificial Intelligence Large Language Models Address Anterior Cruciate Ligament Reconstruction: Superior Clarity and Completeness by Gemini Compared With ChatGPT‐4 in Response to American Academy of Orthopaedic Surgeons Clinical Practice GuidelinesArthroscopy, 2024
- Automation Bias and Assistive AIJAMA, 2023
- Large language models propagate race-based medicinenpj Digital Medicine, 2023
- Large language models encode clinical knowledgeNature, 2023
- Return to sport decisions after an acute lateral ankle sprain injury: introducing the PAASS framework—an international multidisciplinary consensusBritish Journal of Sports Medicine, 2021
- Criteria-Based Return to Sport Decision-Making Following Lateral Ankle Sprain Injury: a Systematic Review and Narrative SynthesisSports Medicine, 2019
- 2016 Consensus statement on return to sport from the First World Congress in Sports Physical Therapy, BernBritish Journal of Sports Medicine, 2016
- High agreement but low Kappa: I. the problems of two paradoxesJournal of Clinical Epidemiology, 1990