Ahmet Murat Çörekci

Background. Large language models LLMs are increasingly used by students as informal learning tools; however, their reliability in solving school-level physics questions remains uncertain. Aim. This study evaluated the performance of three LLMs—ChatGPT-5, Gemini 2.5 Flash, and DeepSeek V3.1—in solving physics questions from Turkish university entrance examinations administered between 2016 and 2025, comprising the Yükseköğretime Geçiş Sınavı (YGS; Transition to Higher Education Examination) in 2016–2017 and the Temel Yeterlilik Testi (TYT; Basic Proficiency Test) in 2018–2025. Methods. A total of 84 publicly available physics questions—28 from the 2016–2017 YGS examinations and 56 from the 2018–2025 TYT examinations—were presented to each model under identical testing conditions using the same neutral student-style prompt, without revealing the examination source. Responses were scored for final-answer correctness using official answer keys. Correct-answer accuracyand net scores were calculated, and model performance was compared using Cochran’s Q test. Results. DeepSeek V3.1 achieved the highest numerical correct-answer accuracy at 78.6% (95%CI 68.7%–86.0%), followed by Gemini 2.5 Flash at 69.0% (95% CI 58.5%–77.9%) and ChatGPT-5 at 65.5% (95% CI 54.8%–74.8%). However, the difference in correct-answer performance among the three models was not statistically significant (Cochran’s Q = 4.732, df = 2, p = 0.094). Conclusion. LLMs may support physics learning and exam preparation, but their responses should be used critically and under teacher guidance rather than as substitutes for structured physics instruction.

Hamstring strain injury is a common sports-related injury with substantial recurrence risk, and large language models are increasingly used as sources of health information by athletes, coaches, and clinicians. This study evaluated the concordance of three contemporary large language models with the 2022 hamstring strain injury clinical practice guideline. Fourteen guideline-derived clinical questions were submitted to ChatGPT-5.2, Gemini 3 Pro, and DeepSeek-3.2v using identical single-turn prompts on January 17, 2026. Forty-two anonymized responses were independently scored by three blinded clinicians using a predefined, study-specific 5-point Clinical Practice Guideline Concordance Scoring Rubric. Inter-rater reliability was good to excellent across models. Mean concordance scores were high for ChatGPT-5.2, Gemini 3 Pro, and DeepSeek-3.2v, with no statistically significant between-model difference. No responses were rated discordant or potentially harmful. Lower concordance was observed in early-phase management and standardized outcome-measure domains. Contemporary large language models showed high overall concordance with the hamstring strain injury guideline, but variability in nuanced rehabilitation and return-to-sport domains supports the need for clinician oversight when AI-generated advice is used in sports rehabilitation contexts.

Background: Large language models (LLMs) are increasingly used by patients and caregivers to obtain medical information. In pediatric acute bacterial arthritis, non-guideline-concordant information may be clinically important because timely diagnosis and management are essential. This study evaluated the concordance of LLM-generated responses to parent-oriented reformulations of Pediatric Infectious Diseases Society/Infectious Diseases Society of America (PIDS/IDSA) guideline recommendations. Methods: In this exploratory cross-sectional benchmarking study, 27 PIDS/IDSA guideline-derived recommendations and good practice statements were reformulated into standardized parent-oriented prompts. The same prompts were submitted to GPT-5.4 Thinking, Gemini 3 Thinking and Claude 4.6 Sonnet through browser-based interfaces on April 12, 2026. Responses were anonymized and independently assessed by 3 blinded reviewers. Each response was classified as concordant or discordant; final classifications were determined by majority decision. Interrater agreement was assessed using Fleiss’ kappa, and model differences were evaluated using Cochran’s Q test. Results: Overall, 75 of 81 responses (92.6%) were concordant with PIDS/IDSA recommendations. Gemini 3 Thinking achieved concordance in 27/27 responses (100.0%), Claude 4.6 Sonnet in 25/27 (92.6%) and GPT-5.4 Thinking in 23/27 (85.2%). Cochran’s Q test showed no significant difference among models (Q = 4.800, df = 2, P = 0.091). No unsupported or hallucinated content was identified. Interrater agreement was moderate (κ = 0.580; 95% CI, 0.454–0.706; P < 0.001). Conclusions: LLMs showed high guideline concordance, but selected item-level discordance persisted. Because these models may have been trained on guideline-derived content, high concordance should not be equated with independent clinical reasoning. These tools may support caregiver-oriented education but should not replace clinician assessment or guideline-based care.

Parents increasingly rely on large language models (LLMs) to obtain pediatric health information; however, the accuracy, clinical appropriateness, and readability of AI-generated responses remain variable. This concern is particularly relevant for rickets, a preventable metabolic bone disease in which delayed recognition or inappropriate guidance may result in adverse outcomes. This study aimed to compare the content quality, clinical appropriateness, and readability of responses generated by contemporary LLMs to parent-oriented questions about rickets using structured, multidisciplinary expert evaluation. Twenty-two frequently asked parent-oriented questions regarding rickets were identified from authoritative patient education resources and categorized into four thematic domains. Each question was posed to three LLMs (GPT-5.1, DeepSeek V3.2, and Gemini 3 Pro) using a standardized parent-focused prompt. Responses were collected as single-turn outputs between November 16 and 20, 2025. All responses were anonymized and independently evaluated by four clinicians (two pediatricians and two orthopedic surgeons). Content quality was assessed using a modified Artificial Intelligence Evaluation Score for Common Patient Questions (AIES-CPQ; range 5–25) and the Global Quality Scale (GQS; range 1–5). Readability was analyzed using five established indices. Inter-model differences were assessed using the Friedman test with Bonferroni-adjusted Wilcoxon signed-rank post-hoc comparisons, and inter-rater reliability was evaluated using intraclass correlation coefficients. Significant differences were observed among models for both AIES-CPQ and GQS scores (p < 0.001). Gemini 3 Pro and DeepSeek V3.2 achieved higher overall content quality and educational scores compared with GPT-5.1, although their relative strengths varied across evaluation domains. DeepSeek V3.2 demonstrated higher inter-rater reliability, while Gemini 3 Pro generated more detailed but linguistically complex responses. Readability analysis revealed substantial variability across models, indicating a trade-off between informational depth and accessibility. LLM-generated responses to parent-oriented questions about rickets vary substantially in quality, clinical appropriateness, and readability. While newer-generation models provide higher-quality information, none demonstrate uniformly reliable performance across all domains. Structured, disease-specific evaluation frameworks combined with multidisciplinary expert oversight are essential before AI-generated content can be safely integrated into parent-facing pediatric education. The online version contains supplementary material available at 10.1186/s12887-026-06851-1.