Quality of AI-generated exercise awareness messages for older adults aligned with the ICFSR consensus: a comparative study of three LLMs
Authors:
- Yannick Laurent Tchenadoyo BayalaYBYannick Laurent Tchenadoyo Bayala
- Louis Sylvain Peng-Wendé OuedraogoLOLouis Sylvain Peng-Wendé Ouedraogo
- Abdoul Rahamane CisséACAbdoul Rahamane Cissé
- Fulgence KaboréFKFulgence Kaboré
- Aristide Relwendé YameogoAYAristide Relwendé Yameogo
- Aboubakar OuedraogoAOAboubakar Ouedraogo
- Yamyellé Enselme ZongoYZYamyellé Enselme Zongo
- Wendlassida Joelle Stéphanie Zabsonré TiendrébeogoWTWendlassida Joelle Stéphanie Zabsonré Tiendrébeogo
- Dieu-Donné OuedraogoDODieu-Donné Ouedraogo
Open Access
- 20 July 2026
- journal article
- research article
- Published by Springer Nature in European Review of Aging and Physical Activity
- Vol. 23 (1)
Abstract
Background: Large Language Models (LLMs) are emerging as potential tools for health communication and patient education. However, their ability to translate complex medical guidelines into accessible, safe, and accurate messages for older adults remains insufficiently evaluated. To assess the capacity of ChatGPT-4, Claude AI, and Deepseek to generate exercise awareness messages for older adults, aligned with international expert consensus. Materials and methods: This was a cross-sectional observational study conducted between April and August 2025. Using the ICFSR 2021 Global Consensus on optimal exercise recommendations as the gold standard, we evaluated messages generated by three LLMs for 13 common chronic conditions in older adults. A standardized prompt was used to generate messages addressing exercise prescription considerations, disease progression, and recommended modalities. Two independent expert evaluators; a geriatrician and a sports medicine physician assessed five dimensions using a 5-point Likert scale; accuracy, clarity, safety, behavioral relevance, and absence of fabrication. Readability was measured using Flesch-Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), and Simple Measure of Gobbledygook (SMOG). Inter-rater agreement was assessed using intraclass correlation coefficient (ICC). Results: Claude achieved the overall scores from both evaluators at 4.63 ± 0.55 and 4.60 ± 0.55, followed by ChatGPT-4 at 4.55 ± 0.50 and 4.48 ± 0.53 and Deepseek at 4.15 ± 0.87 and 4.17 ± 0.86. Inter-rater agreement was moderate was 0.668. Claude demonstrated accuracy scores at 4.58 ± 0.64, while ChatGPT-4 excelled in clarity with 5 ± 0. All models achieved perfect scores for absence of fabrication (5 ± 0). Readability indices revealed high complexity across all LLMs, with median FKGL values ranging from 10.84 to 10.93, corresponding to 10th-11th grade reading level, exceeding recommended levels for older populations. A significant correlation was found between Claude’s accuracy scores and FKGL ( r = 0.599, p = 0.030). Conclusion: LLMs, particularly Claude, ChatGPT-4 and Deepseek demonstrate strong potential for generating accurate, safe, and hallucination-free exercise awareness messages for older adults. However, readability remains above recommended levels, requiring optimization.Keywords
This publication has 17 references indexed in Scilit:
- Helpful, harmless, honest? Sociotechnical limits of AI alignment and safety through Reinforcement Learning from Human FeedbackEthics and Information Technology, 2025
- Current and future state of evaluation of large language models for medical summarization tasksnpj Health Systems, 2025
- Global consensus on optimal exercise recommendations for enhancing healthy longevity in older adults (ICFSR)The Journal of nutrition, health and aging, 2025
- Still Using Only ChatGPT? The Comparison of Five Different Artificial Intelligence Chatbots’ Answers to the Most Common Questions About Kidney StonesJournal of Endourology, 2024
- A framework for human evaluation of large language models in healthcare derived from literature reviewnpj Digital Medicine, 2024
- Large Language Models in Health Care: Charting a Path Toward Accurate, Explainable, and Secure AIJournal of General Internal Medicine, 2024
- International Exercise Recommendations in Older Adults (ICFSR): Expert Consensus GuidelinesThe Journal of nutrition, health and aging, 2021
- Global Incidence of Frailty and Prefrailty Among Community-Dwelling Older AdultsJAMA Network Open, 2019
- A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability ResearchJournal of Chiropractic Medicine, 2016
- Assessing readability formula differences with written health information materials: Application, results, and recommendationsResearch in Social and Administrative Pharmacy, 2013