Evaluating the appropriateness and safety of generative AI in delivering lifestyle guidance for atrial fibrillation patients
Authors:
Open Access
- 29 December 2025
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 16 (1)
Abstract
Lifestyle factors play a major role in atrial fibrillation (AF) incidence, but the effectiveness of lifestyle counseling varies among individuals. Due to limited consultation time, physicians often provide only brief guidance, leaving patients to manage changes on their own. This study assessed the clinical utility of three Large Language Models (LLMs) for delivering accurate and personalized lifestyle guidance: (1) GPT-4o, (2) a retrieval-augmented model using a curated Q&A database (DB GPT), and (3) a modular RAG model retrieving evidence from PubMed (PubMed GPT). Sixty-six questions from 16 AF patients were categorized into exercise, diet, lifestyle, and other domains. Five experienced electrophysiologists independently evaluated LLM-generated lifestyle guidance and physician-provided counseling responses using ten dimensions. GPT-4o demonstrated a comparable level of scientific consensus to electrophysiologists, while achieving a lower error rate and significantly higher levels of specialized content, empathy, and helpfulness. DB GPT and PubMed GPT showed similar error rates, proportions of specialized content, empathy, and helpfulness compared to electrophysiologists, but exhibited strengths in specialized content in exercise-related and accuracy in diet-related dimensions. These findings suggest that integrating complementary model strengths may help develop safer and more reliable medical AI systems.Keywords
This publication has 35 references indexed in Scilit:
- Evaluating performance of large language models for atrial fibrillation management using different prompting strategies and languagesScientific Reports, 2025
- MedExpQA: Multilingual benchmarking of Large Language Models for Medical Question AnsweringArtificial Intelligence in Medicine, 2024
- Applying generative AI with retrieval augmented generation to summarize and extract key clinical information from electronic health recordsJournal of Biomedical Informatics, 2024
- Large language models encode clinical knowledgeNature, 2023
- Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media ForumJAMA Internal Medicine, 2023
- Appropriateness of Cardiovascular Disease Prevention Recommendations Obtained From a Popular Online Chat-Based Artificial Intelligence ModelJAMA, 2023
- Effectiveness of a chatbot for eating disorders prevention: A randomized clinical trialInternational Journal of Eating Disorders, 2021
- Obesity and the Risk of Incident, Post-Operative, and Post-Ablation Atrial FibrillationJACC: Clinical Electrophysiology, 2015
- A randomized study of the effects of exercise training on patients with atrial fibrillationAmerican Heart Journal, 2011
- Barriers to Providing Nutrition Counseling by Physicians: A Survey of Primary Care PractitionersPreventive Medicine, 1995