Evaluating Large Language Model–Generated Clinical Summaries Through a Dual-Perspective Framework: Retrospective Observational Study
Open Access
- 10 February 2026
- journal article
- Published by JMIR Publications Inc. in JMIR AI
- Vol. 5, e85221
Abstract
Large language models (LLMs) are increasingly used by patients and families to interpret complex medical documentation, yet most evaluations focus only on clinician-judged accuracy. In this study, 50 pediatric cardiac intensive care unit notes were summarized using GPT-4o mini and reviewed by both physicians and parents, who rated readability, clinical fidelity, and helpfulness. There were important discrepancies between parents and clinicians in the realm of helpfulness, along with important insights by clinicians assessing clinical accuracy and parents assessing readability. This study highlights the need for dual-perspective frameworks that balance clinical precision with patient understanding.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Assessing the Accuracy and Reliability of Large Language Models in Psychiatry Using Standardized Multiple-Choice Questions: Cross-Sectional StudyJournal of Medical Internet Research, 2025
- Laypeople’s Use of and Attitudes Toward Large Language Models and Search Engines for Health Queries: Survey StudyJournal of Medical Internet Research, 2025
- GPT-4 assistance for improvement of physician performance on patient care tasks: a randomized controlled trialNature Medicine, 2025
- Testing and Evaluation of Health Care Applications of Large Language ModelsJAMA, 2025
- An evaluation framework for clinical use of large language models in patient interaction tasksNature Medicine, 2025
- Enhancing Health Care Communication With Large Language Models—The Role, Challenges, and Future DirectionsJAMA Network Open, 2024
- Efficient healthcare with large language models: optimizing clinical workflow and enhancing patient careJournal of the American Medical Informatics Association, 2024
- Large language models in medicineNature Medicine, 2023