Automated Brief Hospital Course Summarization in Cardiac Surgery Using a Lightweight Large Language Model–Based Framework: Development and Evaluation Study on the Medical Information Mart for Intensive Care-IV
Open Access
- 24 September 2026
- journal article
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 28, e90870
Abstract
Background: Physician documentation requirements are a known contributor to clinician burnout, with the manual creation of brief hospital course (BHC) summaries being particularly time-consuming. Automating BHC summarization may mitigate this workload and reduce documentation errors. However, current natural language processing (NLP) methods are often limited to single-document inputs, and large language models (LLMs) face privacy and deployment challenges. Furthermore, existing methods often require manual information extraction and struggle to maintain temporal accuracy. Objective: We developed and evaluated LiteMedDoc, a lightweight, locally deployable LLM-based framework to automatically generate BHC summaries without model fine-tuning. Our objective was to determine whether clinically useful clinical summaries could be generated under strict privacy and resource constraints, making automated summarization feasible in real-world hospital environments. Methods: LiteMedDoc is a modular pipeline built upon an 8-billion-parameter open-source LLM (Llama 3.1). It features 3 specialized modules: a static dynamic information hierarchy module to condense multisource inputs and structure clinical events chronologically; a similar document retrieval-augmented generation module that retrieves contextually relevant prior case summaries; and a self-adaptive feedback optimization module used offline for prompt optimization. All processing was performed locally without any model fine-tuning. We evaluated the framework on a retrospective cohort of 4538 coronary artery bypass grafting (CABG) surgery cases from the Medical Information Mart for Intensive Care (MIMIC)-IV database. A held-out test set of 403 cases was used to generate BHC summaries. The model-generated summaries were compared to reference BHCs using 8 standard NLP metrics covering lexical overlap (BLEU-4 [Bilingual Evaluation Understudy-4 gram] and ROUGE [Recall-Oriented Understudy for Gisting Evaluation]), semantic similarity (BERTScore and METEOR [Metric for Evaluation of Translation With Explicit Ordering]), and clinical relevance (AlignScore [Alignment Score] and MEDCON [Medical Concept Overlap]). Additionally, 15 cardiac surgeons conducted a clinical evaluation of a sample of model-generated summaries, rating them on completeness, correctness, readability, conciseness, and global quality using a 5-point Likert scale. Results: Without any model training, LiteMedDoc achieved strong performance across individual automated metrics, nearly matching a fine-tuned model and exceeding a 70-billion-parameter model on all metrics. Additionally, in a within-database cross-domain evaluation on lobectomy cases, the framework maintained encouraging performance after prompt adaptation and outperformed both the base model and the CABG-fine-tuned model. Surgeons rated the AI-generated summaries above the prespecified acceptability threshold across all domains (mean scores ≥3.0), specifically praising their structure and conciseness. Conclusions: By integrating 3 specialized modules, the proposed framework offers a practical, locally deployable solution for clinician-in-the-loop BHC draft generation under privacy and resource constraints, and holds promise for improving documentation efficiency and enhancing information continuity during care transitions.Keywords
All Related Versions
This publication has 44 references indexed in Scilit:
- Evaluating Hospital Course Summarization by an Electronic Health Record–Based Large Language ModelJAMA Network Open, 2025
- Advancing Toward Clinical Deployment of AI-Generated Discharge Summaries—Beyond the BenchJAMA Network Open, 2025
- Optimizing Discharge Summaries: A Survey of Inpatient Clinician Perspectives and the Path to StandardizationJournal of General Internal Medicine, 2025
- Physician- and Large Language Model–Generated Hospital Discharge SummariesJAMA Internal Medicine, 2025
- Artificial Intelligence–Generated Emergency Department Summaries and Hospital HandoffsJAMA Network Open, 2024
- Developing and Evaluating Large Language Model–Generated Emergency Medicine Handoff NotesJAMA Network Open, 2024
- Parameter-efficient fine-tuning large language model approach for hospital discharge paper summarizationApplied Soft Computing, 2024
- Harnessing the Power of Generative AI for Clinical Summaries: Perspectives From Emergency PhysiciansAnnals of Emergency Medicine, 2024
- Association of Changing Hospital Readmission Rates With Mortality Rates After Hospital DischargeJAMA, 2017
- Deficits in Communication and Information Transfer Between Hospital-Based and Primary Care PhysiciansJAMA, 2007