Fine-tuned large Language model for extracting newly identified acute brain infarcts based on computed tomography or magnetic resonance imaging reports
Open Access
- 2 June 2025
- journal article
- research article
- Published by Springer Nature in Emergency Radiology
- Vol. 32 (4) , 495-501
Abstract
Purpose: This study aimed to develop an automated early warning system using a large language model (LLM) to identify acute to subacute brain infarction from free-text computed tomography (CT) or magnetic resonance imaging (MRI) radiology reports. Methods: In this retrospective study, 5,573, 1,883, and 834 patients were included in the training (mean age, 67.5 ± 17.2 years; 2,831 males), validation (mean age, 61.5 ± 18.3 years; 994 males), and test (mean age, 66.5 ± 16.1 years; 488 males) datasets. An LLM (Japanese Bidirectional Encoder Representations from Transformers model) was fine-tuned to classify the CT and MRI reports into three groups (group 0, newly identified acute to subacute infarction; group 1, known acute to subacute infarction or old infarction; group 2, without infarction). The training and validation processes were repeated 15 times, and the best-performing model on the validation dataset was selected to further evaluate its performance on the test dataset. Results: The best fine-tuned model exhibited sensitivities of 0.891, 0.905, and 0.959 for groups 0, 1, and 2, respectively, in the test dataset. The macrosensitivity (the average of sensitivity for all groups) and accuracy were 0.918 and 0.923, respectively. The model’s performance in extracting newly identified acute brain infarcts was high, with an area under the receiver operating characteristic curve of 0.979 (95% confidence interval, 0.956–1.000). The average prediction time was 0.115 ± 0.037 s per patient. Conclusion: A fine-tuned LLM could extract newly identified acute to subacute brain infarcts based on CT or MRI findings with high performance.Keywords
Funding Information
- The University of Tokyo
This publication has 21 references indexed in Scilit:
- Efficacy of Fine-Tuned Large Language Model in CT Protocol Assignment as Clinical Decision-Supporting SystemJournal of Imaging Informatics in Medicine, 2025
- Classification of Interventional Radiology Reports into Technique Categories with a Fine-Tuned Large Language ModelJournal of Imaging Informatics in Medicine, 2024
- The Fine-Tuned Large Language Model for Extracting the Progressive Bone Metastasis from Unstructured Radiology ReportsJournal of Imaging Informatics in Medicine, 2024
- Automated classification of brain MRI reports using fine-tuned large language modelsNeuroradiology, 2024
- Fine-Tuned Large Language Model for Extracting Patients on Pretreatment for Lung Cancer from a Picture Archiving and Communication System Based on Radiological ReportsJournal of Imaging Informatics in Medicine, 2024
- Data set terminology of deep learning in medicine: a historical review and recommendationJapanese Journal of Radiology, 2024
- Preliminary assessment of automated radiology report generation with generative pre-trained transformers: comparing results to radiologist-generated reportsJapanese Journal of Radiology, 2023
- Management of Acute Ischemic StrokeCritical Care Medicine, 2020
- Acute Ischemic StrokeNew England Journal of Medicine, 2020
- Failure to Follow-Up Test Results for Ambulatory Patients: A Systematic ReviewJournal of General Internal Medicine, 2011