Fine-Tuning LLMs on Small Medical Datasets: Text Classification and Normalization Effectiveness on Cardiology reports and Discharge records
Preprint
Abstract
We investigate the effectiveness of fine-tuning large language models (LLMs) on small medical datasets for text classification and named entity recognition tasks. Using a German cardiology report dataset and the i2b2 Smoking Challenge dataset, we demonstrate that fine-tuning small LLMs locally on limited training data can improve performance achieving comparable results to larger models. Our experiments show that fine-tuning improves performance on both tasks, with notable gains observed with as few as 200-300 training examples. Overall, the study highlights the potential of task-specific fine-tuning of LLMs for automating clinical workflows and efficiently extracting structured data from unstructured medical text.Keywords
All Related Versions
This publication has 5 references indexed in Scilit:
- Zero-Shot LLMs for Named Entity Recognition: Targeting Cardiac Function Indicators in German Clinical Texts2024
- Unlocking the Potential of Free Text in Electronic Health Records with Large Language Models (LLM): Enhancing Patient Safety and Consultation Interactions2024
- MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning2024
- BioBERT: a pre-trained biomedical language representation model for biomedical text miningBioinformatics, 2019
- Identifying Patient Smoking Status from Medical Discharge RecordsJournal of the American Medical Informatics Association, 2008