A Study of Large Language Models for Patient Information Extraction: Model Architecture, Fine-Tuning Strategy, and Multi-task Instruction Tuning
Authors:
- Xinyu DongXDXinyu Dong
- Yaoyun ZhangYZYaoyun Zhang
- Yonghui WuYonghui Wu
Preprint
Abstract
Natural language processing (NLP) is a key technology to extract important patient information from clinical narratives to support healthcare applications. The rapid development of large language models (LLMs) has revolutionized many NLP tasks in the clinical domain, yet their optimal use in patient information extraction tasks requires further exploration. This study examines LLMs' effectiveness in patient information extraction, focusing on LLM architectures, fine-tuning strategies, and multi-task instruction tuning techniques for developing robust and generalizable patient information extraction systems. This study aims to explore key concepts of using LLMs for clinical concept and relation extraction tasks, including: (1) encoder-only or decoder-only LLMs, (2) prompt-based parameter-efficient fine-tuning (PEFT) algorithms, and (3) multi-task instruction tuning on few-shot learning performance. We benchmarked a suite of LLMs, including encoder-based LLMs (BERT, GatorTron) and decoder-based LLMs (GatorTronGPT, Llama 3.1, GatorTronLlama), across five datasets. We compared traditional full-size fine-tuning and prompt-based PEFT. We explored a multi-task instruction tuning framework that combines both tasks across four datasets to evaluate the zero-shot and few-shot learning performance using the leave-one-dataset-out strategy.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- Matching patients to clinical trials with large language modelsNature Communications, 2024
- Identifying social determinants of health from clinical narratives: A study of performance, documentation ratio, and potential biasJournal of Biomedical Informatics, 2024
- Model tuning or prompt Tuning? a study of large language models for clinical concept and relation extractionJournal of Biomedical Informatics, 2024
- GPT understands, tooAI Open, 2024
- A study of generative large language model for medical research and healthcarenpj Digital Medicine, 2023
- Contextualized medication information extraction using Transformer-based deep learning architecturesJournal of Biomedical Informatics, 2023
- A large language model for electronic health recordsnpj Digital Medicine, 2022
- Information extraction from electronic medical documents: state of the art and future research directionsKnowledge and Information Systems, 2022
- BioBERT: a pre-trained biomedical language representation model for biomedical text miningBioinformatics, 2019
- 2017 Roadmap for Innovation—ACC Health Policy Statement on Healthcare Transformation in the Era of Digital Health, Big Data, and Precision HealthJACC, 2017