AlpaPICO: Extraction of PICO Frames from Clinical Trial Documents Using LLMs
Preprint
Abstract
In recent years, there has been a surge in the publication of clinical trial reports, making it challenging to conduct systematic reviews. Automatically extracting Population, Intervention, Comparator, and Outcome (PICO) from clinical trial studies can alleviate the traditionally time-consuming process of manually scrutinizing systematic reviews. Existing approaches of PICO frame extraction involves supervised approach that relies on the existence of manually annotated data points in the form of BIO label tagging. Recent approaches, such as In-Context Learning (ICL), which has been shown to be effective for a number of downstream NLP tasks, require the use of labeled examples. In this work, we adopt ICL strategy by employing the pretrained knowledge of Large Language Models (LLMs), gathered during the pretraining phase of an LLM, to automatically extract the PICO-related terminologies from clinical trial documents in unsupervised set up to bypass the availability of large number of annotated data instances. Additionally, to showcase the highest effectiveness of LLM in oracle scenario where large number of annotated samples are available, we adopt the instruction tuning strategy by employing Low Rank Adaptation (LORA) to conduct the training of gigantic model in low resource environment for the PICO frame extraction task. Our empirical results show that our proposed ICL-based framework produces comparable results on all the version of EBM-NLP datasets and the proposed instruction tuned version of our framework produces state-of-the-art results on all the different EBM-NLP datasets. Our project is available at \url{https://github.com/shrimonmuke0202/AlpaPICO.git}.Keywords
All Related Versions
- Version 1, 2024-09-15, ArXiv
- Published version: Methods, 226, 78.
This publication has 36 references indexed in Scilit:
- KronA: Parameter-Efficient Tuning with Kronecker Adapter2025
- PMC-LLaMA: toward building open-source language models for medicineJournal of the American Medical Informatics Association, 2024
- Sequence labeling with MLTA: Multi-level topic-aware mechanismInformation Sciences, 2023
- Joint extraction of entities and relations via an entity correlated attention neural modelInformation Sciences, 2021
- Learning with joint cross-document information via multi-task learning for named entity recognitionInformation Sciences, 2021
- BioBERT: a pre-trained biomedical language representation model for biomedical text miningBioinformatics, 2019
- Information extraction from historical handwritten document images with a context-aware neural modelPattern Recognition, 2019
- Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World GraphsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books2015
- Automated systems for the de-identification of longitudinal clinical narratives: Overview of 2014 i2b2/UTHealth shared task Track 1Journal of Biomedical Informatics, 2015