Zero-Shot Clinical Trial Patient Matching with LLMs
Preprint
Abstract
Matching patients to clinical trials is a key unsolved challenge in bringing new drugs to market. Today, identifying patients who meet a trial's eligibility criteria is highly manual, taking up to 1 hour per patient. Automated screening is challenging, however, as it requires understanding unstructured clinical text. Large language models (LLMs) offer a promising solution. In this work, we explore their application to trial matching. First, we design an LLM-based system which, given a patient's medical history as unstructured clinical text, evaluates whether that patient meets a set of inclusion criteria (also specified as free text). Our zero-shot system achieves state-of-the-art scores on the n2c2 2018 cohort selection benchmark. Second, we improve the data and cost efficiency of our method by identifying a prompting strategy which matches patients an order of magnitude faster and more cheaply than the status quo, and develop a two-stage retrieval pipeline that reduces the number of tokens processed by up to a third while retaining high performance. Third, we evaluate the interpretability of our system by having clinicians evaluate the natural language justifications generated by the LLM for each eligibility decision, and show that it can output coherent explanations for 97% of its correct decisions and 75% of its incorrect ones. Our results establish the feasibility of using LLMs to accelerate clinical trial operations.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Matching patients to clinical trials with large language modelsNature Communications, 2024
- Distilling large language models for matching patients to clinical trialsJournal of the American Medical Informatics Association, 2024
- Evaluating eligibility criteria of oncology trials using real-world data and AINature, 2021
- Cohort selection for clinical trials: n2c2 2018 shared task track 1Journal of the American Medical Informatics Association, 2019
- Evaluating shallow and deep learning strategies for the 2018 n2c2 shared task on clinical text classificationJournal of the American Medical Informatics Association, 2019
- Criteria2Query: a natural language interface to clinical databases for cohort definitionJournal of the American Medical Informatics Association, 2019
- EliIE: An open-source information extraction system for clinical trial eligibility criteriaJournal of the American Medical Informatics Association, 2017
- Creation of a new longitudinal corpus of clinical narrativesJournal of Biomedical Informatics, 2015
- Annotating longitudinal clinical narratives for de-identification: The 2014 i2b2/UTHealth corpusJournal of Biomedical Informatics, 2015
- Challenges in clinical natural language processing for automated disorder normalizationJournal of Biomedical Informatics, 2015