Automating clinical phenotyping using natural language processing
Open Access
- 14 January 2026
- journal article
- research article
- Published by Springer Nature in Communications Medicine
- Vol. 6 (1)
Abstract
Background: Real-world studies based on electronic health records often require manual chart review to derive patients’ clinical phenotypes, a labor-intensive task with limited scalability. Here, we developed and compared computable phenotyping based on rules using the spaCy framework and a Large Language Model (LLM), GPT-4, for sub-phenotyping of patients with Crohn’s disease, considering age at diagnosis and disease behavior. Methods: For our rule-based approach, we leveraged the spaCy framework and for the LLM-based approach, we used the GPT-4 model. The underlying data included 49,572 clinical notes and 2204 radiology reports from 584 Crohn’s disease patients. A test set of 280 clinical texts was labeled at sentence-level, in addition to patient-level ground truth data. The algorithms were evaluated based on their recall, precision, specificity values, and F1 scores. Results: Overall, we observe similar or better performance using GPT-4 compared to the rules. On a note-level, the F1 score is at least 0.90 for disease behavior and 0.82 for age at diagnosis, and on patient level at least 0.66 for disease behavior and 0.71 for age at diagnosis. Conclusions: To our knowledge, this is the first study to explore computable phenotyping algorithms based on clinical narrative text for these complex tasks, where prior inter-annotator agreements ranged from 0.54 to 0.98. There is no statistical evidence for a difference to the performance of human experts on this task. Our findings underline the potential of LLMs for computable phenotyping and may support large-scale cohort analyses from electronic health records and streamline chart review processes in the future.Keywords
All Related Versions
This publication has 42 references indexed in Scilit:
- Trends and opportunities in computable clinical phenotyping: A scoping reviewJournal of Biomedical Informatics, 2023
- Development of a Core Outcome Set for Real-world Data in Inflammatory Bowel Disease: A European Crohn’s and Colitis Organisation [ECCO] Position PaperJournal of Crohn's and Colitis, 2022
- Efficacy of Biologic Drugs in Short-Duration Versus Long-Duration Inflammatory Bowel Disease: A Systematic Review and an Individual-Patient Data Meta-Analysis of Randomized Controlled TrialsGastroenterology, 2022
- Deep Analysis of the Peripheral Immune System in IBD Reveals New Insight in Disease Subtyping and Response to Monotherapy or Combination TherapyCellular and Molecular Gastroenterology and Hepatology, 2021
- The use of ICD codes to identify IBD subtypes and phenotypes of the Montreal classification in the Swedish National Patient RegisterScandinavian Journal of Gastroenterology, 2020
- Privacy-Preserving Record Linkage to Identify Fragmented Electronic Medical Records in the All of Us Research Program2020
- Crohn's diseaseThe Lancet, 2017
- Development of the Lémann Index to Assess Digestive Tract Damage in Patients With Crohn's DiseaseGastroenterology, 2015
- Inter-observer agreement for Crohn's disease sub-phenotypes using the Montreal Classification: How good are we? A multi-centre Australasian studyJournal of Crohn's and Colitis, 2012
- Assessment of reliability and validity of IBD phenotyping within the National Institutes of Diabetes and Digestive and Kidney Diseases (NIDDK) IBD Genetics Consortium (IBDGC)Inflammatory Bowel Diseases, 2007