Automating Clinical Phenotyping Using Natural Language Processing: An Application for Crohn’s Disease
Preprint
Abstract
Background Crohn’s Disease (CD) patient heterogeneity in clinical practice is captured by the Montreal Classification. While the underlying concepts, disease behavior and age at diagnosis, are relevant outcomes and covariates in studies from real-world data, extracting this clinical information through manual chart review is labor-intensive and with limited scalability. Methods We developed and evaluated automated phenotyping algorithms to extract disease behavior and age at diagnosis from clinical narrative texts, using a rule-based approach based on the spaCy framework, and an approach based on zero-shot inference. The underlying data included 49,572 clinical notes and 2,204 radiology reports from 584 CD patients of the Mount Sinai Crohn’s and Colitis Registry. A test set of 200 clinical texts per classification category was labeled at sentence-level, in addition to patient-level ground truth data. The algorithms were evaluated based on their recall, precision, specificity values, and F1-scores. Results For the labeled dataset, an overall Cohen’s kappa inter-annotator agreement of 0.84 was achieved. The rule-based approach yielded high recall and precision values (0.75 - 1.00) on a note level for the behavioral disease phenotype using clinical notes, with slightly reduced performance using radiology reports. For age at diagnosis, recall and precision values of 0.81 and 0.88 were achieved on note-level, respectively. For both categories, the performance on patient-compared to note-level was reduced, potentially due to the accumulation of false positives and limitations in the data availability. Conclusion Based on our newly annotated dataset, we demonstrated the feasibility of automatically extracting disease behavior and age at diagnosis from clinical text. The resulting labels may facilitate extensive cohort analyses based on electronic health records, and support chart review processes in the future.Keywords
All Related Versions
- Published version: Communications Medicine, 6 (1).
This publication has 40 references indexed in Scilit:
- Development of the Lémann Index to Assess Digestive Tract Damage in Patients With Crohn's DiseaseGastroenterology, 2015
- Pathophysiology of fistula formation in Crohn's diseaseWorld Journal of Gastrointestinal Pathophysiology, 2014
- Performance of the Montreal classification for inflammatory bowel diseasesWorld Journal of Gastroenterology, 2014
- Improving Case Definition of Crohnʼs Disease and Ulcerative Colitis in Electronic Medical Records Using Natural Language ProcessingInflammatory Bowel Diseases, 2013
- Crohn's disease complicated by strictures: a systematic reviewGut, 2013
- Patients Enrolled in Randomized Controlled Trials Do Not Represent the Inflammatory Bowel Disease Patient PopulationClinical Gastroenterology and Hepatology, 2012
- Inter-observer agreement for Crohn's disease sub-phenotypes using the Montreal Classification: How good are we? A multi-centre Australasian studyJournal of Crohn's and Colitis, 2012
- The Montreal classification of inflammatory bowel disease: controversies, consensus, and implicationsGut, 2006
- Toward an Integrated Clinical, Molecular and Serological Classification of Inflammatory Bowel Disease: Report of a Working Party of the 2005 Montreal World Congress of GastroenterologyCanadian Journal of Gastroenterology and Hepatology, 2005
- A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 1960