Rule-based natural language processing for automation of stroke data extraction: a validation study
Authors:
- 1 August 2022
- journal article
- research article
- Published by Springer Nature in Neuroradiology
- Vol. 64 (12) , 2357-2362
Abstract
Purpose Data extraction from radiology free-text reports is time consuming when performed manually. Recently, more automated extraction methods using natural language processing (NLP) are proposed. A previously developed rule-based NLP algorithm showed promise in its ability to extract stroke-related data from radiology reports. We aimed to externally validate the accuracy of CHARTextract, a rule-based NLP algorithm, to extract stroke-related data from free-text radiology reports. Methods Free-text reports of CT angiography (CTA) and perfusion (CTP) studies of consecutive patients with acute ischemic stroke admitted to a regional stroke center for endovascular thrombectomy were analyzed from January 2015 to 2021. Stroke-related variables were manually extracted as reference standard from clinical reports, including proximal and distal anterior circulation occlusion, posterior circulation occlusion, presence of ischemia or hemorrhage, Alberta stroke program early CT score (ASPECTS), and collateral status. These variables were simultaneously extracted using a rule-based NLP algorithm. The NLP algorithm’s accuracy, specificity, sensitivity, positive predictive value (PPV), and negative predictive value (NPV) were assessed. Results The NLP algorithm’s accuracy was > 90% for identifying distal anterior occlusion, posterior circulation occlusion, hemorrhage, and ASPECTS. Accuracy was 85%, 74%, and 79% for proximal anterior circulation occlusion, presence of ischemia, and collateral status respectively. The algorithm confirmed the absence of variables from radiology reports with an 87–100% accuracy. Conclusions Rule-based NLP has a moderate to good performance for stroke-related data extraction from free-text imaging reports. The algorithm’s accuracy was affected by inconsistent report styles and lexicon among reporting radiologists.Keywords
This publication has 20 references indexed in Scilit:
- Effect of definition and methods on estimates of prevalence of large vessel occlusion in acute ischemic stroke: a systematic review and meta-analysisJournal of NeuroInterventional Surgery, 2019
- Automating Ischemic Stroke Subtype Classification Using Machine Learning and Natural Language ProcessingJournal of Stroke and Cerebrovascular Diseases, 2019
- Natural language processing and machine learning algorithm to identify brain MRI reports with acute ischemic strokePLOS ONE, 2019
- NegBio: a high-performance tool for negation and uncertainty detection in radiology reports2018
- Natural Language–based Machine Learning Models for the Annotation of Clinical Radiology ReportsRadiology, 2018
- Use and Utility of Administrative Health Data for Stroke Research and SurveillanceStroke, 2016
- Natural Language Processing in Radiology: A Systematic ReviewRadiology, 2016
- Acute Stroke Imaging Research Roadmap IIStroke, 2013
- Recommendations on Angiographic Revascularization Grading Standards for Acute Ischemic StrokeStroke, 2013
- Coding Neuroradiology Reports for the Northern Manhattan Stroke Study: A Comparison of Natural Language Processing and Manual ReviewComputers and Biomedical Research, 2000