Natural Language Processing to identify pneumonia from radiology reports
- 1 April 2013
- journal article
- research article
- Published by Wiley in Pharmacoepidemiology and Drug Safety
- Vol. 22 (8) , 834-841
Abstract
Purpose: This study aimed to develop Natural Language Processing (NLP) approaches to supplement manual outcome validation, specifically to validate pneumonia cases from chest radiograph reports.Methods: We trained one NLP system, ONYX, using radiograph reports from children and adults that were previously manually reviewed. We then assessed its validity on a test set of 5000 reports. We aimed to substantially decrease manual review, not replace it entirely, and so, we classified reports as follows: (1) consistent with pneumonia; (2) inconsistent with pneumonia; or (3) requiring manual review because of complex features. We developed processes tailored either to optimize accuracy or to minimize manual review. Using logistic regression, we jointly modeled sensitivity and specificity of ONYX in relation to patient age, comorbidity, and care setting. We estimated positive and negative predictive value (PPV and NPV) assuming pneumonia prevalence in the source data.Results: Tailored for accuracy, ONYX identified 25% of reports as requiring manual review (34% of true pneumonias and 18% of non‐pneumonias). For the remainder, ONYX's sensitivity was 92% (95% CI 90–93%), specificity 87% (86–88%), PPV 74% (72–76%), and NPV 96% (96–97%). Tailored to minimize manual review, ONYX classified 12% as needing manual review. For the remainder, ONYX had sensitivity 75% (72–77%), specificity 95% (94–96%), PPV 86% (83–88%), and NPV 91% (90–91%).Conclusions: For pneumonia validation, ONYX can replace almost 90% of manual review while maintaining low to moderate misclassification rates. It can be tailored for different outcomes and study needs and thus warrants exploration in other settings. Copyright © 2013 John Wiley & Sons, Ltd.Keywords
This publication has 24 references indexed in Scilit:
- Automating classification of free‐text electronic health records for epidemiological studiesPharmacoepidemiology and Drug Safety, 2012
- Use of Opioids or Benzodiazepines and Risk of Pneumonia in Older Adults: A Population‐Based Case–Control StudyJournal of the American Geriatrics Society, 2011
- ConText: An algorithm for determining negation, experiencer, and temporal status from clinical reportsJournal of Biomedical Informatics, 2009
- Acid-Suppressive Medication Use and the Risk for Hospital-Acquired PneumoniaJAMA, 2009
- Influenza vaccination and risk of community-acquired pneumonia in immunocompetent elderly people: a population-based, nested case-control studyThe Lancet, 2008
- CDC/NHSN surveillance definition of health care–associated infection and criteria for specific types of infections in the acute care settingAmerican Journal of Infection Control, 2008
- Use of Proton Pump Inhibitors and the Risk of Community-Acquired PneumoniaArchives of internal medicine (1960), 2007
- Extracting information on pneumonia in infants using natural language processing of radiology reportsJournal of Biomedical Informatics, 2005
- Risk of Community-Acquired Pneumonia and Use of Gastric Acid–Suppressive DrugsJAMA, 2004
- Longitudinal data analysis using generalized linear modelsBiometrika, 1986