Natural Language Processing Accurately Differentiates Cancer Symptom Information in Electronic Health Record Narratives
- 1 August 2024
- journal article
- research article
- Published by American Society of Clinical Oncology (ASCO) in JCO Clinical Cancer Informatics
- Vol. 8 (8) , e2300235
Abstract
PURPOSE: Identifying cancer symptoms in electronic health record (EHR) narratives is feasible with natural language processing (NLP). However, more efficient NLP systems are needed to detect various symptoms and distinguish observed symptoms from negated symptoms and medication-related side effects. We evaluated the accuracy of NLP in (1) detecting 14 symptom groups (ie, pain, fatigue, swelling, depressed mood, anxiety, nausea/vomiting, pruritus, headache, shortness of breath, constipation, numbness/tingling, decreased appetite, impaired memory, disturbed sleep) and (2) distinguishing observed symptoms in EHR narratives among patients with cancer. METHODS: We extracted 902,508 notes for 11,784 unique patients diagnosed with cancer and developed a gold standard corpus of 1,112 notes labeled for presence or absence of 14 symptom groups. We trained an embeddings-augmented NLP system integrating human and machine intelligence and conventional machine learning algorithms. NLP metrics were calculated on a gold standard corpus subset for testing. RESULTS: The interannotator agreement for labeling the gold standard corpus was excellent at 92%. The embeddings-augmented NLP model achieved the best performance (F1 score = 0.877). The highest NLP accuracy was observed in pruritus (F1 score = 0.937) while the lowest accuracy was in swelling (F1 score = 0.787). After classifying the entire data set with embeddings-augmented NLP, we found that 41% of the notes included symptom documentation. Pain was the most documented symptom (29% of all notes) while impaired memory was the least documented (0.7% of all notes). CONCLUSION: We illustrated the feasibility of detecting 14 symptom groups in EHR narratives and showed that an embeddings-augmented NLP system outperforms conventional machine learning algorithms in detecting symptom information and differentiating observed symptoms from negated symptoms and medication-related side effects.Keywords
This publication has 38 references indexed in Scilit:
- The quest for better clinical word vectors: Ontology based and lexical vector augmentation versus clinical contextual embeddingsComputers in Biology and Medicine, 2021
- Identifying Urinary Tract Infection-Related Information in Home Care Nursing NotesJournal of the American Medical Directors Association, 2021
- Summarization of biomedical articles using domain-specific word embeddings and graph rankingJournal of Biomedical Informatics, 2020
- Mining fall-related information in clinical notes: Comparison of rule-based and novel word embedding-based machine learning approachesJournal of Biomedical Informatics, 2019
- Recent Named Entity Recognition and Classification techniques: A systematic reviewComputer Science Review, 2018
- Clinical information extraction applications: A literature reviewJournal of Biomedical Informatics, 2018
- Natural Language Processing in OncologyJAMA Oncology, 2016
- A Brief Symptom Index for Advanced Lung CancerClinical Lung Cancer, 2012
- The importance of early symptom recognition in the context of early detection and cancer survivalEuropean Journal of Cancer, 2009
- Prevalence of symptoms among patients with advanced cancer: An international collaborative studyJournal of Pain and Symptom Management, 1996