Extracting social determinants of health events with transformer-based multitask, multilabel named entity recognition
- 21 March 2023
- journal article
- research article
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 30 (8) , 1379-1388
Abstract
Objective: Social determinants of health (SDOH) are nonclinical, socioeconomic conditions that influence patient health and quality of life. Identifying SDOH may help clinicians target interventions. However, SDOH are more frequently available in narrative notes compared to structured electronic health records. The 2022 n2c2 Track 2 competition released clinical notes annotated for SDOH to promote development of NLP systems for extracting SDOH. We developed a system addressing 3 limitations in state-of-the-art SDOH extraction: the inability to identify multiple SDOH events of the same type per sentence, overlapping SDOH attributes within text spans, and SDOH spanning multiple sentences. Materials and Methods: We developed and evaluated a 2-stage architecture. In stage 1, we trained a BioClinical-BERT-based named entity recognition system to extract SDOH event triggers, that is, text spans indicating substance use, employment, or living status. In stage 2, we trained a multitask, multilabel NER to extract arguments (eg, alcohol “type”) for events extracted in stage 1. Evaluation was performed across 3 subtasks differing by provenance of training and validation data using precision, recall, and F1 scores. Results: When trained and validated on data from the same site, we achieved 0.87 precision, 0.89 recall, and 0.88 F1. Across all subtasks, we ranked between second and fourth place in the competition and always within 0.02 F1 from first. Conclusions: Our 2-stage, deep-learning-based NLP system effectively extracted SDOH events from clinical notes. This was achieved with a novel classification framework that leveraged simpler architectures compared to state-of-the-art systems. Improved SDOH extraction may help clinicians improve health outcomes.Keywords
Funding Information
- National Institute of Mental Health (R21MH130853-01)
This publication has 24 references indexed in Scilit:
- Detecting Social and Behavioral Determinants of Health with Structured and Free-Text Clinical DataApplied Clinical Informatics, 2020
- Moonstone: a novel natural language processing system for inferring social risk from clinical narrativesJournal of Biomedical Semantics, 2019
- Natural language processing of lifestyle modification documentationHealth Informatics Journal, 2019
- Exploring Pre-trained Language Models for Event Extraction and Generation2019
- Lessons Learned from Implementation of the Food Insecurity Screening and Referral Program at Kaiser Permanente ColoradoThe Permanente Journal, 2018
- Using Clinical Notes and Natural Language Processing for Automated HIV Risk AssessmentJAIDS Journal of Acquired Immune Deficiency Syndromes, 2018
- Social Determinants of Health 101 for Health Care: Five Plus FiveNAM Perspectives, 2017
- Hospital Readmission and Social Risk Factors Identified from Physician NotesHealth Services Research, 2017
- The Effectiveness of Food Insecurity Screening in Pediatric Primary CareInternational Journal of Child Health and Nutrition, 2014
- Extent of illicit drug use and dependence, and their contribution to the global burden of diseaseThe Lancet, 2012