Avijit Mitra
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Avijit Mitra has a total of 20 co-authors. Here are the top co-authors by number of shared publications:
- Hong Yu
University of Massachusetts Lowell
7 shared publications - Jack Tsai
National Center on Homelessness Among Veterans, US Department of Veterans Affairs, Washington, DC, USA
4 shared publications - Weisong Liu
Miner School of Computer and Information Sciences, University of Massachusetts Lowell, MA, United States
3 shared publications - Bhanu Pratap Singh Rawat
Manning College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA, United States
2 shared publications - David D. McManus
Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, United States
2 shared publications - David Smelson
Center for Healthcare Organization and Implementation Research, Department of Veterans Affairs Edith Nourse Rogers VA Hospital, Bedford, MA, USA
2 shared publications
Latest Publications
- Open Access
- 19 October 2026
- Published by JMIR Publications Inc. in JMIR AI
- Vol. 5, e104586
Large Language Model–Generated Synthetic Data for Parole and Probation Detection in Clinical Notes: Development and Evaluation Study
Background: Identifying individuals on parole or probation can help inform care pathways and ensure continuity of care in health care systems. However, this sensitive justice-related information is often absent from structured electronic health record fields and is instead documented in narrative clinical notes. Developing automated parole and probation detection (PPD) systems typically requires expert-annotated clinical data, but manual annotation is costly, time-consuming, and constrained by data sparsity and privacy concerns. Objective: This study aimed to develop and evaluate a synthetic-only training paradigm for PPD from clinical text that eliminates the need for expert-annotated training data while maintaining strong performance and cross-domain transfer to real-world data. Methods: We developed Synth-PPD, a synthetic dataset comprising 7579 labeled clinical notes generated using large language models (LLMs), redundancy filtering, and automated quality evaluation. Dataset quality was assessed through human review of 1517 synthetic notes by 2 annotators. The primary outcome was the macro–F1-score for PPD. We evaluated Synth-PPD across unsupervised and supervised learning settings using LLMs, traditional machine learning models, and transformer encoders. Performance was assessed on 293 expert-annotated real-world clinical notes from the US Department of Veterans Affairs. Results: Synth-PPD achieved 90.05% human-LLM alignment. In the unsupervised setting, Llama 3.3-70B-Instruct with chain-of-thought prompting and few-shot examples from Synth-PPD achieved a macro–F1-score of 87.22%, outperforming both the corresponding zero-shot model (80.10%) and the best rule-based baseline (77.55%; P<.001 for all comparisons). In the supervised setting, Llama 3.1-8B fine-tuned on Synth-PPD achieved a macro–F1-score of 81.43%, exceeding the best rule-based baseline (77.55%; P<.001) and achieving a numerically higher score than the substantially larger zero-shot Llama 3.3-70B-Instruct model (80.10%; P=.08). In contrast, traditional machine learning models and transformer encoders exhibited greater performance degradation under domain shift and derived limited benefit from increasing synthetic training data. Additional experiments showed that increasing synthetic training data improved cross-domain performance for LLM-based models but yielded comparatively smaller gains for traditional approaches. Conclusions: Synthetic data generated by LLMs can serve as a scalable, cost-effective, and privacy-preserving alternative to expert-annotated real-world clinical data for PPD from clinical text. The findings demonstrate successful cross-domain transfer to real-world US Department of Veterans Affairs clinical notes.
- 19 October 2026
- Published by JMIR Publications Inc. in JMIR AI
- Vol. 5, e104586
Open AccessLarge Language Model–Generated Synthetic Data for Parole and Probation Detection in Clinical Notes: Development and Evaluation Study
Background: Identifying individuals on parole or probation can help inform care pathways and ensure continuity of care in health care systems. However, this sensitive justice-related information is often absent from structured electronic health record fields and is instead documented in narrative clinical notes. Developing automated parole and probation detection (PPD) systems typically requires expert-annotated clinical data, but manual annotation is costly, time-consuming, and constrained by data sparsity and privacy concerns. Objective: This study aimed to develop and evaluate a synthetic-only training paradigm for PPD from clinical text that eliminates the need for expert-annotated training data while maintaining strong performance and cross-domain transfer to real-world data. Methods: We developed Synth-PPD, a synthetic dataset comprising 7579 labeled clinical notes generated using large language models (LLMs), redundancy filtering, and automated quality evaluation. Dataset quality was assessed through human review of 1517 synthetic notes by 2 annotators. The primary outcome was the macro–F1-score for PPD. We evaluated Synth-PPD across unsupervised and supervised learning settings using LLMs, traditional machine learning models, and transformer encoders. Performance was assessed on 293 expert-annotated real-world clinical notes from the US Department of Veterans Affairs. Results: Synth-PPD achieved 90.05% human-LLM alignment. In the unsupervised setting, Llama 3.3-70B-Instruct with chain-of-thought prompting and few-shot examples from Synth-PPD achieved a macro–F1-score of 87.22%, outperforming both the corresponding zero-shot model (80.10%) and the best rule-based baseline (77.55%; P<.001 for all comparisons). In the supervised setting, Llama 3.1-8B fine-tuned on Synth-PPD achieved a macro–F1-score of 81.43%, exceeding the best rule-based baseline (77.55%; P<.001) and achieving a numerically higher score than the substantially larger zero-shot Llama 3.3-70B-Instruct model (80.10%; P=.08). In contrast, traditional machine learning models and transformer encoders exhibited greater performance degradation under domain shift and derived limited benefit from increasing synthetic training data. Additional experiments showed that increasing synthetic training data improved cross-domain performance for LLM-based models but yielded comparatively smaller gains for traditional approaches. Conclusions: Synthetic data generated by LLMs can serve as a scalable, cost-effective, and privacy-preserving alternative to expert-annotated real-world clinical data for PPD from clinical text. The findings demonstrate successful cross-domain transfer to real-world US Department of Veterans Affairs clinical notes.
- Open Access
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 6234-6245
Scalable Counterfactual Risk Estimation for Rare Events in Longitudinal Data
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 6234-6245
Open AccessScalable Counterfactual Risk Estimation for Rare Events in Longitudinal Data
- Open Access
- 14 June 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Large Language Model–Generated Synthetic Data for Parole and Probation Detection in Clinical Notes: Development and Evaluation Study (Preprint)
BACKGROUND Identifying individuals on parole or probation can help inform care pathways and ensure continuity of care in health care systems. However, this sensitive justice-related information is often absent from structured electronic health record fields and is instead documented in narrative clinical notes. Developing automated parole and probation detection (PPD) systems typically requires expert-annotated clinical data, but manual annotation is costly, time-consuming, and constrained by data sparsity and privacy concerns. OBJECTIVE This study aimed to develop and evaluate a synthetic-only training paradigm for PPD from clinical text that eliminates the need for expert-annotated training data while maintaining strong performance and cross-domain transfer to real-world data. METHODS We developed Synth-PPD, a synthetic dataset comprising 7579 labeled clinical notes generated using large language models (LLMs), redundancy filtering, and automated quality evaluation. Dataset quality was assessed through human review of 1517 synthetic notes by 2 annotators. The primary outcome was the macro–F1-score for PPD. We evaluated Synth-PPD across unsupervised and supervised learning settings using LLMs, traditional machine learning models, and transformer encoders. Performance was assessed on 293 expert-annotated real-world clinical notes from the US Department of Veterans Affairs. RESULTS Synth-PPD achieved 90.05% human-LLM alignment. In the unsupervised setting, Llama 3.3-70B-Instruct with chain-of-thought prompting and few-shot examples from Synth-PPD achieved a macro–F1-score of 87.22%, outperforming both the corresponding zero-shot model (80.10%) and the best rule-based baseline (77.55%; P<.001 for all comparisons). In the supervised setting, Llama 3.1-8B fine-tuned on Synth-PPD achieved a macro–F1-score of 81.43%, exceeding the best rule-based baseline (77.55%; P<.001) and achieving a numerically higher score than the substantially larger zero-shot Llama 3.3-70B-Instruct model (80.10%; P=.08). In contrast, traditional machine learning models and transformer encoders exhibited greater performance degradation under domain shift and derived limited benefit from increasing synthetic training data. Additional experiments showed that increasing synthetic training data improved cross-domain performance for LLM-based models but yielded comparatively smaller gains for traditional approaches. CONCLUSIONS Synthetic data generated by LLMs can serve as a scalable, cost-effective, and privacy-preserving alternative to expert-annotated real-world clinical data for PPD from clinical text. The findings demonstrate successful cross-domain transfer to real-world US Department of Veterans Affairs clinical notes.
- 14 June 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessLarge Language Model–Generated Synthetic Data for Parole and Probation Detection in Clinical Notes: Development and Evaluation Study (Preprint)
BACKGROUND Identifying individuals on parole or probation can help inform care pathways and ensure continuity of care in health care systems. However, this sensitive justice-related information is often absent from structured electronic health record fields and is instead documented in narrative clinical notes. Developing automated parole and probation detection (PPD) systems typically requires expert-annotated clinical data, but manual annotation is costly, time-consuming, and constrained by data sparsity and privacy concerns. OBJECTIVE This study aimed to develop and evaluate a synthetic-only training paradigm for PPD from clinical text that eliminates the need for expert-annotated training data while maintaining strong performance and cross-domain transfer to real-world data. METHODS We developed Synth-PPD, a synthetic dataset comprising 7579 labeled clinical notes generated using large language models (LLMs), redundancy filtering, and automated quality evaluation. Dataset quality was assessed through human review of 1517 synthetic notes by 2 annotators. The primary outcome was the macro–F1-score for PPD. We evaluated Synth-PPD across unsupervised and supervised learning settings using LLMs, traditional machine learning models, and transformer encoders. Performance was assessed on 293 expert-annotated real-world clinical notes from the US Department of Veterans Affairs. RESULTS Synth-PPD achieved 90.05% human-LLM alignment. In the unsupervised setting, Llama 3.3-70B-Instruct with chain-of-thought prompting and few-shot examples from Synth-PPD achieved a macro–F1-score of 87.22%, outperforming both the corresponding zero-shot model (80.10%) and the best rule-based baseline (77.55%; P<.001 for all comparisons). In the supervised setting, Llama 3.1-8B fine-tuned on Synth-PPD achieved a macro–F1-score of 81.43%, exceeding the best rule-based baseline (77.55%; P<.001) and achieving a numerically higher score than the substantially larger zero-shot Llama 3.3-70B-Instruct model (80.10%; P=.08). In contrast, traditional machine learning models and transformer encoders exhibited greater performance degradation under domain shift and derived limited benefit from increasing synthetic training data. Additional experiments showed that increasing synthetic training data improved cross-domain performance for LLM-based models but yielded comparatively smaller gains for traditional approaches. CONCLUSIONS Synthetic data generated by LLMs can serve as a scalable, cost-effective, and privacy-preserving alternative to expert-annotated real-world clinical data for PPD from clinical text. The findings demonstrate successful cross-domain transfer to real-world US Department of Veterans Affairs clinical notes.
- Open Access
- 18 December 2021
- Published by Springer Nature in BMC Health Services Research
- Vol. 21 (1) , 1-15
Guideline-discordant dosing of direct-acting oral anticoagulants in the veterans health administration
Background: Clear guidelines exist to guide the dosing of direct-acting oral anticoagulants (DOACs). It is not known how consistently these guidelines are followed in practice. Methods: We studied patients from the Veterans Health Administration (VA) with non-valvular atrial fibrillation who received DOACs (dabigatran, rivaroxaban, apixaban) between 2010 and 2016. We used patient characteristics (age, creatinine, body mass) to identify which patients met guideline recommendations for low-dose therapy and which for full-dose therapy. We examined how often patient dosing was concordant with these recommendations. We examined variation in guideline-concordant dosing by site of care and over time. We examined patient-level predictors of guideline-concordant dosing using multivariable logistic models. Results: A total of 73,672 patients who were prescribed DOACS were included. Of 5837 patients who were recommended to receive low-dose therapy, 1331 (23%) received full-dose therapy instead. Of 67,935 patients recommended to receive full-dose therapy, 4079 (6%) received low-dose therapy instead. Sites varied widely on guideline discordant dosing; on inappropriate low-dose therapy, sites varied from 0 to 15%, while on inappropriate high-dose therapy, from 0 to 41%. Guideline discordant therapy decreased by about 20% in a relative sense over time, but its absolute numbers grew as DOAC therapy became more common. The most important patient-level predictors of receiving guideline-discordant therapy were older age and creatinine function being near the cutoff value. Conclusions: A substantial portion of DOAC prescriptions in the VA system are dosed contrary to clinical guidelines. This phenomenon varies widely across sites of care and has persisted over time.
- 18 December 2021
- Published by Springer Nature in BMC Health Services Research
- Vol. 21 (1) , 1-15
Open AccessGuideline-discordant dosing of direct-acting oral anticoagulants in the veterans health administration
Background: Clear guidelines exist to guide the dosing of direct-acting oral anticoagulants (DOACs). It is not known how consistently these guidelines are followed in practice. Methods: We studied patients from the Veterans Health Administration (VA) with non-valvular atrial fibrillation who received DOACs (dabigatran, rivaroxaban, apixaban) between 2010 and 2016. We used patient characteristics (age, creatinine, body mass) to identify which patients met guideline recommendations for low-dose therapy and which for full-dose therapy. We examined how often patient dosing was concordant with these recommendations. We examined variation in guideline-concordant dosing by site of care and over time. We examined patient-level predictors of guideline-concordant dosing using multivariable logistic models. Results: A total of 73,672 patients who were prescribed DOACS were included. Of 5837 patients who were recommended to receive low-dose therapy, 1331 (23%) received full-dose therapy instead. Of 67,935 patients recommended to receive full-dose therapy, 4079 (6%) received low-dose therapy instead. Sites varied widely on guideline discordant dosing; on inappropriate low-dose therapy, sites varied from 0 to 15%, while on inappropriate high-dose therapy, from 0 to 41%. Guideline discordant therapy decreased by about 20% in a relative sense over time, but its absolute numbers grew as DOAC therapy became more common. The most important patient-level predictors of receiving guideline-discordant therapy were older age and creatinine function being near the cutoff value. Conclusions: A substantial portion of DOAC prescriptions in the VA system are dosed contrary to clinical guidelines. This phenomenon varies widely across sites of care and has persisted over time.
- Open Access
- 8 November 2021
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 9 (11) , e32851
Risk Factors Associated With Nonfatal Opioid Overdose Leading to Intensive Care Unit Admission: A Cross-sectional Study
Background: Opioid overdose (OD) and related deaths have significantly increased in the United States over the last 2 decades. Existing studies have mostly focused on demographic and clinical risk factors in noncritical care settings. Social and behavioral determinants of health (SBDH) are infrequently coded in the electronic health record (EHR) and usually buried in unstructured EHR notes, reflecting possible gaps in clinical care and observational research. Therefore, SBDH often receive less attention despite being important risk factors for OD. Natural language processing (NLP) can alleviate this problem. Objective: The objectives of this study were two-fold: First, we examined the usefulness of NLP for SBDH extraction from unstructured EHR text, and second, for intensive care unit (ICU) admissions, we investigated risk factors including SBDH for nonfatal OD. Methods: We performed a cross-sectional analysis of admission data from the EHR of patients in the ICU of Beth Israel Deaconess Medical Center between 2001 and 2012. We used patient admission data and International Classification of Diseases, Ninth Revision (ICD-9) diagnoses to extract demographics, nonfatal OD, SBDH, and other clinical variables. In addition to obtaining SBDH information from the ICD codes, an NLP model was developed to extract 6 SBDH variables from EHR notes, namely, housing insecurity, unemployment, social isolation, alcohol use, smoking, and illicit drug use. We adopted a sequential forward selection process to select relevant clinical variables. Multivariable logistic regression analysis was used to evaluate the associations with nonfatal OD, and relative risks were quantified as covariate-adjusted odds ratios (aOR). Results: The strongest association with nonfatal OD was found to be drug use disorder (aOR 8.17, 95% CI 5.44-12.27), followed by bipolar disorder (aOR 2.69, 95% CI 1.68-4.29). Among others, major depressive disorder (aOR 2.57, 95% CI 1.12-5.88), being on a Medicaid health insurance program (aOR 2.26, 95% CI 1.43-3.58), history of illicit drug use (aOR 2.09, 95% CI 1.15-3.79), and current use of illicit drugs (aOR 2.06, 95% CI 1.20-3.55) were strongly associated with increased risk of nonfatal OD. Conversely, Blacks (aOR 0.51, 95% CI 0.28-0.94), older age groups (40-64 years: aOR 0.65, 95% CI 0.44-0.96; >64 years: aOR 0.16, 95% CI 0.08-0.34) and those with tobacco use disorder (aOR 0.53, 95% CI 0.32-0.89) or alcohol use disorder (aOR 0.64, 95% CI 0.42-1.00) had decreased risk of nonfatal OD. Moreover, 99.82% of all SBDH information was identified by the NLP model, in contrast to only 0.18% identified by the ICD codes. Conclusions: This is the first study to analyze the risk factors for nonfatal OD in an ICU setting using NLP-extracted SBDH from EHR notes. We found several risk factors associated with nonfatal OD including SBDH. SBDH are richly described in EHR notes, supporting the importance of integrating NLP-derived SBDH into OD risk assessment. More studies in ICU settings can help health care systems better understand and respond to the opioid epidemic.
- 8 November 2021
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 9 (11) , e32851
Open AccessRisk Factors Associated With Nonfatal Opioid Overdose Leading to Intensive Care Unit Admission: A Cross-sectional Study
Background: Opioid overdose (OD) and related deaths have significantly increased in the United States over the last 2 decades. Existing studies have mostly focused on demographic and clinical risk factors in noncritical care settings. Social and behavioral determinants of health (SBDH) are infrequently coded in the electronic health record (EHR) and usually buried in unstructured EHR notes, reflecting possible gaps in clinical care and observational research. Therefore, SBDH often receive less attention despite being important risk factors for OD. Natural language processing (NLP) can alleviate this problem. Objective: The objectives of this study were two-fold: First, we examined the usefulness of NLP for SBDH extraction from unstructured EHR text, and second, for intensive care unit (ICU) admissions, we investigated risk factors including SBDH for nonfatal OD. Methods: We performed a cross-sectional analysis of admission data from the EHR of patients in the ICU of Beth Israel Deaconess Medical Center between 2001 and 2012. We used patient admission data and International Classification of Diseases, Ninth Revision (ICD-9) diagnoses to extract demographics, nonfatal OD, SBDH, and other clinical variables. In addition to obtaining SBDH information from the ICD codes, an NLP model was developed to extract 6 SBDH variables from EHR notes, namely, housing insecurity, unemployment, social isolation, alcohol use, smoking, and illicit drug use. We adopted a sequential forward selection process to select relevant clinical variables. Multivariable logistic regression analysis was used to evaluate the associations with nonfatal OD, and relative risks were quantified as covariate-adjusted odds ratios (aOR). Results: The strongest association with nonfatal OD was found to be drug use disorder (aOR 8.17, 95% CI 5.44-12.27), followed by bipolar disorder (aOR 2.69, 95% CI 1.68-4.29). Among others, major depressive disorder (aOR 2.57, 95% CI 1.12-5.88), being on a Medicaid health insurance program (aOR 2.26, 95% CI 1.43-3.58), history of illicit drug use (aOR 2.09, 95% CI 1.15-3.79), and current use of illicit drugs (aOR 2.06, 95% CI 1.20-3.55) were strongly associated with increased risk of nonfatal OD. Conversely, Blacks (aOR 0.51, 95% CI 0.28-0.94), older age groups (40-64 years: aOR 0.65, 95% CI 0.44-0.96; >64 years: aOR 0.16, 95% CI 0.08-0.34) and those with tobacco use disorder (aOR 0.53, 95% CI 0.32-0.89) or alcohol use disorder (aOR 0.64, 95% CI 0.42-1.00) had decreased risk of nonfatal OD. Moreover, 99.82% of all SBDH information was identified by the NLP model, in contrast to only 0.18% identified by the ICD codes. Conclusions: This is the first study to analyze the risk factors for nonfatal OD in an ICU setting using NLP-extracted SBDH from EHR notes. We found several risk factors associated with nonfatal OD including SBDH. SBDH are richly described in EHR notes, supporting the importance of integrating NLP-derived SBDH into OD risk assessment. More studies in ICU settings can help health care systems better understand and respond to the opioid epidemic.
- Open Access
- 12 August 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Risk Factors Associated With Nonfatal Opioid Overdose Leading to Intensive Care Unit Admission: A Cross-sectional Study (Preprint)
BACKGROUND Opioid overdose (OD) and related deaths have significantly increased in the United States over the last 2 decades. Existing studies have mostly focused on demographic and clinical risk factors in noncritical care settings. Social and behavioral determinants of health (SBDH) are infrequently coded in the electronic health record (EHR) and usually buried in unstructured EHR notes, reflecting possible gaps in clinical care and observational research. Therefore, SBDH often receive less attention despite being important risk factors for OD. Natural language processing (NLP) can alleviate this problem. OBJECTIVE The objectives of this study were two-fold: First, we examined the usefulness of NLP for SBDH extraction from unstructured EHR text, and second, for intensive care unit (ICU) admissions, we investigated risk factors including SBDH for nonfatal OD. METHODS We performed a cross-sectional analysis of admission data from the EHR of patients in the ICU of Beth Israel Deaconess Medical Center between 2001 and 2012. We used patient admission data and International Classification of Diseases, Ninth Revision (ICD-9) diagnoses to extract demographics, nonfatal OD, SBDH, and other clinical variables. In addition to obtaining SBDH information from the ICD codes, an NLP model was developed to extract 6 SBDH variables from EHR notes, namely, housing insecurity, unemployment, social isolation, alcohol use, smoking, and illicit drug use. We adopted a sequential forward selection process to select relevant clinical variables. Multivariable logistic regression analysis was used to evaluate the associations with nonfatal OD, and relative risks were quantified as covariate-adjusted odds ratios (aOR). RESULTS The strongest association with nonfatal OD was found to be drug use disorder (aOR 8.17, 95% CI 5.44-12.27), followed by bipolar disorder (aOR 2.69, 95% CI 1.68-4.29). Among others, major depressive disorder (aOR 2.57, 95% CI 1.12-5.88), being on a Medicaid health insurance program (aOR 2.26, 95% CI 1.43-3.58), history of illicit drug use (aOR 2.09, 95% CI 1.15-3.79), and current use of illicit drugs (aOR 2.06, 95% CI 1.20-3.55) were strongly associated with increased risk of nonfatal OD. Conversely, Blacks (aOR 0.51, 95% CI 0.28-0.94), older age groups (40-64 years: aOR 0.65, 95% CI 0.44-0.96; >64 years: aOR 0.16, 95% CI 0.08-0.34) and those with tobacco use disorder (aOR 0.53, 95% CI 0.32-0.89) or alcohol use disorder (aOR 0.64, 95% CI 0.42-1.00) had decreased risk of nonfatal OD. Moreover, 99.82% of all SBDH information was identified by the NLP model, in contrast to only 0.18% identified by the ICD codes. CONCLUSIONS This is the first study to analyze the risk factors for nonfatal OD in an ICU setting using NLP-extracted SBDH from EHR notes. We found several risk factors associated with nonfatal OD including SBDH. SBDH are richly described in EHR notes, supporting the importance of integrating NLP-derived SBDH into OD risk assessment. More studies in ICU settings can help health care systems better understand and respond to the opioid epidemic.
- 12 August 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessRisk Factors Associated With Nonfatal Opioid Overdose Leading to Intensive Care Unit Admission: A Cross-sectional Study (Preprint)
BACKGROUND Opioid overdose (OD) and related deaths have significantly increased in the United States over the last 2 decades. Existing studies have mostly focused on demographic and clinical risk factors in noncritical care settings. Social and behavioral determinants of health (SBDH) are infrequently coded in the electronic health record (EHR) and usually buried in unstructured EHR notes, reflecting possible gaps in clinical care and observational research. Therefore, SBDH often receive less attention despite being important risk factors for OD. Natural language processing (NLP) can alleviate this problem. OBJECTIVE The objectives of this study were two-fold: First, we examined the usefulness of NLP for SBDH extraction from unstructured EHR text, and second, for intensive care unit (ICU) admissions, we investigated risk factors including SBDH for nonfatal OD. METHODS We performed a cross-sectional analysis of admission data from the EHR of patients in the ICU of Beth Israel Deaconess Medical Center between 2001 and 2012. We used patient admission data and International Classification of Diseases, Ninth Revision (ICD-9) diagnoses to extract demographics, nonfatal OD, SBDH, and other clinical variables. In addition to obtaining SBDH information from the ICD codes, an NLP model was developed to extract 6 SBDH variables from EHR notes, namely, housing insecurity, unemployment, social isolation, alcohol use, smoking, and illicit drug use. We adopted a sequential forward selection process to select relevant clinical variables. Multivariable logistic regression analysis was used to evaluate the associations with nonfatal OD, and relative risks were quantified as covariate-adjusted odds ratios (aOR). RESULTS The strongest association with nonfatal OD was found to be drug use disorder (aOR 8.17, 95% CI 5.44-12.27), followed by bipolar disorder (aOR 2.69, 95% CI 1.68-4.29). Among others, major depressive disorder (aOR 2.57, 95% CI 1.12-5.88), being on a Medicaid health insurance program (aOR 2.26, 95% CI 1.43-3.58), history of illicit drug use (aOR 2.09, 95% CI 1.15-3.79), and current use of illicit drugs (aOR 2.06, 95% CI 1.20-3.55) were strongly associated with increased risk of nonfatal OD. Conversely, Blacks (aOR 0.51, 95% CI 0.28-0.94), older age groups (40-64 years: aOR 0.65, 95% CI 0.44-0.96; >64 years: aOR 0.16, 95% CI 0.08-0.34) and those with tobacco use disorder (aOR 0.53, 95% CI 0.32-0.89) or alcohol use disorder (aOR 0.64, 95% CI 0.42-1.00) had decreased risk of nonfatal OD. Moreover, 99.82% of all SBDH information was identified by the NLP model, in contrast to only 0.18% identified by the ICD codes. CONCLUSIONS This is the first study to analyze the risk factors for nonfatal OD in an ICU setting using NLP-extracted SBDH from EHR notes. We found several risk factors associated with nonfatal OD including SBDH. SBDH are richly described in EHR notes, supporting the importance of integrating NLP-derived SBDH into OD risk assessment. More studies in ICU settings can help health care systems better understand and respond to the opioid epidemic.
- Open Access
- 2 July 2021
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 9 (7) , e27527
Relation Classification for Bleeding Events From Electronic Health Records Using Deep Learning Systems: An Empirical Study
Background: Accurate detection of bleeding events from electronic health records (EHRs) is crucial for identifying and characterizing different common and serious medical problems. To extract such information from EHRs, it is essential to identify the relations between bleeding events and related clinical entities (eg, bleeding anatomic sites and lab tests). With the advent of natural language processing (NLP) and deep learning (DL)-based techniques, many studies have focused on their applicability for various clinical applications. However, no prior work has utilized DL to extract relations between bleeding events and relevant entities. Objective: In this study, we aimed to evaluate multiple DL systems on a novel EHR data set for bleeding event–related relation classification. Methods: We first expert annotated a new data set of 1046 deidentified EHR notes for bleeding events and their attributes. On this data set, we evaluated three state-of-the-art DL architectures for the bleeding event relation classification task, namely, convolutional neural network (CNN), attention-guided graph convolutional network (AGGCN), and Bidirectional Encoder Representations from Transformers (BERT). We used three BERT-based models, namely, BERT pretrained on biomedical data (BioBERT), BioBERT pretrained on clinical text (Bio+Clinical BERT), and BioBERT pretrained on EHR notes (EhrBERT). Results: Our experiments showed that the BERT-based models significantly outperformed the CNN and AGGCN models. Specifically, BioBERT achieved a macro F1 score of 0.842, outperforming both the AGGCN (macro F1 score, 0.828) and CNN models (macro F1 score, 0.763) by 1.4% (P<.001) and 7.9% (P<.001), respectively. Conclusions: In this comprehensive study, we explored and compared different DL systems to classify relations between bleeding events and other medical concepts. On our corpus, BERT-based models outperformed other DL models for identifying the relations of bleeding-related entities. In addition to pretrained contextualized word representation, BERT-based models benefited from the use of target entity representation over traditional sequence representation
- 2 July 2021
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 9 (7) , e27527
Open AccessRelation Classification for Bleeding Events From Electronic Health Records Using Deep Learning Systems: An Empirical Study
Background: Accurate detection of bleeding events from electronic health records (EHRs) is crucial for identifying and characterizing different common and serious medical problems. To extract such information from EHRs, it is essential to identify the relations between bleeding events and related clinical entities (eg, bleeding anatomic sites and lab tests). With the advent of natural language processing (NLP) and deep learning (DL)-based techniques, many studies have focused on their applicability for various clinical applications. However, no prior work has utilized DL to extract relations between bleeding events and relevant entities. Objective: In this study, we aimed to evaluate multiple DL systems on a novel EHR data set for bleeding event–related relation classification. Methods: We first expert annotated a new data set of 1046 deidentified EHR notes for bleeding events and their attributes. On this data set, we evaluated three state-of-the-art DL architectures for the bleeding event relation classification task, namely, convolutional neural network (CNN), attention-guided graph convolutional network (AGGCN), and Bidirectional Encoder Representations from Transformers (BERT). We used three BERT-based models, namely, BERT pretrained on biomedical data (BioBERT), BioBERT pretrained on clinical text (Bio+Clinical BERT), and BioBERT pretrained on EHR notes (EhrBERT). Results: Our experiments showed that the BERT-based models significantly outperformed the CNN and AGGCN models. Specifically, BioBERT achieved a macro F1 score of 0.842, outperforming both the AGGCN (macro F1 score, 0.828) and CNN models (macro F1 score, 0.763) by 1.4% (P<.001) and 7.9% (P<.001), respectively. Conclusions: In this comprehensive study, we explored and compared different DL systems to classify relations between bleeding events and other medical concepts. On our corpus, BERT-based models outperformed other DL models for identifying the relations of bleeding-related entities. In addition to pretrained contextualized word representation, BERT-based models benefited from the use of target entity representation over traditional sequence representation
- Open Access
- 27 January 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Relation Classification for Bleeding Events From Electronic Health Records Using Deep Learning Systems: An Empirical Study (Preprint)
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Accurate detection of bleeding events from electronic health records (EHRs) is crucial for identifying and characterizing different common and serious medical problems. To extract such information from EHRs, it is essential to identify the relations between bleeding events and related clinical entities (eg, bleeding anatomic sites and lab tests). With the advent of natural language processing (NLP) and deep learning (DL)-based techniques, many studies have focused on their applicability for various clinical applications. However, no prior work has utilized DL to extract relations between bleeding events and relevant entities. Objective: In this study, we aimed to evaluate multiple DL systems on a novel EHR data set for bleeding event–related relation classification. Methods: We first expert annotated a new data set of 1046 deidentified EHR notes for bleeding events and their attributes. On this data set, we evaluated three state-of-the-art DL architectures for the bleeding event relation classification task, namely, convolutional neural network (CNN), attention-guided graph convolutional network (AGGCN), and Bidirectional Encoder Representations from Transformers (BERT). We used three BERT-based models, namely, BERT pretrained on biomedical data (BioBERT), BioBERT pretrained on clinical text (Bio+Clinical BERT), and BioBERT pretrained on EHR notes (EhrBERT). Results: Our experiments showed that the BERT-based models significantly outperformed the CNN and AGGCN models. Specifically, BioBERT achieved a macro F1 score of 0.842, outperforming both the AGGCN (macro F1 score, 0.828) and CNN models (macro F1 score, 0.763) by 1.4% (P<.001) and 7.9% (P<.001), respectively. Conclusions: In this comprehensive study, we explored and compared different DL systems to classify relations between bleeding events and other medical concepts. On our corpus, BERT-based models outperformed other DL models for identifying the relations of bleeding-related entities. In addition to pretrained contextualized word representation, BERT-based models benefited from the use of target entity representation over traditional sequence representation
- 27 January 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessRelation Classification for Bleeding Events From Electronic Health Records Using Deep Learning Systems: An Empirical Study (Preprint)
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Accurate detection of bleeding events from electronic health records (EHRs) is crucial for identifying and characterizing different common and serious medical problems. To extract such information from EHRs, it is essential to identify the relations between bleeding events and related clinical entities (eg, bleeding anatomic sites and lab tests). With the advent of natural language processing (NLP) and deep learning (DL)-based techniques, many studies have focused on their applicability for various clinical applications. However, no prior work has utilized DL to extract relations between bleeding events and relevant entities. Objective: In this study, we aimed to evaluate multiple DL systems on a novel EHR data set for bleeding event–related relation classification. Methods: We first expert annotated a new data set of 1046 deidentified EHR notes for bleeding events and their attributes. On this data set, we evaluated three state-of-the-art DL architectures for the bleeding event relation classification task, namely, convolutional neural network (CNN), attention-guided graph convolutional network (AGGCN), and Bidirectional Encoder Representations from Transformers (BERT). We used three BERT-based models, namely, BERT pretrained on biomedical data (BioBERT), BioBERT pretrained on clinical text (Bio+Clinical BERT), and BioBERT pretrained on EHR notes (EhrBERT). Results: Our experiments showed that the BERT-based models significantly outperformed the CNN and AGGCN models. Specifically, BioBERT achieved a macro F1 score of 0.842, outperforming both the AGGCN (macro F1 score, 0.828) and CNN models (macro F1 score, 0.763) by 1.4% (P<.001) and 7.9% (P<.001), respectively. Conclusions: In this comprehensive study, we explored and compared different DL systems to classify relations between bleeding events and other medical concepts. On our corpus, BERT-based models outperformed other DL models for identifying the relations of bleeding-related entities. In addition to pretrained contextualized word representation, BERT-based models benefited from the use of target entity representation over traditional sequence representation