JNJohn Novoa-Laurentiev

John Novoa-Laurentiev

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


John Novoa-Laurentiev has a total of 66 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Extracting language information from clinical notes using large language models

    by Lingfei Qian, Na Hong, Yujia Zhou, Qianqian Xie, Ruey-Ling Weng, Pitchaya Chairuengjitjaras, Xinsong Du, John Lian, Gad A. Marshall, Suzanne V. Blackley, et al.


  • Open Access

    An Extraction Tool for Venous Thromboembolism Symptom Identification in Primary Care Notes to Facilitate Electronic Clinical Quality Measure Reporting: Algorithm Development and Validation Study

    by John Novoa-Laurentiev, Mica Bowen, Avery Pullman, Wenyu Song, Ania Syrowatka, Jin Chen, Michael Sainlaire, Frank Chang, Krissy Gray, Purushottam Panta, et al.

    Background: Diagnosis of venous thromboembolism (VTE) is often delayed, and facilitating earlier diagnosis may improve associated morbidity and mortality. Clinical notes contain information not found elsewhere in the medical record that could facilitate timely VTE diagnosis and accurate quality measurement. However, extracting relevant information from unstructured clinical notes is complex. Today, there are relatively few electronic clinical quality measures (eCQMs) in our national payment program and none that use natural language processing (NLP) techniques for data extraction. NLP holds great promise for making quality measurement more accurate and more efficient. Given the potential of NLP-based applications to facilitate more accurate VTE detection, primary care is one clinical setting in urgent need of this type of tool. Objective: This study aimed to develop a tool that extracts VTE symptoms from clinical notes for use within an eCQM to quantify the rate of delayed diagnosis of VTE in primary care settings. Methods: We iteratively developed an NLP-based data extraction tool, venous thromboembolism symptom extractor (VTExt), on an internal dataset using a rule-based approach to extract VTE symptoms from primary care clinical note text. The VTE symptoms lexicon was derived and optimized with physician guidance and externally validated using datasets from 2 independent health care organizations. We performed 26 rounds of performance evaluation of notes sampled from the case cohort (17,585 patient progress note sentences from 279 patient notes), and 5 rounds of evaluation of the control cohort (2838 patient progress note sentences from 50 patient notes). VTExt’s performance was evaluated using evaluation metrics, including area under the curve, positive predictive value, negative predictive value, sensitivity, and specificity. Results: VTExt achieved near-perfect performance in extracting VTE symptoms from primary care notes sampled from records of patients diagnosed with or without VTE. In external validation, VTExt achieved promising performance in 2 additional geographically distant organizations using different electronic health record systems. When compared against a deep learning model and 4 machine learning models, VTExt exhibited similar or even improved performance across all metrics. Conclusions: This study demonstrates a data-driven NLP-based approach to clinical note information extraction that can be generalized to different electronic health record systems across different institutions. Due to the robust performance of this tool, VTExt is the first NLP application to be used in a nationally endorsed eCQM.


  • Identification of an ANCA-associated vasculitis cohort using deep learning and electronic health records

    by Liqin Wang, John Novoa-Laurentiev, Claire Cook, Shruthi Srivatsan, Yining Hua, Jie Yang, Eli Miloslavsky, Hyon K. Choi, Li Zhou, Zachary S. Wallace


  • Open Access

    An Extraction Tool for Venous Thromboembolism Symptom Identification in Primary Care Notes to Facilitate Electronic Clinical Quality Measure Reporting: Algorithm Development and Validation (Preprint)

    by John Novoa-Laurentiev, Mica Bowen, Avery Pullman, Wenyu Song, Ania Syrowatka, Jin Chen, Michael Sainlaire, Frank Chang, Krissy Gray, Purushottam Panta, et al.

    BACKGROUND Diagnosis of venous thromboembolism (VTE) is often delayed and facilitating earlier diagnosis may improve associated morbidity and mortality. Clinical notes contain information not found elsewhere in the medical record that could facilitate timely VTE diagnosis and accurate quality measurement. However, extracting relevant information from unstructured clinical notes is complex. Today there are relatively few electronic clinical quality measures (eCQMs) in our national payment program and none that use NLP for data extraction. NLP holds great promise for making quality measurement more accurate and more efficient. OBJECTIVE We developed a rule-based NLP tool, VTExt, that extracts VTE symptoms from clinical note text, for use within an eCQM to quantify the rate of delayed diagnosis of VTE in primary care settings. METHODS We iteratively developed VTExt on an internal dataset using a rule-based approach to extract VTE symptoms from primary care clinical note text. The VTE symptoms lexicon was derived and modified with physician guidance and externally validated using data from two external healthcare organizations. RESULTS VTExt achieved near-perfect performance in extracting VTE symptoms from primary care notes sampled from records of patients diagnosed with and without VTE. In external validation VTExt achieved promising performance in two additional geographically distant organizations using different electronic health record systems. When compared against a deep learning-based model, VTExt exhibited similar or improved performance across all metrics. CONCLUSIONS This study demonstrates a data-driven NLP-based approach to clinical note information extraction that can be generalized to different electronic health record (EHR) systems across different institutions. VTExt is the first NLP application to be used in a nationally endorsed eCQM.


  • Open Access

    Identifying Functional Status Impairment in People Living With Dementia Through Natural Language Processing of Clinical Documents: Cross-Sectional Study

    by John Laurentiev, Dae Hyun Kim, Mufaddal Mahesri, Kuan-Yuan Wang, Lily G Bessette, Cassandra York, Heidi Zakoul, Su Been Lee, Li Zhou, Kueiyu Joshua Lin

    Background: Assessment of activities of daily living (ADLs) and instrumental ADLs (iADLs) is key to determining the severity of dementia and care needs among older adults. However, such information is often only documented in free-text clinical notes within the electronic health record and can be challenging to find. Objective: This study aims to develop and validate machine learning models to determine the status of ADL and iADL impairments based on clinical notes. Methods: This cross-sectional study leveraged electronic health record clinical notes from Mass General Brigham’s Research Patient Data Repository linked with Medicare fee-for-service claims data from 2007 to 2017 to identify individuals aged 65 years or older with at least 1 diagnosis of dementia. Notes for encounters both 180 days before and after the first date of dementia diagnosis were randomly sampled. Models were trained and validated using note sentences filtered by expert-curated keywords (filtered cohort) and further evaluated using unfiltered sentences (unfiltered cohort). The model’s performance was compared using area under the receiver operating characteristic curve and area under the precision-recall curve (AUPRC). Results: The study included 10,000 key-term–filtered sentences representing 441 people (n=283, 64.2% women; mean age 82.7, SD 7.9 years) and 1000 unfiltered sentences representing 80 people (n=56, 70% women; mean age 82.8, SD 7.5 years). Area under the receiver operating characteristic curve was high for the best-performing ADL and iADL models on both cohorts (>0.97). For ADL impairment identification, the random forest model achieved the best AUPRC (0.89, 95% CI 0.86-0.91) on the filtered cohort; the support vector machine model achieved the highest AUPRC (0.82, 95% CI 0.75-0.89) for the unfiltered cohort. For iADL impairment, the Bio+Clinical bidirectional encoder representations from transformers (BERT) model had the highest AUPRC (filtered: 0.76, 95% CI 0.68-0.82; unfiltered: 0.58, 95% CI 0.001-1.0). Compared with a keyword-search approach on the unfiltered cohort, machine learning reduced false-positive rates from 4.5% to 0.2% for ADL and 1.8% to 0.1% for iADL. Conclusions: In this study, we demonstrated the ability of machine learning models to accurately identify ADL and iADL impairment based on free-text clinical notes, which could be useful in determining the severity of dementia.


  • Open Access

    Identifying Functional Status Impairment in People Living With Dementia Through Natural Language Processing of Clinical Documents: Cross-Sectional Study (Preprint)

    by John Laurentiev, Dae Hyun Kim, Mufaddal Mahesri, Kuan-Yuan Wang, Lily G Bessette, Cassandra York, Heidi Zakoul, Su Been Lee, Li Zhou, Kueiyu Joshua Lin

    BACKGROUND Assessment of activities of daily living (ADLs) and instrumental ADLs (iADLs) is key to determining the severity of dementia and care needs among older adults. However, such information is often only documented in free-text clinical notes within the electronic health record and can be challenging to find. OBJECTIVE This study aims to develop and validate machine learning models to determine the status of ADL and iADL impairments based on clinical notes. METHODS This cross-sectional study leveraged electronic health record clinical notes from Mass General Brigham’s Research Patient Data Repository linked with Medicare fee-for-service claims data from 2007 to 2017 to identify individuals aged 65 years or older with at least 1 diagnosis of dementia. Notes for encounters both 180 days before and after the first date of dementia diagnosis were randomly sampled. Models were trained and validated using note sentences filtered by expert-curated keywords (filtered cohort) and further evaluated using unfiltered sentences (unfiltered cohort). The model’s performance was compared using area under the receiver operating characteristic curve and area under the precision-recall curve (AUPRC). RESULTS The study included 10,000 key-term–filtered sentences representing 441 people (n=283, 64.2% women; mean age 82.7, SD 7.9 years) and 1000 unfiltered sentences representing 80 people (n=56, 70% women; mean age 82.8, SD 7.5 years). Area under the receiver operating characteristic curve was high for the best-performing ADL and iADL models on both cohorts (>0.97). For ADL impairment identification, the random forest model achieved the best AUPRC (0.89, 95% CI 0.86-0.91) on the filtered cohort; the support vector machine model achieved the highest AUPRC (0.82, 95% CI 0.75-0.89) for the unfiltered cohort. For iADL impairment, the Bio+Clinical bidirectional encoder representations from transformers (BERT) model had the highest AUPRC (filtered: 0.76, 95% CI 0.68-0.82; unfiltered: 0.58, 95% CI 0.001-1.0). Compared with a keyword-search approach on the unfiltered cohort, machine learning reduced false-positive rates from 4.5% to 0.2% for ADL and 1.8% to 0.1% for iADL. CONCLUSIONS In this study, we demonstrated the ability of machine learning models to accurately identify ADL and iADL impairment based on free-text clinical notes, which could be useful in determining the severity of dementia.