LBLily G. Bessette

Lily G. Bessette

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Lily G. Bessette has a total of 71 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Extracting Cognitive Impairment Assessment Information From Unstructured Notes in Electronic Health Records Using Natural Language Processing Tools: Validation with Clinical Assessment Data

    by Kuan-Yuan Wang, Mufaddal Mahesri, John Novoa-Laurentiev, Lily G Bessette, Cassandra York, Heidi Zakoul, Su Been Lee, Kerry Ngan, Li Zhou, Dae Hyun Kim, et al.

    Purpose: We aimed to develop a Natural Language Processing (NLP) algorithm to extract cognitive scores from electronic health records (EHR) data and compare them with cognitive function recorded by Centers for Medicare & Medicaid Services (CMS)-mandated clinical assessments in nursing homes and home health visits. Patients and Methods: We identified a cohort of Medicare beneficiaries who had either the Minimum Data Set (MDS) or Outcome and Assessment Information Set (OASIS) linked to EHR data from the Research Patient Data Registry (Mass General Brigham, Boston, MA) from 2010 to 2019. We applied an NLP approach to identify the Montreal Cognitive Assessment (MoCA) and the Mini-Mental State Examination (MMSE) scores from unstructured clinician notes in EHR. Using the NLP-extracted MoCA or MMSE scores from EHR, we compared mean differences of extracted MoCA or MMSE by cognition status determined by MDS (impaired vs intact cognition) and OASIS (severe impairment vs intact cognition) data, respectively. Results: Our study cohort had 7419 patients who had MDS (19.7%) or OASIS (80.3%) assessments, with a mean age of 80 (SD=7) years and 60% female. In EHR, the NLP algorithm extracted cognitive test scores with 97% accuracy (95% CI: 92– 99%) for MoCA and 100% accuracy (95% CI: 84– 100%) for MMSE. In MDS, the mean difference in extracted MoCA was − 5.6 (95% CI: − 8.7, − 2.4, p=0.0008), and the mean difference in extracted MMSE was − 7.9 (95% CI: − 12.4, − 3.5, p=0.0012). In OASIS, the mean difference in extracted MoCA and extracted MMSE was − 4.8 (95% CI: − 9.1, − 0.6, p=0.0006) and − 4.5 (95% CI: − 9.5, − 0.5, p=0.0182), respectively. Conclusion: We developed an NLP algorithm to accurately extract cognitive scores from unstructured EHR, and these extracted cognitive scores were well correlated with cognition function recorded in CMS-mandated clinical assessments. This could help researchers identify patients with various degrees of cognitive impairment in EHR-based research.


  • Open Access

    Development and Validation of an Intracranial Hemorrhage Risk Score in Older Adults with Atrial Fibrillation Treated with Oral Anticoagulant

    by Lily G Bessette, Daniel Singer, Ajinkya Pawar, Vincent Wong, Dae Hyun Kim, Kueiyu Joshua Lin

    Background: High risk of intracranial hemorrhage (ICH) is a leading reason for withholding anticoagulation in patients with atrial fibrillation (AF). We aimed to develop a claims-based ICH risk prediction model in older adults with AF initiating oral anticoagulation (OAC). Methods: We used US Medicare claims data to identify new users of OAC aged ≥ 65 years with AF in 2010– 2017. We used regularized Cox regression to select predictors of ICH. We compared our AF ICH risk score with the HAS-BLED bleed risk and Homer fall risk scores by area under the receiver operating characteristic curve (AUC) and assessed net reclassification improvement (NRI) when predicting 1-year risk of ICH. Results: Our study cohort comprised 840,020 patients (mean [SD] age 77.5 [7.4] years and female 52.2%) split geographically into training (3963 ICH events [0.6%] in 629,804 patients) and validation (1397 ICH events [0.7%] in 210,216 patients) sets. Our AF ICH risk score, including 50 predictors, had superior AUCs of 0.653 and 0.650 in the training and validation sets than the HAS-BLED score of 0.580 and 0.567 (p< 0.001) and the Homer score of 0.624 and 0.623 (p< 0.001). In the validation set, our AF ICH risk score reclassified 57.8%, 42.5%, and 43.9% of low, intermediate, and high-risk patients, respectively, by HAS-BLED score (NRI: 15.3%, p< 0.001). Similarly, it reclassified 0.0, 44.1, and 19.4% of low, intermediate, and high-risk patients, respectively, by the Homer score (NRI: 21.9%, p< 0.001). Conclusion: Our novel claims-based ICH risk prediction model outperformed the standard HAS-BLED score and can inform OAC prescribing decisions.


  • Open Access

    The impact of using reinforcement learning to personalize communication on medication adherence: findings from the REINFORCE trial

    by Julie C. Lauffenburger, Elad Yom-Tov, Punam A. Keller, Marie E. McDonnell, Katherine L. Crum, Gauri Bhatkhande, Ellen S. Sears, Kaitlin Hanken, Lily G. Bessette, Constance P. Fontanet, et al.

    Text messaging can promote healthy behaviors, like adherence to medication, yet its effectiveness remains modest, in part because message content is rarely personalized. Reinforcement learning has been used in consumer technology to personalize content but with limited application in healthcare. We tested a reinforcement learning program that identifies individual responsiveness (“adherence”) to text message content and personalizes messaging accordingly. We randomized 60 individuals with diabetes and glycated hemoglobin A1c [HbA1c] ≥ 7.5% to reinforcement learning intervention or control (no messages). Both arms received electronic pill bottles to measure adherence. The intervention improved absolute adjusted adherence by 13.6% (95%CI: 1.7%–27.1%) versus control and was more effective in patients with HbA1c 7.5- < 9.0% (36.6%, 95%CI: 25.1%–48.2%, interaction p < 0.001). We also explored whether individual patient characteristics were associated with differential response to tested behavioral factors and unique clusters of responsiveness. Reinforcement learning may be a promising approach to improve adherence and personalize communication at scale.


  • Open Access

    Identifying Functional Status Impairment in People Living With Dementia Through Natural Language Processing of Clinical Documents: Cross-Sectional Study

    by John Laurentiev, Dae Hyun Kim, Mufaddal Mahesri, Kuan-Yuan Wang, Lily G Bessette, Cassandra York, Heidi Zakoul, Su Been Lee, Li Zhou, Kueiyu Joshua Lin

    Background: Assessment of activities of daily living (ADLs) and instrumental ADLs (iADLs) is key to determining the severity of dementia and care needs among older adults. However, such information is often only documented in free-text clinical notes within the electronic health record and can be challenging to find. Objective: This study aims to develop and validate machine learning models to determine the status of ADL and iADL impairments based on clinical notes. Methods: This cross-sectional study leveraged electronic health record clinical notes from Mass General Brigham’s Research Patient Data Repository linked with Medicare fee-for-service claims data from 2007 to 2017 to identify individuals aged 65 years or older with at least 1 diagnosis of dementia. Notes for encounters both 180 days before and after the first date of dementia diagnosis were randomly sampled. Models were trained and validated using note sentences filtered by expert-curated keywords (filtered cohort) and further evaluated using unfiltered sentences (unfiltered cohort). The model’s performance was compared using area under the receiver operating characteristic curve and area under the precision-recall curve (AUPRC). Results: The study included 10,000 key-term–filtered sentences representing 441 people (n=283, 64.2% women; mean age 82.7, SD 7.9 years) and 1000 unfiltered sentences representing 80 people (n=56, 70% women; mean age 82.8, SD 7.5 years). Area under the receiver operating characteristic curve was high for the best-performing ADL and iADL models on both cohorts (>0.97). For ADL impairment identification, the random forest model achieved the best AUPRC (0.89, 95% CI 0.86-0.91) on the filtered cohort; the support vector machine model achieved the highest AUPRC (0.82, 95% CI 0.75-0.89) for the unfiltered cohort. For iADL impairment, the Bio+Clinical bidirectional encoder representations from transformers (BERT) model had the highest AUPRC (filtered: 0.76, 95% CI 0.68-0.82; unfiltered: 0.58, 95% CI 0.001-1.0). Compared with a keyword-search approach on the unfiltered cohort, machine learning reduced false-positive rates from 4.5% to 0.2% for ADL and 1.8% to 0.1% for iADL. Conclusions: In this study, we demonstrated the ability of machine learning models to accurately identify ADL and iADL impairment based on free-text clinical notes, which could be useful in determining the severity of dementia.


  • Predicting Treatment Effects of a New‐to‐Market Drug in Clinical Practice Based on Phase III Randomized Trial Results

    by HoJin Shin, Shirley V. Wang, Dae Hyun Kim, Ethan Alt, Mufaddal Mahesri, Lily G. Bessette, Sebastian Schneeweiss, Mehdi Najafzadeh

    Trial results may not be generalizable to target populations treated in clinical practice with different distributions of baseline characteristics that modify the treatment effect. We used outcome models developed with trial data to predict treatment effects in Medicare populations. We used data from the Randomized Evaluation of Long-Term Anticoagulation Therapy trial (RE-LY), which investigated the effect of dabigatran vs. warfarin on stroke or systemic embolism (stroke/SE) among patients with atrial fibrillation. We developed outcome models by fitting proportional hazards models in trial data. Target populations were trial-eligible Medicare beneficiaries who initiated dabigatran or warfarin in 2010-2011 ("early") and 2010-2017 ("extended"). We predicted 2-year risk ratios (RRs) and risk differences (RDs) for stroke/SE, major bleeding, and all-cause death in the Medicare populations using the observed baseline characteristics. The trial and early target populations had similar mean (SD) CHADS(2) scores (2.15 (SD 1.13) vs. 2.15 (SD 0.91)) but different mean ages (71 vs. 79 years). Compared with RE-LY, the early Medicare population had similar predicted benefit of dabigatran vs. warfarin for stroke/SE (trial RR = 0.63, 95% confidence interval (CI) = 0.50 to 0.76 and RD = -1.37%, -1.96% to -0.77%, Medicare RR = 0.73, 0.65 to 0.82 and RD = -0.92%, -1.26% to -0.59%) and risks for major bleeding and all-cause death. The time-extended target population showed similar results. Outcome model-based prediction facilitates estimating the average treatment effects of a drug in different target populations when treatment and outcome data are unreliable or unavailable. The predicted effects may inform payers' coverage decisions for patients, especially shortly after a drug's launch when observational data are scarce.


  • Open Access

    The impact of electronic health record discontinuity on prediction modeling

    by Shreyas Kar, Lily G. Bessette, Richard Wyss, Aaron S. Kesselheim, Kueiyu Joshua Lin

    Background: To determine the impact of electronic health record (EHR)-discontinuity on the performance of prediction models.Methods: The study population consisted of patients with a history of cardiovascular (CV) comorbidities identified using US Medicare claims data from 2007 to 2017, linked to EHR from two networks (used as model training and validation set, respectively). We built models predicting one-year risk of mortality, major CV events, and major bleeding events, stratified by high vs. low algorithm-predicted EHR-continuity. The best-performing models for each outcome were chosen among 5 commonly used machine-learning models. We compared model performance by Area under the ROC curve (AUROC) and Area under the precision-recall curve (AUPRC).Results: Based on 180,950 in the training and 103,061 in the validation set, we found EHR captured only 21.0–28.1% of all the non-fatal outcomes in the low EHR-continuity cohort but 55.4–66.1% of that in the high EHR-continuity cohort. In the validation set, the best-performing model developed among high EHR-continuity patients had consistently higher AUROC than that based on low-continuity patients: AUROC was 0.849 vs. 0.743 when predicting mortality; AUROC was 0.802 vs. 0.659 predicting the CV events; AUROC was 0.635 vs. 0.567 predicting major bleeding. We observed a similar pattern when using AUPRC as the outcome metric.Conclusions: Among patients with CV comorbidities, when predicting mortality, major CV events, and bleeding outcomes, the prediction models developed in datasets with low EHR-continuity consistently had worse performance compared to models developed with high EHR-continuity.


  • Open Access

    Identifying Functional Status Impairment in People Living With Dementia Through Natural Language Processing of Clinical Documents: Cross-Sectional Study (Preprint)

    by John Laurentiev, Dae Hyun Kim, Mufaddal Mahesri, Kuan-Yuan Wang, Lily G Bessette, Cassandra York, Heidi Zakoul, Su Been Lee, Li Zhou, Kueiyu Joshua Lin

    BACKGROUND Assessment of activities of daily living (ADLs) and instrumental ADLs (iADLs) is key to determining the severity of dementia and care needs among older adults. However, such information is often only documented in free-text clinical notes within the electronic health record and can be challenging to find. OBJECTIVE This study aims to develop and validate machine learning models to determine the status of ADL and iADL impairments based on clinical notes. METHODS This cross-sectional study leveraged electronic health record clinical notes from Mass General Brigham’s Research Patient Data Repository linked with Medicare fee-for-service claims data from 2007 to 2017 to identify individuals aged 65 years or older with at least 1 diagnosis of dementia. Notes for encounters both 180 days before and after the first date of dementia diagnosis were randomly sampled. Models were trained and validated using note sentences filtered by expert-curated keywords (filtered cohort) and further evaluated using unfiltered sentences (unfiltered cohort). The model’s performance was compared using area under the receiver operating characteristic curve and area under the precision-recall curve (AUPRC). RESULTS The study included 10,000 key-term–filtered sentences representing 441 people (n=283, 64.2% women; mean age 82.7, SD 7.9 years) and 1000 unfiltered sentences representing 80 people (n=56, 70% women; mean age 82.8, SD 7.5 years). Area under the receiver operating characteristic curve was high for the best-performing ADL and iADL models on both cohorts (>0.97). For ADL impairment identification, the random forest model achieved the best AUPRC (0.89, 95% CI 0.86-0.91) on the filtered cohort; the support vector machine model achieved the highest AUPRC (0.82, 95% CI 0.75-0.89) for the unfiltered cohort. For iADL impairment, the Bio+Clinical bidirectional encoder representations from transformers (BERT) model had the highest AUPRC (filtered: 0.76, 95% CI 0.68-0.82; unfiltered: 0.58, 95% CI 0.001-1.0). Compared with a keyword-search approach on the unfiltered cohort, machine learning reduced false-positive rates from 4.5% to 0.2% for ADL and 1.8% to 0.1% for iADL. CONCLUSIONS In this study, we demonstrated the ability of machine learning models to accurately identify ADL and iADL impairment based on free-text clinical notes, which could be useful in determining the severity of dementia.

  • Scalable Feature Engineering from Electronic Free Text Notes to Supplement Confounding Adjustment of Claims‐Based Pharmacoepidemiologic Studies

    by Richard Wyss, Joseph M. Plasek, Li Zhou, Lily G. Bessette, Sebastian Schneeweiss, Jeremy A. Rassen, Theodore Tsacogianis, Kueiyu Joshua Lin

    Natural language processing (NLP) tools turn free-text notes (FTNs) from electronic health records (EHRs) into data features that can supplement confounding adjustment in pharmacoepidemiologic studies. However, current applications are difficult to scale. We used unsupervised NLP to generate high-dimensional feature spaces from FTNs to improve prediction of drug exposure and outcomes compared with claims-based analyses. We linked Medicare claims with EHR data to generate three cohort studies comparing different classes of medications on the risk of various clinical outcomes. We used "bag-of-words " to generate features for the top 20,000 most prevalent terms from FTNs. We compared machine learning (ML) prediction algorithms using different sets of candidate predictors: Set1 (39 researcher-specified variables), Set2 (Set1 + ML-selected claims codes), and Set3 (Set1 + ML-selected NLP-generated features), vs. Set4 (Set1 + 2 + 3). When modeling treatment choice, we observed a consistent pattern across the examples: ML models utilizing Set4 performed best followed by Set2, Set3, then Set1. When modeling the outcome risk, there was little to no improvement beyond models based on Set1. Supplementing claims data with NLP-generated features from free text notes improved prediction of prescribing choices but had little or no improvement on clinical risk prediction. These findings have implications for strategies to improve confounding using EHR data in pharmacoepidemiologic studies.


  • Comparative effectiveness of Empagliflozin in reducing the burden of recurrent cardiovascular hospitalizations among older adults with diabetes in routine clinical care

    by Rishi J. Desai, Robert J. Glynn, Brendan M. Everett, Sebastian Schneeweiss, Deborah J. Wexler, Lily G. Bessette, Anouk Déruaz-Luyet, Ola Vedin, Kimberly Brodovicz, Elisabetta Patorno


  • Understanding variation in the results of real-world evidence studies that seem to address the same question

    by Shirley V. Wang, Sushama Kattinakere Sreedhara, Lily G. Bessette, Sebastian Schneeweiss