MLMichelle B. Leavy

Michelle B. Leavy

*This is an algorithmically generated author record based on publicly available information.

Michelle B. Leavy has a total of 53 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Performance of a foundation model to generate thousands of predicted outcomes across multiple use cases

    by Matteo Mischiati, Kyriakos Vattis, Richard Gliklich, Michelle Leavy, Costas Boussios, Jigar Bandaria

    Background: Predictive artificial intelligence (AI) models hold great promise for medical care. While many types of models have proliferated across healthcare and the life sciences, systematic, rapid generation of multi-condition predictions of diagnoses, future medical events and future clinical outcomes from a single foundation-model architecture remains rare. PhenOM® is a generative AI foundation model trained and validated on longitudinal medical-event data from more than 400 million patient records in the OM1 Real World Data Cloud (RWDC) and designed to complete predictive tasks for a broad array of medical diagnoses, events and clinical outcomes (collectively ‘outcomes’) at varying time intervals, currently up to five (5) years. This paper describes the implementation of PhenOM to complete a vast number of predictive tasks across clinical domains and outcomes. Methods: PhenOM was evaluated across four outcome categories (Future Risk, Screening, Care Opportunity, and Treatment Effect) in multiple domains spanning chronic disease, acute medical events, healthcare utilization, and rare and frequently undiagnosed conditions. Performance was assessed using area under the receiver operating characteristic curve (AUROC). Outcomes were defined within an eligible-population framework using standardized medical code sets (ICD-10, SNOMED, CPT, NDC, LOINC) unless outcomes could not be specified with an existing code. All benchmarks reported were derived from data excluded from training. Results: As of this publication, the single PhenOM model has been used to complete and validate thousands of predictive tasks covering more than 1,770 outcomes spanning most organ systems and clinical specialties, with new predictive tasks being validated each day. Overall, 80% of predictive tasks achieved AUROC of at least 0.70 and 30% achieved AUROC of at least 0.80. We present a sample of 177 of these tasks as examples that show the breadth of tasks the model can complete. Performance varied systematically by outcome and clinical domain. Conclusions: PhenOM demonstrates the ability to complete thousands of predictive tasks across clinical domains and outcomes at the individual patient level, without additional training. Outcomes include screening for undiagnosed conditions, predicting future conditions, medical events, effects of treatments, and assessing care opportunities. These can be ascertained at the individual patient and population levels. These findings support applications in individual and population health management, risk management, drug development, and real-world evidence generation, and establish a transparent, reproducible performance reference for a large-scale real-world predictive AI platform.

  • Open Access

    Validation of a machine learning approach to estimate expanded disability status scale scores for multiple sclerosis

    by Pedro Alves, Eric Green, Michelle Leavy, Haley Friedler, Gary Curhan, Carl Marci, Costas Boussios

    Background: Disability assessment using the Expanded Disability Status Scale (EDSS) is important to inform treatment decisions and monitor the progression of multiple sclerosis. Yet, EDSS scores are documented infrequently in electronic medical records. Objective: To validate a machine learning model to estimate EDSS scores for multiple sclerosis patients using clinical notes from neurologists. Methods: A machine learning model was developed to estimate EDSS scores on specific encounter dates using clinical notes from neurologist visits. The OM1 MS Registry data were used to create a training cohort of 2632 encounters and a separate validation cohort of 857 encounters, all with clinician-recorded EDSS scores. Model performance was assessed using the area under the receiver-operating-characteristic curve (AUC), positive predictive value (PPV), and negative predictive value (NPV), calculated using a binarized version of the outcome. The Spearman R and Pearson R values were calculated. The model was then applied to encounters without clinician-recorded EDSS scores in the MS Registry. Results: The model had a PPV of 0.85, NPV of 0.85, and AUC of 0.91. The model had a Spearman R value of 0.75 and Pearson R value of 0.74 when evaluating performance using the continuous estimated EDSS and clinician-recorded EDSS scores. Application of the model to eligible encounters resulted in the generation of eEDSS scores for an additional 190,282 encounters from 13,249 patients. Conclusion: EDSS scores can be estimated with very good performance using a machine learning model applied to clinical notes, thus increasing the utility of real-world data sources for research purposes.


  • Open Access

    Validation of a machine learning approach to estimate Clinical Disease Activity Index Scores for rheumatoid arthritis

    by Alison K. Spencer, Jigar Bandaria, Michelle B. Leavy, Benjamin Gliklich, Zhaohui Su, Gary Curhan, Costas Boussios

    Objective Disease activity measures, such as the Clinical Disease Activity Index (CDAI), are important tools for informing treatment decisions and monitoring patient outcomes in rheumatoid arthritis (RA). Yet, documentation of CDAI scores in electronic medical records and other real-world data sources is inconsistent, making it challenging to use these data for research. The purpose of this study was to validate a machine learning model to estimate CDAI scores for patients with RA using clinical notes. Methods A machine learning model was developed to estimate CDAI score values using clinical notes from a specific rheumatology visit. Data from the OM1 RA Registry were used to create a training cohort of 56 177 encounters and a separate validation cohort of 18 726 encounters, 11 985 of which passed a model-derived confidence filter; all included encounters had both a clinician-recorded CDAI score and a clinical note. Model performance was assessed using the area under the receiver operating characteristic curve (AUC), positive predictive value (PPV) and negative predictive value (NPV), calculated using a binarised version of the outcome. The Spearman’s R and Pearson’s R values were also calculated. Results The model had a PPV of 0.80, NPV of 0.84 and AUC of 0.88 when evaluating performance using the binarised version of the outcome. The model had a Spearman’s R value of 0.72 and a Pearson’s R value of 0.69 when evaluating performance using the continuous CDAI numeric scores. Conclusion A machine learning model estimates CDAI scores from clinical notes with good performance. Application of the model to real-world data sets may allow estimated CDAI scores to be used for research purposes.


  • Harmonized Outcome Measures for Use in Non–Small Cell Lung Cancer Patient Registries and Clinical Practice

    by Martin J. Edelman, Daniel P. Raymond, Dwight H. Owen, Michelle B. Leavy, Kari Chansky, Sriram Yennu, Felix G. Fernandez, Carolyn J. Presley, Tithi Biswas, Gwendolyn P. Quinn, et al.

    Background: Lung cancer is the leading cause of cancer-related death in the United States and globally, and many questions exist about treatment options. Harmonizing data across registries and other data collection efforts would yield a robust data infrastructure to help address many research questions. The purpose of this project was to develop a minimum set of patient and clinician relevant harmonized outcome measures that can be collected in non–small cell lung cancer (NSCLC) patient registries and clinical practice. Methods: Seventeen lung cancer registries and related efforts were identified and invited to submit outcome measures. Representatives from medical specialty societies, government agencies, health systems, health information technology groups, patient advocacy organizations, and industry formed a stakeholder panel to categorize the measures and harmonize definitions using the Agency for Healthcare Research and Quality’s supported Outcome Measures Framework (OMF). Results: The panel reviewed 66 outcome measures and identified a minimum set of 8 broadly relevant measures in the OMF categories of patient survival, clinical response, events of interest, and resource utilization. The panel harmonized definitions for the 8 measures through in-person and virtual meetings. The panel did not reach consensus on 1 specific validated instrument for capturing patient-reported outcomes. The minimum set of harmonized outcome measures is broadly relevant to clinicians and patients and feasible to capture across NSCLC disease stages and treatment pathways. A pilot test of these measures would be useful to document the burden and value of the measures for research and in clinical practice. Conclusions: By collecting the harmonized measures consistently, registries and other data collection systems could contribute to the development research infrastructure and learning health systems to support new research and improve patient outcomes.


  • Harmonized outcome measures for use in degenerative lumbar spondylolisthesis patient registries and clinical practice

    by Robert E. Harbaugh, Clinton Devin, Michelle B. Leavy, Zoher Ghogawala, Kristin R. Archer, Mohamad Bydon, Christine Goertz, Doron Dinstein, David R. Nerenz, Guy S. Eakin, et al.

    OBJECTIVE: The development of new treatment approaches for degenerative lumbar spondylolisthesis (DLS) has introduced many questions about comparative effectiveness and long-term outcomes. Patient registries collect robust, longitudinal data that could be combined or aggregated to form a national and potentially international research data infrastructure to address these and other research questions. However, linking data across registries is challenging because registries typically define and capture different outcome measures. Variation in outcome measures occurs in clinical practice and other types of research studies as well, limiting the utility of existing data sources for addressing new research questions. The purpose of this project was to develop a minimum set of patient- and clinician-relevant standardized outcome measures that are feasible for collection in DLS registries and clinical practice. METHODS: Nineteen DLS registries, observational studies, and quality improvement efforts were invited to participate and submit outcome measures. A stakeholder panel was organized that included representatives from medical specialty societies, health systems, government agencies, payers, industries, health information technology organizations, and patient advocacy groups. The panel categorized the measures using the Agency for Healthcare Research and Quality’s Outcome Measures Framework (OMF), identified a minimum set of outcome measures, and developed standardized definitions through a consensus-based process. RESULTS: The panel identified and harmonized 57 outcome measures into a minimum set of 10 core outcome measure areas and 6 supplemental outcome measure areas. The measures are organized into the OMF categories of survival, clinical response, events of interest, patient-reported outcomes, and resource utilization. CONCLUSIONS: This effort identified a minimum set of standardized measures that are relevant to patients and clinicians and appropriate for use in DLS registries, other research efforts, and clinical practice. Collection of these measures across registries and clinical practice is an important step for building research data infrastructure, creating learning healthcare systems, and improving patient management and outcomes in DLS.


  • Open Access

    Validation of a machine learning approach to estimate Systemic Lupus Erythematosus Disease Activity Index score categories and application in a real-world dataset

    by Pedro Alves, Jigar Bandaria, Michelle B Leavy, Benjamin Gliklich, Costas Boussios, Zhaohui Su, Gary Curhan

    Objective Use of the Systemic Lupus Erythematosus Disease Activity Index (SLEDAI) in routine clinical practice is inconsistent, and availability of clinician-recorded SLEDAI scores in real-world datasets is limited. This study aimed to validate a machine learning model to estimate SLEDAI score categories using clinical notes and to apply the model to a large, real-world dataset to generate estimated score categories for use in future research studies. Methods A machine learning model was developed to estimate an individual patient’s SLEDAI score category (no activity, mild activity, moderate activity or high/very high activity) for a specific encounter date using clinical notes. A training cohort of 3504 encounters and a separate validation cohort of 1576 encounters were created from the OM1 SLE Registry. Model performance was assessed using the area under the receiver operating characteristic curve (AUC), calculated using a binarised version of the outcome that sets the positive class to be those records with clinician-recorded SLEDAI scores >5 and the negative class to be records with scores ≤5. Model performance was evaluated by categorising the scores into the four disease activity categories and by calculating the Spearman’s R value and Pearson’s R value. Results The AUC for the two categories was 0.93 for the development cohort and 0.91 for the validation cohort. The model had a Spearman’s R value of 0.7 and a Pearson’s R value of 0.7 when calculated using the four disease activity categories. Conclusion The model performs well when estimating SLEDAI score categories using unstructured clinical notes.


  • Usingreal‐worldevidence to support a changing paradigm for cancer screening: A commentary

    by Michelle B. Leavy, Kathryn Starzyk, Evan Myers, Gary Curhan, Richard Gliklich


  • Assessing Real-World Data Quality: The Application of Patient Registry Quality Criteria to Real-World Data and Real-World Evidence

    by Richard E. Gliklich, Michelle B. Leavy

    The use of real-world data and real-world evidence to inform health care decisions is increasing. Yet, the variable quality of these data and the lack of widely-accepted criteria by which to assess quality create uncertainty about how and when to use these data and the associated evidence in decision making. Patient registries are an important source of real-world data and real-world evidence. The good practices and evaluation criteria developed for patient registries are highly relevant to real-world data and real-world evidence and offer a foundation for a unified set of quality criteria that can be applied across sources of real-world data and real-world evidence intended for use in medical product evaluation.


  • Assessing Real-World Data Quality: The Application of Patient Registry Quality Criteria to Real-World Data and Real-World Evidence

    by Richard E. Gliklich, Michelle B. Leavy

    The use of real-world data and real-world evidence to inform health care decisions is increasing. Yet, the variable quality of these data and the lack of widely-accepted criteria by which to assess quality create uncertainty about how and when to use these data and the associated evidence in decision making. Patient registries are an important source of real-world data and real-world evidence. The good practices and evaluation criteria developed for patient registries are highly relevant to real-world data and real-world evidence and offer a foundation for a unified set of quality criteria that can be applied across sources of real-world data and real-world evidence intended for use in medical product evaluation.