Christopher A. Harle

Background: Clinical decision support (CDS) systems may improve chronic pain management by providing guideline-based recommendations within clinical workflows. However, low clinician use can limit their effectiveness. Understanding the association between CDS use and guideline-recommended care may help distinguish implementation challenges from limitations in CDS functionality. Objective: To estimate the association of OneSheet, a CDS system for chronic pain management, and guideline-recommended care when the system was used during patient encounters. Methods: We conducted a per-protocol secondary data analysis of data from a pragmatic randomized clinical trial conducted from October 2020 to May 2022 across 25 primary care clinics at 2 academic health systems in Indiana and North Carolina. The parent trial enrolled 137 primary care clinicians. This analysis included 145511 eligible chronic pain encounters and 15480 long term opioid therapy (LTOT) encounters that occurred after OneSheet implementation. OneSheet use at a patient encounter was the exposure of interest. Propensity score matching was used to create comparable groups of encounters with and without OneSheet use. Outcomes included documentation of pain-related goals, documentation of pain and function using the Pain, Enjoyment of Life and General Activity (PEG) scale, prescription drug monitoring (PDMP) review, urine drug screening (UDS) orders, and naloxone orders. Results: OneSheet was used in 959 of 145511 chronic pain encounters (0.7%) and 582 of 15480 LTOT encounters (4.0%). In matched analyses, OneSheet use was associated with increased documentation of pain-related goals (relative risk (RR)=11.70 (95% CI 4.25-32.50) and PEG score documentation (RR=9.70 (95% CI 4.14-22.31). Among LTOT encounters, OneSheet use was associated with increased UDS ordering (RR=1.52 (95% CI 1.40-1.65) and PDMP review (RR=1.06 (95% CI 1.03-1.09), but decreased naloxone ordering (RR=0.13 (95% CI, 0.02-0.90). Conclusions: OneSheet use was associated with several guideline-recommended chronic pain management actions, suggesting that they system may support care quality when clinicians use it. However, use was rare, indicating that increasing routine use through better workflow integration may be necessary to achieve population-level improvements in chronic pain care.

Background: Opioid use disorder (OUD) remains a critical public health crisis in the United States. Despite widespread policy and clinical interventions, early identification of individuals at risk for developing OUD remains challenging due to limitations in traditional screening approaches and a lack of individualized risk stratification methods. Machine learning (ML) methods offer an opportunity to develop timely, high-performing, and explainable predictive models that can enhance OUD prevention strategies in clinical settings. Objective: This study aims to develop and validate an ML model using electronic health record (EHR) data to predict the 3-month risk of incident OUD among adults initiating opioid therapy and to stratify patients into clinically actionable risk groups. Methods: This prognostic modeling study used 2017‐2022 OneFlorida+ EHR data to develop and validate ML algorithms predicting 3-month incident OUD risk. We included 182,083 adults (≥18 y) without cancer, overdose, or OUD or hospice history who received ≥1 outpatient, noninjectable opioid prescription. Using 183 predictors measured in sequential 3-month intervals, we developed an elastic net, least absolute shrinkage and selection operator, gradient boosting machine (GBM), and random forest models on randomly split training, testing, and validation sets. Model performance was assessed using C-statistics, predictive values, and number needed to evaluate, with patients stratified into risk deciles for clinical applicability. Model explainability was assessed using Shapley additive explanations, and fairness was evaluated using standard metrics. We externally validated the best-performing model using an independent cohort from the 2018‐2020 UPMC (formerly University of Pittsburgh Medical Center) health system. Results: In the validation sample (n=60,694), GBM (C-statistics=0.879, 95% CI 0.874‐0.884) and elastic net (C-statistics=0.872, 95% CI 0.867‐0.877) outperformed least absolute shrinkage and selection operator (C-statistics=0.846, 95% CI 0.840‐0.851) and random forest (C-statistics=0.798, 95% CI 0.792‐0.804), with GBM model requiring the fewest predictors (n=75) for predicting 3-month incident OUD. Using the GBM algorithm to predict the subsequent 3-month OUD risk, the top decile subgroup had a positive predictive value of 3.26%, a negative predictive value of 99.8%, and a number needed to evaluate of 31. The top decile (n=6696) captured ~68% of patients with OUD. Shapley additive explanations analysis identified age, number of outpatient visits, history of back and other pain conditions, comorbidity burden, and opioid prescribing patterns as the strongest predictors of incident OUD. Fairness assessment showed an acceptable false negative rate parity across race, age, and sex. In external validation on the UPMC cohort, the GBM model maintained good discrimination (C-statistics=0.756, 95% CI 0.750‐0.762) and effective risk stratification. Conclusions: An ML algorithm predicting incident OUD derived from OneFlorida+ EHR data performed well in external validation with data using UPMC. The algorithm might be valuable for incident OUD risk prediction and stratification across health systems, with potential to inform early intervention.

Background/Objectives: The Developing and Evaluating a Machine Learning Opioid Prediction & Risk-Stratification E-Platform (DEMONSTRATE) trial aims to assess the usability, acceptability, feasibility, and effectiveness of implementing a machine learning (ML)-based clinical decision support (CDS) tool—the Overdose Prevention Alert—which predicts a patient’s risk of opioid overdose within three months. Methods: This single-arm study uses a pre–post implementation design with mixed-methods evaluation in 13 University of Florida Health, Gainesville, internal medicine and family medicine clinics. Eligible patients are aged ≥18 years, received an opioid prescription within the year prior to their upcoming primary care visit, are not receiving hospice care, do not have a malignant cancer diagnosis, and are identified by the ML algorithm as high risk for overdose. The Overdose Prevention Alert triggers when a primary care provider (PCP) signs an opioid order in electronic health records. We will evaluate effectiveness by comparing pre- and post-implementation outcomes using a composite patient-level measure defined by the presence of any of the following 6 favorable indicators: (1) evidence of naloxone access; (2) absence of opioid overdose diagnoses and naloxone administration; (3) absence of emergency department (ED) visits or hospitalizations due to opioid overdose or opioid use disorder (OUD); (4) absence of overlapping opioid and benzodiazepine use within a 7-day window; (5) absence of opioid use ≥50 morphine milligram equivalent daily average; (6) receipt of referrals to non-pharmacological pain management. Additional quantitative metrics will include alert penetration, usage patterns, and clinical actions taken. Usability and acceptability will be assessed using a 12-item questionnaire for PCPs and semi-structured interviews. Expected Results: The trial will provide insights into real-world ML-driven CDS implementation and inform future strategies to reduce opioid-related harm.

Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Chronic pain management in older adults can be challenging for primary care clinicians because of comorbidities, side effects, and complicated guideline recommendations. Clinical decision support systems (CDSS) can enhance guideline adherence in chronic pain management by collecting and organizing patient information and aiding clinician decision-making. This study examined clinicians’ views on challenges in managing chronic pain and their opinions on a CDSS that gathered patient preferences and provided clinicians with decision support for chronic pain management. Objective: The objective of this study was to explore primary care clinicians’ perspectives on the challenges of managing chronic pain in older adults and evaluate their opinions on a clinical decision support system (CDSS) designed to gather patient preferences and facilitate guideline-based, multimodal pain management. Methods: We conducted semi-structured interviews with 18 clinicians from two University of Chicago Medicine primary care clinics piloting the CDSS. The interview guide was informed by the Consolidated Framework for Implementation Research. Results: Participants included 89% physicians and 11% advanced practice nurses. Participants stressed the importance of a comprehensive, patient-centered approach to chronic pain management and favored multimodal and non-pharmacological treatments. Challenges included complex medical histories, competing priorities, insurance limitations, and opioid misuse concerns. Clinicians found the CDSS beneficial for promoting multimodal care discussions and enhancing visit efficiency. However, there were concerns regarding its complexity, workflow compatibility, and older patients' technology navigation difficulties. While tools, such as the pre-visit questionnaire and conversation tool, were valued, clinicians emphasized the need for adaptability and streamlined usability. Conclusions: The primary care clinicians in this study were aligned with clinical practice guidelines to provide patient-centered pain management using multimodal treatments. However, they had several concerns regarding how complex chronic pain management can be for older adult patients. They expressed interest in using the CDSS but were concerned about its complexity. I-COPE offers a promising approach to support guideline-based chronic pain and opioid management in primary care. By addressing usability and workflow compatibility, CDSS tools like I-COPE can better equip clinicians to provide comprehensive, patient-centered care, ultimately enhancing treatment outcomes for older adults with chronic pain.

BACKGROUND Chronic pain management in older adults can be challenging for primary care clinicians because of comorbidities, side effects, and complicated guideline recommendations. Clinical decision support systems (CDSS) can enhance guideline adherence in chronic pain management by collecting and organizing patient information and aiding clinician decision-making. This study examined clinicians’ views on challenges in managing chronic pain and their opinions on a CDSS that gathered patient preferences and provided clinicians with decision support for chronic pain management. OBJECTIVE The objective of this study was to explore primary care clinicians’ perspectives on the challenges of managing chronic pain in older adults and evaluate their opinions on a clinical decision support system (CDSS) designed to gather patient preferences and facilitate guideline-based, multimodal pain management. METHODS We conducted semi-structured interviews with 18 clinicians from two University of Chicago Medicine primary care clinics piloting the CDSS. The interview guide was informed by the Consolidated Framework for Implementation Research. RESULTS Participants included 89% physicians and 11% advanced practice nurses. Participants stressed the importance of a comprehensive, patient-centered approach to chronic pain management and favored multimodal and non-pharmacological treatments. Challenges included complex medical histories, competing priorities, insurance limitations, and opioid misuse concerns. Clinicians found the CDSS beneficial for promoting multimodal care discussions and enhancing visit efficiency. However, there were concerns regarding its complexity, workflow compatibility, and older patients' technology navigation difficulties. While tools, such as the pre-visit questionnaire and conversation tool, were valued, clinicians emphasized the need for adaptability and streamlined usability. CONCLUSIONS The primary care clinicians in this study were aligned with clinical practice guidelines to provide patient-centered pain management using multimodal treatments. However, they had several concerns regarding how complex chronic pain management can be for older adult patients. They expressed interest in using the CDSS but were concerned about its complexity. I-COPE offers a promising approach to support guideline-based chronic pain and opioid management in primary care. By addressing usability and workflow compatibility, CDSS tools like I-COPE can better equip clinicians to provide comprehensive, patient-centered care, ultimately enhancing treatment outcomes for older adults with chronic pain.

Objective: Measurement of health-related social needs (HRSNs) is complex. We sought to develop and validate computable phenotypes (CPs) using structured electronic health record (EHR) data for food insecurity, housing instability, financial insecurity, transportation barriers, and a composite-type measure of these, using human-defined rule-based and machine learning (ML) classifier approaches. Materials and Methods: We collected HRSN surveys as the reference standard and obtained EHR data from 1550 patients in 3 health systems from 2 states. We followed a Delphi-like approach to develop the human-defined rule-based CP. For the ML classifier approach, we trained supervised ML (XGBoost) models using 78 features. Using surveys as the reference standard, we calculated sensitivity, specificity, positive predictive values, and area under the curve (AUC). We compared AUCs using the Delong test and other performance measures using McNemar's test, and checked for differential performance. Results: Most patients (63%) reported at least one HRSN on the reference standard survey. Human-defined rule-based CPs exhibited poor performance (AUCs=.52 to .68). ML classifier CPs performed significantly better, but still poor-to-fair (AUCs = .68 to .75). Significant differences for race/ethnicity were found for ML classifier CPs (higher AUCs for White non-Hispanic patients). Important features included number of encounters and Medicaid insurance. Discussion: Using a supervised ML classifier approach, HRSN CPs approached thresholds of fair performance, but exhibited differential performance by race/ethnicity. Conclusion: CPs may help to identify patients who may benefit from additional social needs screening. Future work should explore the use of area-level features via geospatial data and natural language processing to improve model performance.

To highlight the often overlooked role of user interface (UI) design in mitigating bias in artificial intelligence (AI)-based clinical decision support (CDS). This perspective paper discusses the interdependency between AI-based algorithm development and UI design and proposes strategies for increasing the safety and efficacy of CDS. The role of design in biasing user behavior is well documented in behavioral economics and other disciplines. We offer an example of how UI designs play a role in how bias manifests in our machine learning-based CDS development. Much discussion on bias in AI revolves around data quality and algorithm design; less attention is given to how UI design can exacerbate or mitigate limitations of AI-based applications. This work highlights important considerations including the role of UI design in reinforcing/mitigating bias, human factors methods for identifying issues before an application is released, and risk communication strategies.

Background Health-related social needs (HRSNs) are the unmet social and economic needs (e.g., housing instability) that affect individuals' health and well-being. HRSNs are associated with more emergency department (ED) visits, longer stays, and worse health outcomes. More than a third of ED patients have at least one HRSN, yet patients are rarely screened for HRSNs in the ED. A clinical decision support (CDS) system with predictive modeling offers a promising approach to identifying patients systematically and efficiently with HRSNs in the ED. Objective This study aimed to identify ED clinician and staff preferences for designing and implementing an HRSN-related CDS system. Methods A multistep, user-centered design study involving qualitative semistructured interviews, observations of ED workflows, and a multidisciplinary design workshop. Results We conducted 16 semistructured interviews with ED clinicians and staff. Following the interviews, three research team members observed ED workflows, focusing on patient entry and clinician and staff usage of the electronic health record (EHR) system. Finally, we conducted a 3-hour multidisciplinary design workshop. An HRSN-related CDS system should be visually appealing, color-coordinated, and easily accessible in the EHR. An HRSN-related CDS system should target a select group of ED patients (to be discharged from the ED) and highlight a select set of critical HRSN issues early in the workflow to adjust clinical care adequately. An HRSN-related CDS system should provide a list of actions and the ability to notify the clinical team if the patient's HRSNs were addressed. Conclusion The user-centered design identified a set of specific preferences for an HRSN-related CDS system to be implemented in the ED. Future work will focus on implementing and refining the CDS system and assessing the rates of changes in clinical care (e.g., rates of referrals) to address patient HRSNs in the ED.

Background/Objectives: Concurrent opioid (OPI) and benzodiazepine (BZD) use may exacerbate injurious fall risk (e.g., falls and fractures) compared to no use or use alone. Yet, patients may need concurrent OPI-BZD use for co-occurring conditions (e.g., pain and anxiety). Therefore, we examined the association between longitudinal OPI-BZD dosing patterns and subsequent injurious fall risk. Methods: We conducted a retrospective cohort study including non-cancer fee-for-service Medicare beneficiaries initiating OPI and/or BZD in 2016–2018. We identified OPI-BZD use patterns during the 3 months following OPI and/or BZD initiation (i.e., trajectory period) using group-based multi-trajectory models. We estimated the time to first injurious falls within the 3-month post-trajectory period using inverse-probability-of-treatment-weighted Cox proportional hazards models. Results: Among 622,588 beneficiaries (age ≥ 65 = 84.6%, female = 58.1%, White = 82.7%; having injurious falls = 0.45%), we identified 13 distinct OPI-BZD trajectories: Group (A): Very-low OPI-only (early discontinuation) (44.9% of the cohort); (B): Low OPI-only (rapid decline) (15.1%); (C): Very-low OPI-only (late discontinuation) (7.7%); (D): Low OPI-only (gradual decline) (4.0%); (E): Moderate OPI-only (rapid decline) (2.3%); (F): Very-low BZD-only (late discontinuation) (11.5%); (G): Low BZD-only (rapid decline) (4.5%); (H): Low BZD-only (stable) (3.1%); (I): Moderate BZD-only (gradual decline) (2.1%); (J): Very-low OPI (rapid decline)/Very-low BZD (late discontinuation) (2.9%); (K): Very-low OPI (rapid decline)/Very-low BZD (increasing) (0.9%); (L): Very-low OPI (stable)/Low BZD (stable) (0.6%); and (M): Low OPI (gradual decline)/Low BZD (gradual decline) (0.6%). Compared with Group (A), six trajectories had an increased 3-month injurious falls risk: (C): HR = 1.78, 95% CI = 1.58–2.01; (D): HR = 2.24, 95% CI = 1.93–2.59; (E): HR = 2.60, 95% CI = 2.18–3.09; (H): HR = 2.02, 95% CI = 1.70–2.40; (L): HR = 2.73, 95% CI = 1.98–3.76; and (M): HR = 1.96, 95% CI = 1.32–2.91. Conclusions: Our findings suggest that 3-month injurious fall risk varied across OPI-BZD trajectories, highlighting the importance of considering both dose and duration when assessing injurious fall risk of OPI-BZD use among older adults.