Katy Trinkley

UNSTRUCTURED Artificial intelligence is accelerating healthcare intervention development and implementation, compressing learning cycles. Without corresponding advances in learning methods, health systems risk missing opportunities for improvement or deploying interventions without sufficient inquiry to ensure appropriateness and safety. We propose a new operating model for health system learning that tailors the timing and intensity of inquiry to intervention risks and opportunities for safe learning while embedding rigorous, continuous evaluation within care delivery.

Medical educators frequently face the challenge of scaling successful pilot innovations while ensuring comparable experiences and outcomes at multiple sites and maintaining success over time. Conte...

Background: Theories, models, and frameworks (TMFs) are frequently used to facilitate rigorous qualitative data collection and analysis of context in dissemination and implementation (D&I) science. The Practical, Robust Implementation and Sustainability Model (PRISM), which includes contextual determinants of Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) dimensions, is one of the most widely used TMFs. Yet, missing from the literature is an assessment of how PRISM can inform the collection of qualitative data across implementation phases (i.e., pre-implementation, implementation, and post-implementation). The purpose of this study was to curate a collection of PRISM-informed qualitative data collection guides and assess how questions relate to PRISM domains and RE-AIM dimensions. Methods: In this retrospective observational study, exemplar interview and focus group guides were collected from a group of D&I science experts to assess how PRISM constructs were operationalized. All guides were compiled into a database, and each question and sub-question were labelled with one or more relevant PRISM domains and RE-AIM dimensions. We calculated descriptive characteristics of all interview guides included in the database. The frequency of each PRISM domain was calculated across all interview guides, by implementation phase and by participant role. Results: Guides (n=31) were collected from 13 research studies and were primarily used in individual interviews (n=30) and across pre-implementation (n=8), implementation (n=8), and post-implementation (n=15) phases. Research settings included the Veterans Health Administration (n=16), community health settings (n=10), schools (n=4), and an academic health center (n=1). Questions and sub-questions were more commonly labelled with PRISM domains (n=667) than RE-AIM dimensions (n=303). The Perspectives of Implementers (n=201), Perspectives of Recipients (n=152) and Implementation and Sustainability Infrastructure (n=103) were the most frequently used PRISM domains and Implementation (n=113) was the most frequent RE-AIM dimension. Conclusions: Our findings demonstrate that PRISM has valuable applications in qualitative data collection with recipients and implementers across all implementation phases and highlights how PRISM can be adapted for different topics and settings. The database of qualitative guides is publicly available and can be used as a resource for D&I investigators using PRISM to guide their qualitative contextual assessment.

Background: Primary care clinics need effective, low-cost tools to screen for and address multiple health risks — and technology can provide important efficiencies. My Own Health Report (MOHR) is a web-based intervention with 3 core functions: (1) risk flagging; (2) goal-setting for patient-prioritized risks; and (3) providing resources. A prior randomized trial in primary care settings showed that MOHR was feasible and effective. However, we identified two key gaps : a need for a flexible menu of delivery options to adopt and sustain MOHR and a pragmatic way to provide structured follow-up. Methods/design: We developed and pilot-tested two distinct primary implementation strategies to provide structured follow-up: the first primary strategy, termed “Reminder-Resource message ( R ² message) ,” involves automated motivational text message nudges to remind patients of their MOHR goal and point to resources. The second, “R ² navigation ,” involves personal support - community health worker phone calls to deliver reminders and resources. Each primary strategy is paired with additional secondary strategies that together address the contextual determinants of successful implementation and sustainment identified using our Pragmatic Robust Implementation and Sustainability Model (PRISM). We engaged clinic staff, patients, and community partners to finalize the R ² message and R ² navigation strategies and secondary strategies. This individually randomized, type 2 hybrid effectiveness-implementation trial will enroll 1,000 adult patients with at least two cancer risks (including both insufficient physical activity and fruit/vegetable intake). We will evaluate the outcomes and cost-effectiveness of R ² message and R ² navigation alone or in combination in 12 primary care clinics including both (a) large metropolitan health system and (b) rural-serving clinics. Expected outcomes include: (1) improvement in behavioral risk factors; (2) implementation outcomes – including representative engagement with R ² message and R ² navigation, and cost; and (3) improvement in practice value outcomes, including patient experience ratings. Based on the trial findings, we will co-develop an implementation, adaptation and sustainment guide for the most cost-effective implementation strategy. Conclusions: We expect the implementation strategies compared will produce different levels of representative engagement, behavior change, cost, and practice value. These findings will inform pragmatic ways to improve outcomes important to patients, primary care, and society. Trial Registration# at ClinicalTrials.gov: NCT07569224 (Registration date: 04/29/2026).

BACKGROUND Clinical decision support (CDS) tools within electronic health records have the potential to improve evidence-based care but often fall short. One potential approach to improve CDS effectiveness is to extend CDS tools to include clinician-specific personalization by leveraging data on clinician behavior in the EHR. OBJECTIVE The objective of this study is to determine clinician attitudes towards CDS tools in general, review common concerns, and anticipate in what ways a clinician-personalized CDS approach can solve issues clinicians have with CDS tools. METHODS Using the case example of guideline directed medical treatment for heart failure, we conducted a qualitative study using semi-structured interviews with outpatient clinicians in primary care and cardiology settings. Participants were purposively sampled to capture diverse perspectives across specialties, practice settings, and clinician types. Interviews were analyzed using a pragmatic thematic approach. RESULTS Nineteen clinicians participated. Across specialties, clinicians viewed CDS as a valuable cognitive support tool, particularly for managing complex guidelines and reducing mental workload. However, traditional CDS were consistently described as burdensome due to high alert volume, generic content, poor timing, and workflow disruption, contributing to alert fatigue and reduced trust. Clinician-personalized CDS was endorsed as a potential solution, with participants highlighting its ability to tailor information based on clinician knowledge and practice patterns, reduce redundant content, and enhance relevance. However, clinicians also raised concerns regarding the ability of algorithms to capture clinical nuance, the validity and transparency of clinician-level data, and the risk of perceived loss of autonomy or punitive feedback. CONCLUSIONS Clinician-personalized CDS represents a promising extension of current CDS approaches by incorporating clinician context to improve relevance and usability. While early perceptions are favorable, important challenges related to trust, feasibility, and implementation complexity remain. Further evaluation is needed to determine whether clinician-personalization improves clinical outcomes and clinician experience, and to identify when and how this approach provides meaningful value within health systems.

Artificial intelligence (AI) is quickly changing health care and expanding research opportunities, including in pharmacy. AI can offer effective solutions for numerous challenges in pharmacy research, such as persistent gaps in evidence‐based prescribing, slow drug discovery‐to‐adoption timelines, complex data environments, and the need for more efficient methods, including qualitative methods. AI tools—ranging from machine learning models to large language models (LLMs)—offer new capabilities for data analysis, prediction, and content generation. Pharmacy researchers can harness these technologies to address persistent gaps in medication management, optimize patient safety and outcomes, and streamline many research workflows. However, successful integration requires understanding the spectrum of AI capabilities, appropriate use cases, and potential limitations. AI must be tailored to the research question, data, and institutional resources. This paper will guide pharmacy researchers in identifying use cases for AI in their research, selecting and applying AI tools, mitigating risks, providing standards for reporting the use of AI in research, and maximizing benefits for scientific discovery and patient outcomes.

Background: Translating evidence-based therapies from “bench to bedside” remains challenging, and implementation science (IS) experts are crucial for this process. Qualitative analyses are essential, but require extensive time and cost for manual coding. Now, many turn to artificial intelligence (AI) to accelerate the pace of qualitative analysis, but significant questions remain about the quality, validity, and ethics of applying large language models like ChatGPT (OpenAI) to qualitative data. To this end, we have developed a method for AI-assisted rapid qualitative analysis that addresses these concerns. Objective: This study aimed to develop AI-assisted rapid qualitative analysis for implementation science as an open-source encoder-based small language model (SLM) to aid IS experts. We focus on 2 efficient and high-performing SLMs: distilled bidirectional encoder representations from transformers (DistilBERT) and efficiently learning an encoder that classifies token replacements accurately (ELECTRA). The objective is to assess these models’ accuracy in reproducing expert coding, their generalizability to new coding scenarios, and enhancing their accessibility for nontechnical experts through user-friendly tools. Methods: Two previously coded IS datasets were used to train DistilBERT and ELECTRA models. These datasets were coded by IS experts using a mixed deductive and inductive approach, with initial categories derived from the domains of an IS framework: Practical, Robust Implementation, and Sustainability Model. We fine-tuned and evaluated DistilBERT and ELECTRA on these datasets, measuring performance by area under the precision-recall curve and Cohen κ. To facilitate use by nonprogrammers, we then developed an open-source Python package (pytranscripts) to streamline transcript processing, model classification, and evaluation. Additionally, a companion Streamlit web application allows users to upload interview transcripts and obtain automated coding and analytics without any coding expertise. Results: Our findings demonstrate the success of leveraging SMLs to significantly accelerate qualitative analysis while maintaining high levels of accuracy and agreement with human annotators, although results are not universal and depend on how researchers approach qualitative coding. On the original dataset, DistilBERT achieved near-perfect agreement with human coders (Cohen κ=0.95), while ELECTRA showed substantial agreement (Cohen κ=0.71). However, both models’ performance declined on the second, more ambiguous dataset, with DistilBERT’s Cohen κ dropping to 0.48 and ELECTRA’s to 0.39. Two primary drivers of performance drop appear to be related to the number of codes applied to the dataset, and whether coders apply multiple codes to each piece of data or constrain themselves to applying one. Conclusions: This work demonstrates that SLMs can meaningfully assist qualitative researchers with coding tasks as long as attention is paid to how experts code data that will train the SLM. This can be especially valuable in settings where deploying large language models is impractical or undesirable.

Background: Embedded pragmatic clinical trials (ePCTs) are conducted as part of routine care, which provides researchers and health systems multiple opportunities to study implementation processes and outcomes. Methods: We conducted a cross-sectional survey of 32 ePCTs associated with the NIH Pragmatic Trials Collaboratory to assess the implementation-related outcomes that were measured or were planned to be measured, including reach (number and percent of eligible patients who participate in an intervention and the representativeness of those patients), patient engagement in or adherence to the intervention, adoption (number and percent of eligible organizations or clinicians that decide to take up or use an intervention), fidelity (clinician’s delivery of an intervention as intended), adaptations (changes or modifications to an intervention), sustainability (potential for an intervention to be maintained or institutionalized after a trial concludes), sustainment (actual maintenance or institutionalization of an intervention after a trial concludes), and costs. The trials represented different phases of progress (planned, ongoing, or completed). Results: 91% of study teams completed the survey, and most (86%) reported measuring reach. The total number of teams measuring other outcomes was 76% for adherence, 45% for clinician adoption, 93% for fidelity, 69% for adaptations, 24% for sustainability, 38% for sustainment, and 31% for costs. Conclusion: There is an opportunity for growth in measuring clinician adoption of the intervention, sustainability, sustainment, and associated costs. Measurement of these constructs in future ePCTs could result in development of improved implementation strategies to increase the likelihood of effective implementation leading to equitable, sustainable, and scalable improvement in practice.