KKKelli Keats

Kelli Keats

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Kelli Keats has a total of 46 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Large language models management of complex medication regimens: a case-based evaluation

    by Steven Xu, Amoreena Most, Aaron Chase, Tanner Hedrick, Brian Murray, Kelli Keats, Susan Smith, Erin Barreto, Tianming Liu, Andrea Sikora

    Background Large language models (LLMs) have shown capability in diagnosing complex medical cases and passing medical licensing exams, but to date, only limited evaluations have studied how LLMs interpret, analyze, and optimize complex medication regimens. The purpose of this evaluation was to test four LLMs ability to identify medication errors and appropriate medication interventions on complex patient cases from the intensive care unit (ICU). Methods A series of eight patient cases were developed by critical care pharmacists including history of present illness, laboratory values, vital signs, and medication regimens. Then, four LLMs (ChatGPT (GPT-3.5), ChatGPT (GPT-4), Claude2, and Llama2-7b) were prompted to develop a medication regimen for the patient. LLM generated medication regimens were then reviewed by a panel of seven critical care pharmacists to assess for presence of medication errors and clinical relevance. For each medication regimen recommended by the LLM, clinicians were asked to assess for if they would continue a medication, identify perceived medication errors in the medications recommended, identify the presence of life-threatening medication choices, and rank overall agreement on a 5-point Likert scale. Results The clinician panel rated to continue therapies recommended by the LLMs between 55.8-67.9% of the time. Clinicians perceived between 1.57-4.29 medication errors per recommended regimen, and life-threatening recommendations were present between 15.0-55.3% of the time. Level agreement was between 1.85-2.67 for the four LLMs. Conclusions LLMs demonstrated potential to serve as clinical decision support for the management of complex medication regimens with further domain specific training; however, caution should be used when employing LLMs for medication management given the present capabilities.


  • Queue the quarter life crisis: The value of mentorship for early career pharmacists

    by Pam M Ku, Kelli Keats

    In an effort to expedite the publication of articles, AJHP is posting manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time.


  • Open Access

    Unsupervised machine learning analysis to identify patterns of ICU medication use for fluid overload prediction

    by Kelli Keats, Shiyuan Deng, Xianyan Chen, Tianyi Zhang, John W. Devlin, David J. Murphy, Susan E. Smith, Brian Murray, Rishikesan Kamaleswaran, Andrea Sikora

    INTRODUCTION Intravenous (IV) medications are a fundamental cause of fluid overload (FO) in the intensive care unit (ICU); however, the association between IV medication use (including volume), administration timing, and FO occurrence remains unclear. METHODS This retrospective cohort study included consecutive adults admitted to an ICU ≥72 hours with available fluid balance data. FO was defined as a positive fluid balance ≥7% of admission body weight within 72 hours of ICU admission. After reviewing medication administration record (MAR) data in three-hour periods, IV medication exposure was categorized into clusters using principal component analysis (PCA) and Restricted Boltzmann Machine (RBM). Medication regimens of patients with and without FO were compared within clusters to assess for temporal clusters associated with FO using the Wilcoxon rank sum test. Exploratory analyses of the medication cluster most associated with FO for medications frequently appearing and used in the first 24 hours was conducted. RESULTS FO occurred in 127/927 (13.7%) of the patients enrolled. Patients received a median (IQR) of 31 (13-65) discrete IV medication administrations over the 72-hour period. Across all 47,803 IV medication administrations, ten unique IV medication clusters were identified with 121-130 medications in each cluster. Among the ten clusters, cluster 7 had the greatest association with FO; the mean number of cluster 7 medications received was significantly greater in patients in the FO cohort compared to patients who did not experience FO (25.6 vs.10.9. p 5 separate 3-hour periods during the 72-hour study window. The most common cluster 7 medications included continuous infusions, antibiotics, and sedatives/analgesics. Addition of cluster 7 medications to a prediction model with APACHE II score and receipt of diuretics improved the ability for the model to predict fluid overload (AUROC 5.65, p =0.0004). CONCLUSIONS Using ML approaches, a unique IV medication cluster was strongly associated with FO. Incorporation of this cluster improved the ability to predict development of fluid overload in ICU patients compared with traditional prediction models. This method may be further developed into real-time clinical applications to improve early detection of adverse outcomes. Questions Can machine learning detect the presence of time-dependent medication administration patterns that are associated with risk of fluid overload in critically ill patients? Findings Using unsupervised machine learning, a unique IV medication cluster was identified that, when combined with the APACHE II score and diuretic use, improved the ability to predict fluid overload in ICU patients. Meaning These findings suggest that machine learning may be an important tool for analyzing IV mediation administration patterns to predict development of fluid overload. Such models may provide insight into areas where medication administration practices could be optimized to mitigate the risk of fluid overload in this patient population.


  • Evaluation of Critical Care Pharmacist Evening Services at an Academic Medical Center

    by Aaron M. Chase, Christy C. Forehand, Kelli R. Keats, Ashley N. Taylor, Timothy W. Jones, Andrea Sikora

    Purpose: Critical care pharmacists are considered essential members of the healthcare team; however, justification and recruitment of new positions, especially in the evening or weekend shifts, remains a significant challenge. The purpose of this study was to investigate the number of interventions, type of interventions, and associated cost savings with the addition of 1 board certified critical care clinical pharmacist to evening shift. Methods: This was a prospective collection and characterization of 1 evening shift critical care pharmacist’s clinical interventions over a 12-week period. Interventions were collected and categorized daily from 13:00 to 22:00 Monday through Friday. After collection was complete, cost savings estimates were calculated using pharmacy wholesaler acquisition cost. Results: Interventions were collected on 52 of 60 weekdays. A total of 510 interventions were collected with an average of 9.8 interventions accepted per day. The most common interventions included transitions of care, medication dose adjustment, and antibiotic de-escalation and the highest proportion of interventions occurred in the medical intensive care unit. An estimated associated cost avoidance of $66 537.80 was calculated for an average of $1279.57 saved per day. Additionally, 22 (4.1%) of interventions were considered high yield interventions upon independent review by 2 pharmacists. Conclusion: The addition of 1 board-certified critical care pharmacist to evening shift resulted in multiple interventions across several categories and a significant cost avoidance when calculated using conservative measures.


  • Open Access

    A Common Data Model for the standardization of intensive care unit (ICU) medication features in artificial intelligence (AI) applications

    by Andrea Sikora, Kelli Keats, David J. Murphy, John W. Devlin, Susan E. Smith, Brian Murray, Mitchell S. Buckley, Sandra Rowe, Lindsey Coppiano, Rishikesan Kamaleswaran, et al.

    Objective Common Data Models provide a standard means of describing data for artificial intelligence (AI) applications, but this process has never been undertaken for medications used in the intensive care unit (ICU). We sought to develop a Common Data Model (CDM) for ICU medications to standardize the medication features needed to support future ICU AI efforts. Materials and Methods A 9-member, multi-professional team of ICU clinicians and AI experts conducted a 5-round modified Delphi process employing conference calls, web-based communication, and electronic surveys to define the most important medication features for AI efforts. Candidate ICU medication features were generated through group discussion and then independently scored by each team member based on relevance to ICU clinical decision-making and feasibility for collection and coding. A key consideration was to ensure the final ontology both distinguished unique medications and met Findable, Accessible, Interoperable, and Reusable (FAIR) guiding principles. Results Using a list of 889 ICU medications, the team initially generated 106 different medication features, and 71 were ranked as being core features for the CDM. Through this process, 106 medication features were assigned to two key feature domains: drug product-related (n=43) and clinical practice-related (n=63). Each feature included a standardized definition and suggested response values housed in the electronic data library. This CDM for ICU medications is available online. Discussion The CDM for ICU medications represents an important first step for the research community focused on exploring how AI can improve patient outcomes and will require ongoing engagement and refinement. Lay Summary Medication data pose a unique challenge for interpretation by artificial intelligence (AI) because of the alphanumerical combinations (e.g., ibuprofen 200mg every 4 hours) and because of the technical detail associated with drug prescriptions (e.g., ibuprofen 200mg and acetaminophen 325mg are both starting doses and round tablet sizes, so it would be incorrect for the machine to view 325mg as ‘more’ than 200mg). Because AI has great potential to improve the safety and efficacy of medication use, a common data model for ICU medications (ICURx) is proposed here to overcome these challenges and support AI efforts in medication analysis.


  • Open Access

    Optimizing Pharmacist Team-Integration for ICU Patient Management: Rationale, Study Design, and Methods for a Multicentered Exploration of Pharmacist-to-Patient Ratio

    by Kelli Keats, Andrea Sikora, Mojdeh S. Heavner, Xianyan Chen, Susan E. Smith

    BACKGROUND: The workload of healthcare professionals including physicians and nurses in the ICU has an established relationship to patient outcomes, including mortality, length of stay, and other quality indicators; however, the relationship of critical care pharmacist workload to outcomes has not been rigorously evaluated and determined. The objective of our study is to characterize the relationship of critical care pharmacist workload in the ICU as it relates to patient-centered outcomes of critically ill patients. METHODS: Optimizing Pharmacist Team-Integration for ICU patient Management is a multicenter, observational cohort study with a target enrollment of 20,000 critically ill patients. Participating critical care pharmacists will enroll patients managed in the ICU. Data collection will consist of two observational phases: prospective and retrospective. During the prospective phase, critical care pharmacists will record daily workload data (e.g., census, number of rounding teams). During the retrospective phase, patient demographics, severity of illness, medication regimen complexity, and outcomes will be recorded. The primary outcome is mortality. Multiple methods will be used to explore the primary outcome including multilevel multiple logistic regression with stepwise variable selection to exclude nonsignificant covariates from the final model, supervised and unsupervised machine learning techniques, and Bayesian analysis. RESULTS: Our protocol describes the processes and methods for an observational study in the ICU. CONCLUSIONS: This study seeks to determine the relationship between pharmacist workload, as measured by pharmacist-to-patient ratio and the pharmacist clinical burden index, and patient-centered outcomes, including mortality and length of stay.


  • Ascorbic Acid for Methemoglobinemia Treatment: A Case Report and Literature Review

    by Kelli R Keats, Rachel Robinson, Mallika Patel, Alexis Wallace, Stephanie Albrecht

    Purpose: Ascorbic acid has been proposed as an alternative treatment for methemoglobinemia in patients with glucose-6-phosphate dehydrogenase (G6PD) deficiency. However, its efficacy has never been compared to that of methylene blue given the inability of patients with G6PD deficiency to receive methylene blue. We present a case of methemoglobinemia treated with ascorbic acid in a patient without G6PD deficiency who had previously received methylene blue. Summary: A 66-year-old male was treated for methemoglobinemia deemed to be secondary to benzocaine throat spray. He received intravenous (IV) methylene blue but had a severe reaction: diaphoresis, lightheadedness, and hypotension. The infusion was stopped prior to completion. Approximately 6 days later he presented with methemoglobinemia following an additional overconsumption of benzocaine and was treated with ascorbic acid. In both instances his methemoglobin levels were >30% on arterial blood gas on admission and decreased to 6.5% and 7.8%, respectively, after administration of methylene blue and ascorbic acid. Conclusion: Ascorbic acid had a similar effect on decreasing the concentration of methemoglobin compared to methylene blue. Further research into use of ascorbic acid as a recommended agent for treatment of methemoglobinemia is warranted.


  • Open Access

    Pharmacophenotype identification of intensive care unit medications using unsupervised cluster analysis of the ICURx common data model

    by Andrea Sikora, Alireza Rafiei, Milad Ghiasi Rad, Kelli Keats, Susan E. Smith, John W. Devlin, David J. Murphy, Brian Murray, Rishikesan Kamaleswaran, MRC-ICU Investigator Team

    Background: Identifying patterns within ICU medication regimens may help artificial intelligence algorithms to better predict patient outcomes; however, machine learning methods incorporating medications require further development, including standardized terminology. The Common Data Model for Intensive Care Unit (ICU) Medications (CDM-ICURx) may provide important infrastructure to clinicians and researchers to support artificial intelligence analysis of medication-related outcomes and healthcare costs. Using an unsupervised cluster analysis approach in combination with this common data model, the objective of this evaluation was to identify novel patterns of medication clusters (termed ‘pharmacophenotypes’) correlated with ICU adverse events (e.g., fluid overload) and patient-centered outcomes (e.g., mortality).Methods: This was a retrospective, observational cohort study of 991 critically ill adults. To identify pharmacophenotypes, unsupervised machine learning analysis with automated feature learning using restricted Boltzmann machine and hierarchical clustering was performed on the medication administration records of each patient during the first 24 h of their ICU stay. Hierarchical agglomerative clustering was applied to identify unique patient clusters. Distributions of medications across pharmacophenotypes were described, and differences among patient clusters were compared using signed rank tests and Fisher's exact tests, as appropriate.Results: A total of 30,550 medication orders for the 991 patients were analyzed; five unique patient clusters and six unique pharmacophenotypes were identified. For patient outcomes, compared to patients in Clusters 1 and 3, patients in Cluster 5 had a significantly shorter duration of mechanical ventilation and ICU length of stay (p < 0.05); for medications, Cluster 5 had a higher distribution of Pharmacophenotype 1 and a smaller distribution of Pharmacophenotype 2, compared to Clusters 1 and 3. For outcomes, patients in Cluster 2, despite having the highest severity of illness and greatest medication regimen complexity, had the lowest overall mortality; for medications, Cluster 2 also had a comparably higher distribution of Pharmacophenotype 6.Conclusion: The results of this evaluation suggest that patterns among patient clusters and medication regimens may be observed using empiric methods of unsupervised machine learning in combination with a common data model. These results have potential because while phenotyping approaches have been used to classify heterogenous syndromes in critical illness to better define treatment response, the entire medication administration record has not been incorporated in those analyses. Applying knowledge of these patterns at the bedside requires further algorithm development and clinical application but may have the future potential to be leveraged in guiding medication-related decision making to improve treatment outcomes.


  • An Evaluation of the Relationship Between Medication Regimen Complexity as Measured by the MRC-ICU to Medication Errors in Critically Ill Patients

    by Aaron M. Chase, Hanna A. Azimi, Christy Cecil Forehand, Kelli Keats, Ashley Taylor, Stephen Wu, Kaitlin Blotske, Andrea Sikora

    Purpose: The purpose of this study was to determine the relationship between medication regimen complexity-intensive care unit (MRC-ICU) score at 24 hours and medication errors identified throughout the ICU. Methods: A single-center, observational study was conducted from August to October 2021. The primary outcome was the association between MRC-ICU at 24 hours and total medication errors identified. During the prospective component, ICU pharmacists recorded medication errors identified over an 8-week period. During the retrospective component, the electronic medical record was reviewed to collect patient demographics, outcomes, and MRC-ICU score at 24 hours. The primary outcome of the relationship of MRC-ICU at 24 hours to medication errors was assessed using Pearson correlation. Results: A total of 150 patients were included. There were 2 pharmacists who recorded 634 errors during the 8-week study period. No significant relationship between MRC-ICU and medication errors was observed ( r2 = .13, P = .11). Exploratory analyses of MRC-ICU relationship to major interventions and harm scores showed that MRC-ICU scores >10 had more major interventions (27 vs 14, P = .27) and higher harm scores (15 vs 7, P = .33), although these values were not statistically significant. Conclusion: Medication errors appear to occur independently of medication regimen complexity. Critical care pharmacists were responsible for mitigating a large number of medication errors.


  • Evaluation of phenytoin loading doses in overweight patients using actual versus adjusted body weight

    by Kelli Keats, Rebecca Powell, Jody Rocker, Jennifer Waller, Lindsey Sellers Coppiano