A. Jay Holmgren
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
A. Jay Holmgren has a total of 139 co-authors. Here are the top co-authors by number of shared publications:
Julia Adler-Milstein
Division of Clinical Informatics and Digital Transformation, Department of Medicine, University of California San Francisco , San Francisco, CA 94131
14 shared publications- Nate C. Apathy
Department of Health Policy and Management, School of Public Health, University of Maryland, College Park, Maryland
9 shared publications Nate C. Apathy
Nate C. Apathy, University of Maryland, College Park, Maryland
8 shared publicationsLisa Rotenstein
University of California, San Francisco
7 shared publications- Christian Rose
Department of Emergency Medicine Stanford University School of Medicine Palo Alto California USA
3 shared publications Christine A. Sinsky
American Medical Association, Chicago, Illinois
3 shared publications
Latest Publications
- 1 August 2026
- Published by Health Affairs Publishing, LLC in Health Affairs
- Vol. 45 (8) , 862-870
The Uneven Distribution Of Patient Portal Messages Across Patients And Physicians
Responding to patient-initiated portal messages is a growing source of work for physicians, yet little is known about how these messages are distributed across patients and physicians. Understanding the distribution is critical for developing interventions to reduce burden and for designing reimbursement policy. Using data from 487,442 patients at University of California San Francisco (UCSF) Health and national electronic health record (EHR) metadata from 224,068 ambulatory care physicians in the US, we examined the distribution of patient-initiated medical advice request messages across patients and physicians. The distribution was highly skewed at both the patient and physician levels. The top 5 percent of UCSF Health patients accounted for 52.8 percent of all messages. Among physicians nationally, primary care physicians received a median of 9.6 messages per week, whereas the top quartile received 53.3 messages per week. Physicians with higher visit volume received fewer messages per visit, and higher message volume was associated with greater EHR work outside of work. These findings highlight the need for targeted interventions and payment models that account for the uneven distribution of asynchronous portal-based care.
- 1 August 2026
- Published by Health Affairs Publishing, LLC in Health Affairs
- Vol. 45 (8) , 862-870
The Uneven Distribution Of Patient Portal Messages Across Patients And Physicians
Responding to patient-initiated portal messages is a growing source of work for physicians, yet little is known about how these messages are distributed across patients and physicians. Understanding the distribution is critical for developing interventions to reduce burden and for designing reimbursement policy. Using data from 487,442 patients at University of California San Francisco (UCSF) Health and national electronic health record (EHR) metadata from 224,068 ambulatory care physicians in the US, we examined the distribution of patient-initiated medical advice request messages across patients and physicians. The distribution was highly skewed at both the patient and physician levels. The top 5 percent of UCSF Health patients accounted for 52.8 percent of all messages. Among physicians nationally, primary care physicians received a median of 9.6 messages per week, whereas the top quartile received 53.3 messages per week. Physicians with higher visit volume received fewer messages per visit, and higher message volume was associated with greater EHR work outside of work. These findings highlight the need for targeted interventions and payment models that account for the uneven distribution of asynchronous portal-based care.
- Open Access
- 9 July 2026
- Published by Oxford University Press (OUP) in JAMIA Open
- Vol. 9 (5)
Using electronic health record audit logs to identify pediatric intensive care unit teams
Background: Teamwork influences patient outcomes, but electronic health records (EHRs) do not reliably capture patient care team members, limiting its study. EHR audit logs provide insights into healthcare workers’ workflows and may help identify patients’ care team members. Objective: To develop, validate, and compare algorithms utilizing EHR audit logs to identify patient-centric primary teams (bedside nurse, frontline clinician, and attending physician) in pediatric intensive care units (ICUs). Methods: We observed rounds for 1931 patient days (development = 678; validation = 1253) across pediatric (PICU), neonatal (NICU) and cardiovascular (CVICU) ICUs at a quaternary children’s hospital and documented each patient’s daytime care team. We developed 2 algorithms that leveraged EHR audit logs to identify team members: (1) clinically informed heuristics, and (2) a Longitudinal Contribution Score (LCS). Accuracy was computed for each role and algorithm. 95% confidence intervals (CIs) and algorithm comparisons (bootstrap P values) were computed via patient-day bootstrap resampling. Results: In the development cohort (PICU), the LCS demonstrated greater accuracy than the heuristics for all roles: nurse (LCS 92.9% [95% CI, 90.9-94.8]; heuristic 67.6% [64.0-71.1]), frontline (83.3% [80.5-86.1]; 77.9% [74.6-81.0]), and attending (66.5% [62.7-70.2]; 61.9% [58.3-65.6]); all P < .005. In the validation cohort (PICU, NICU, and CVICU), only the nurse advantage was statistically replicated (LCS 91.2% [89.5-92.8], heuristic 74.2% [71.7-76.4]; P < .001); with frontline (83.0% [80.9-85.1], 81.2% [79.1-83.4]; P = .110) and attending (69.0 [66.5-71.6] vs 70.0% [67.4-72.4]; P = .326) roles showing comparable accuracy. Conclusion: EHR audit log data can accurately identify patients’ bedside nurses in pediatric ICUs. For frontline clinicians and attending physicians whose clinical coproduction and interdependence blurs individual EHR signals, identification is less accurate and supplementary data sources may be needed. Both algorithms provide a scalable foundation for team dynamics research, although the LCS offering greater generalizability than predefined heuristics.
- 9 July 2026
- Published by Oxford University Press (OUP) in JAMIA Open
- Vol. 9 (5)
Open AccessUsing electronic health record audit logs to identify pediatric intensive care unit teams
Background: Teamwork influences patient outcomes, but electronic health records (EHRs) do not reliably capture patient care team members, limiting its study. EHR audit logs provide insights into healthcare workers’ workflows and may help identify patients’ care team members. Objective: To develop, validate, and compare algorithms utilizing EHR audit logs to identify patient-centric primary teams (bedside nurse, frontline clinician, and attending physician) in pediatric intensive care units (ICUs). Methods: We observed rounds for 1931 patient days (development = 678; validation = 1253) across pediatric (PICU), neonatal (NICU) and cardiovascular (CVICU) ICUs at a quaternary children’s hospital and documented each patient’s daytime care team. We developed 2 algorithms that leveraged EHR audit logs to identify team members: (1) clinically informed heuristics, and (2) a Longitudinal Contribution Score (LCS). Accuracy was computed for each role and algorithm. 95% confidence intervals (CIs) and algorithm comparisons (bootstrap P values) were computed via patient-day bootstrap resampling. Results: In the development cohort (PICU), the LCS demonstrated greater accuracy than the heuristics for all roles: nurse (LCS 92.9% [95% CI, 90.9-94.8]; heuristic 67.6% [64.0-71.1]), frontline (83.3% [80.5-86.1]; 77.9% [74.6-81.0]), and attending (66.5% [62.7-70.2]; 61.9% [58.3-65.6]); all P < .005. In the validation cohort (PICU, NICU, and CVICU), only the nurse advantage was statistically replicated (LCS 91.2% [89.5-92.8], heuristic 74.2% [71.7-76.4]; P < .001); with frontline (83.0% [80.9-85.1], 81.2% [79.1-83.4]; P = .110) and attending (69.0 [66.5-71.6] vs 70.0% [67.4-72.4]; P = .326) roles showing comparable accuracy. Conclusion: EHR audit log data can accurately identify patients’ bedside nurses in pediatric ICUs. For frontline clinicians and attending physicians whose clinical coproduction and interdependence blurs individual EHR signals, identification is less accurate and supplementary data sources may be needed. Both algorithms provide a scalable foundation for team dynamics research, although the LCS offering greater generalizability than predefined heuristics.
- Open Access
- 3 July 2026
- Published by Oxford University Press (OUP) in Health Affairs Scholar
- Vol. 4 (7) , qxag154
Ten high-priority areas for health policy research in 2026-2027
Health policy is undergoing a period of rapid transformation, driven by major health reforms, changing market structures, pharmaceutical policy developments, technological innovation, and evolving models of care delivery. These changes have profound implications for health care access, affordability, quality, and health system sustainability. In this editorial, we identify 10 priority areas for health services and policy research that we believe will shape policy debates and evidence needs in the coming years. These priorities are organized across 5 domains: health care access, coverage, quality and delivery system reform; corporatization and financialization of health care systems; pharmaceutical policy and health services research; international comparisons and cross-national learning; and innovation, technological advancements, and resiliency of health systems. Specific areas we discuss include the effects of the H.R.1 on Medicaid coverage and economic security; the future of Medicare Advantage; the impact of immigration policies on access; the consequences of increasing corporatization of health care; reforms to pharmaceutical pricing; the role of international comparative research; and the implications of precision nutrition, climate-related disruptions, and artificial intelligence for health policy and health system performance. Across these topics, we emphasize the need for rigorous, policy-relevant research that moves beyond descriptive analyses to evaluate mechanisms, implementation, and real-world impacts.
- 3 July 2026
- Published by Oxford University Press (OUP) in Health Affairs Scholar
- Vol. 4 (7) , qxag154
Open AccessTen high-priority areas for health policy research in 2026-2027
Health policy is undergoing a period of rapid transformation, driven by major health reforms, changing market structures, pharmaceutical policy developments, technological innovation, and evolving models of care delivery. These changes have profound implications for health care access, affordability, quality, and health system sustainability. In this editorial, we identify 10 priority areas for health services and policy research that we believe will shape policy debates and evidence needs in the coming years. These priorities are organized across 5 domains: health care access, coverage, quality and delivery system reform; corporatization and financialization of health care systems; pharmaceutical policy and health services research; international comparisons and cross-national learning; and innovation, technological advancements, and resiliency of health systems. Specific areas we discuss include the effects of the H.R.1 on Medicaid coverage and economic security; the future of Medicare Advantage; the impact of immigration policies on access; the consequences of increasing corporatization of health care; reforms to pharmaceutical pricing; the role of international comparative research; and the implications of precision nutrition, climate-related disruptions, and artificial intelligence for health policy and health system performance. Across these topics, we emphasize the need for rigorous, policy-relevant research that moves beyond descriptive analyses to evaluate mechanisms, implementation, and real-world impacts.
- Open Access
- 11 June 2026
- Published by Oxford University Press (OUP) in JAMIA Open
- Vol. 9 (4) , ooag116
Patterns and predictors of clinician use of interoperability tools
Objectives: Clinicians’ adoption of interoperability tools influences care quality, but evidence of actual use is limited. We analyze clinicians’ use of outside records delivered via Epic Care Everywhere (CE), focusing on use frequency and predictors such as gender, experience, specialty, and role. Differences between pre-pandemic (2018-2019) and pandemic (2020-2021) periods are also examined to see how COVID-19 affected use of outside records. Materials and methods: De-identified EHR metadata from UCSF clinicians (n = 1442) during pre-pandemic and pandemic periods, totaling 686 797 clinician-day observations, were analyzed. We measured usage intensity (mean CE lookups per appointment) and breadth (percentage of appointments with ≥1 lookup). Generalized linear models (GLMs) with a negative binomial distribution for overdispersion were used to estimate predictors of intensity. Results: CE usage intensity rose by 43.0% post-COVID-19 onset compared with the pre-pandemic. Clinician specialty most strongly predicted use, with Nephrology and Cardiology showing the highest breadth (56.0% and 53.0% of visits, respectively), while Dermatology (13.2%) and Pediatrics (23.4%) were lowest. Residents used CE at 23.0% greater intensity than attendings, and each additional year of experience was linked to a 0.57% decrease in intensity. Discussion: Clinician use of interoperability tools was higher in specialties such as Nephrology and Cardiology that require more care coordination, and among less experienced clinicians including resident physicians. Use increased after the pandemic began, likely due to ongoing adoption trends and increased clinical demands during system strain and uncertainty. Conclusion: These findings underscore the critical importance of considering clinician behavior and contextual factors, such as specialty care needs, in addition to technical capabilities, when promoting the adoption and use of interoperability tools.
- 11 June 2026
- Published by Oxford University Press (OUP) in JAMIA Open
- Vol. 9 (4) , ooag116
Open AccessPatterns and predictors of clinician use of interoperability tools
Objectives: Clinicians’ adoption of interoperability tools influences care quality, but evidence of actual use is limited. We analyze clinicians’ use of outside records delivered via Epic Care Everywhere (CE), focusing on use frequency and predictors such as gender, experience, specialty, and role. Differences between pre-pandemic (2018-2019) and pandemic (2020-2021) periods are also examined to see how COVID-19 affected use of outside records. Materials and methods: De-identified EHR metadata from UCSF clinicians (n = 1442) during pre-pandemic and pandemic periods, totaling 686 797 clinician-day observations, were analyzed. We measured usage intensity (mean CE lookups per appointment) and breadth (percentage of appointments with ≥1 lookup). Generalized linear models (GLMs) with a negative binomial distribution for overdispersion were used to estimate predictors of intensity. Results: CE usage intensity rose by 43.0% post-COVID-19 onset compared with the pre-pandemic. Clinician specialty most strongly predicted use, with Nephrology and Cardiology showing the highest breadth (56.0% and 53.0% of visits, respectively), while Dermatology (13.2%) and Pediatrics (23.4%) were lowest. Residents used CE at 23.0% greater intensity than attendings, and each additional year of experience was linked to a 0.57% decrease in intensity. Discussion: Clinician use of interoperability tools was higher in specialties such as Nephrology and Cardiology that require more care coordination, and among less experienced clinicians including resident physicians. Use increased after the pandemic began, likely due to ongoing adoption trends and increased clinical demands during system strain and uncertainty. Conclusion: These findings underscore the critical importance of considering clinician behavior and contextual factors, such as specialty care needs, in addition to technical capabilities, when promoting the adoption and use of interoperability tools.
- 1 May 2026
- Published by American College of Physicians in Annals of Internal Medicine
- Vol. 179 (5) , 761-763
A Retrospective Study Evaluating the Utilization of G2211
- 1 May 2026
- Published by American College of Physicians in Annals of Internal Medicine
- Vol. 179 (5) , 761-763
A Retrospective Study Evaluating the Utilization of G2211
- Open Access
- 6 January 2026
- Published by Oxford University Press (OUP) in JAMIA Open
- Vol. 9 (1)
Electronic health record use factors linked to efficiency and productivity: an explainable machine learning analysis
Objective: To explore the relationship between ambulatory physician electronic health record (EHR) use characteristics and proxies for physician efficiency. Materials and Methods: A longitudinal cohort study was conducted to examine physician-month EHR use metadata in 413 US organizations between May 2019 and April 2022. A multi-model machine learning classifier was developed to predict physician efficiency. The main outcomes of the study were physician efficiency, measured as the proportion of same-day chart completion by specialty, and productivity, measured as daily patient visit volume, both segmented into quintiles. Results: The study included 218 610 unique physicians with 5 193 385 physician-month observations from 413 organizations with an average chart completion efficiency of 72.9% and 10.8 visits per scheduled day. The primary ML analysis achieved an accuracy of 0.74 in classifying physician-months with high chart completion efficiency and highlighted associations with key features, such as inbox message turnaround time <1.5 days and after-hours documentation <25 min/scheduled day. A secondary analysis achieved an accuracy of 0.84 in classifying physician-months with high visit volumes, indicating that factors such as EHR time outside scheduled hours <4.1 min/visit and clinical review time <3.2 min/visit were associated with higher visit volumes. Discussion and Conclusion: Implementing specific EHR use measures with distinct thresholds, such as inbox management and after-hours documentation, could help target interventions to enhance productivity, providing actionable insights to create balanced and efficient work environments that improve patient care and reduce EHR time.
- 6 January 2026
- Published by Oxford University Press (OUP) in JAMIA Open
- Vol. 9 (1)
Open AccessElectronic health record use factors linked to efficiency and productivity: an explainable machine learning analysis
Objective: To explore the relationship between ambulatory physician electronic health record (EHR) use characteristics and proxies for physician efficiency. Materials and Methods: A longitudinal cohort study was conducted to examine physician-month EHR use metadata in 413 US organizations between May 2019 and April 2022. A multi-model machine learning classifier was developed to predict physician efficiency. The main outcomes of the study were physician efficiency, measured as the proportion of same-day chart completion by specialty, and productivity, measured as daily patient visit volume, both segmented into quintiles. Results: The study included 218 610 unique physicians with 5 193 385 physician-month observations from 413 organizations with an average chart completion efficiency of 72.9% and 10.8 visits per scheduled day. The primary ML analysis achieved an accuracy of 0.74 in classifying physician-months with high chart completion efficiency and highlighted associations with key features, such as inbox message turnaround time <1.5 days and after-hours documentation <25 min/scheduled day. A secondary analysis achieved an accuracy of 0.84 in classifying physician-months with high visit volumes, indicating that factors such as EHR time outside scheduled hours <4.1 min/visit and clinical review time <3.2 min/visit were associated with higher visit volumes. Discussion and Conclusion: Implementing specific EHR use measures with distinct thresholds, such as inbox management and after-hours documentation, could help target interventions to enhance productivity, providing actionable insights to create balanced and efficient work environments that improve patient care and reduce EHR time.
- Top Cited PapersOpen Access
- 2 January 2026
- Published by American Medical Association (AMA) in JAMA Network Open
- Vol. 9 (1) , e2553233
Ambient Artificial Intelligence Scribes and Physician Financial Productivity
This cohort study evaluates physician revenue, patient volumes, and claim denials among adopters and nonadopters of artificial intelligence–based clinical documentation tools in one health system.
- 2 January 2026
- Published by American Medical Association (AMA) in JAMA Network Open
- Vol. 9 (1) , e2553233
Top Cited PapersOpen AccessAmbient Artificial Intelligence Scribes and Physician Financial Productivity
This cohort study evaluates physician revenue, patient volumes, and claim denials among adopters and nonadopters of artificial intelligence–based clinical documentation tools in one health system.
- 1 November 2025
- Published by American Society of Clinical Oncology (ASCO) in JCO Clinical Cancer Informatics
- Vol. 9 (9) , e2500283
Rapid Growth in Patient Portal Messages Underscores the Need for Actionable Paths Forward
- 1 November 2025
- Published by American Society of Clinical Oncology (ASCO) in JCO Clinical Cancer Informatics
- Vol. 9 (9) , e2500283
Rapid Growth in Patient Portal Messages Underscores the Need for Actionable Paths Forward
- Open Access
- 10 October 2025
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 32 (12) , 1947-1951
A SNAPpy use of large language models: using large language models to classify treatment plans in pediatric acute otitis media
Background and significance: Acute otitis media (AOM) is a leading cause of pediatric antibiotic overuse. Safety Net Antibiotic Prescriptions (SNAPs) are recommended for antibiotic stewardship but are difficult to identify due to lack of structured documentation. Objective: This study validates the accuracy of Versa, a GPT-4o based HIPAA-compliant large language model (LLM), to classify AOM treatment plans from physician notes. Methods: A retrospective cross-sectional study analyzed pediatric AOM encounters. Multiple prompting strategies were used to classify treatment plans and validated against a representative sample of manual reviews by 2 pediatricians. A locally fine-tuned model, Clinical-Longformer was also trained and tested against Versa and human review. Results: In total, 5707 encounters were included; 374 reviewed manually. Zero-shot accuracy was 97.8%; few-shot accuracy was 85%. Clinical-Longformer achieved 93.3% accuracy. Conclusion: Versa effectively identifies AOM treatment plans, providing a cost-efficient quality improvement tracking tool for prescription practice patterns in pediatric antibiotic stewardship efforts.
- 10 October 2025
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 32 (12) , 1947-1951
Open AccessA SNAPpy use of large language models: using large language models to classify treatment plans in pediatric acute otitis media
Background and significance: Acute otitis media (AOM) is a leading cause of pediatric antibiotic overuse. Safety Net Antibiotic Prescriptions (SNAPs) are recommended for antibiotic stewardship but are difficult to identify due to lack of structured documentation. Objective: This study validates the accuracy of Versa, a GPT-4o based HIPAA-compliant large language model (LLM), to classify AOM treatment plans from physician notes. Methods: A retrospective cross-sectional study analyzed pediatric AOM encounters. Multiple prompting strategies were used to classify treatment plans and validated against a representative sample of manual reviews by 2 pediatricians. A locally fine-tuned model, Clinical-Longformer was also trained and tested against Versa and human review. Results: In total, 5707 encounters were included; 374 reviewed manually. Zero-shot accuracy was 97.8%; few-shot accuracy was 85%. Clinical-Longformer achieved 93.3% accuracy. Conclusion: Versa effectively identifies AOM treatment plans, providing a cost-efficient quality improvement tracking tool for prescription practice patterns in pediatric antibiotic stewardship efforts.
- Open Access
- 11 July 2025
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 32 (9) , 1462-1470
Differences in physician electronic health record use by telemedicine intensity: evidence from 2 academic medical centers
Objective: Evaluate the association between telemedicine intensity and ambulatory physician electronic health record (EHR) use following the COVID-19 pandemic. Materials and Methods: This retrospective study included ambulatory physicians in 11 specialties at 2 large academic medical centers (Washington University in St Louis [WashU], University of California San Francisco [UCSF]). EHR use measures, including time-based and frequency-based, were analyzed in the post-COVID-19 period (March 1, 2021, through March 7, 2022). Multivariable regression models with 2-way fixed effects were used to assess the association between telemedicine intensity and EHR use. Results: Fully telemedicine physician-weeks were associated with higher EHR (hours per 8 patient scheduled hours; β = 3.2 at WashU, β = 1.4 at UCSF; P < .001) and documentation time (β = 2.7 at WashU, β = 1.4 at UCSF; P < .001). Several differences in discrete EHR-based tasks were observed: fully telemedicine physician-days were associated with lesser ordering, and there were mixed patterns for information seeking and clinical communication tasks. Discussion: Expanded use of telemedicine was associated with significant changes in physician EHR use post-COVID-19 onset. Increased EHR time may suggest a shift in workload, whereas decreased ordering may suggest constraints in virtual care, such as ability to perform physical examination and the reliance on patient-reported symptoms. Institutional differences usage patterns suggest that telemedicine’s impact is context-specific and provides opportunities for understanding how to optimize EHRs to support telemedicine. Conclusion: Telemedicine shifts physician EHR. Supporting physicians through optimized EHR tools, tailored workflows, and team-based interventions is essential for sustainable virtual care delivery without exacerbating EHR burden.
- 11 July 2025
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 32 (9) , 1462-1470
Open AccessDifferences in physician electronic health record use by telemedicine intensity: evidence from 2 academic medical centers
Objective: Evaluate the association between telemedicine intensity and ambulatory physician electronic health record (EHR) use following the COVID-19 pandemic. Materials and Methods: This retrospective study included ambulatory physicians in 11 specialties at 2 large academic medical centers (Washington University in St Louis [WashU], University of California San Francisco [UCSF]). EHR use measures, including time-based and frequency-based, were analyzed in the post-COVID-19 period (March 1, 2021, through March 7, 2022). Multivariable regression models with 2-way fixed effects were used to assess the association between telemedicine intensity and EHR use. Results: Fully telemedicine physician-weeks were associated with higher EHR (hours per 8 patient scheduled hours; β = 3.2 at WashU, β = 1.4 at UCSF; P < .001) and documentation time (β = 2.7 at WashU, β = 1.4 at UCSF; P < .001). Several differences in discrete EHR-based tasks were observed: fully telemedicine physician-days were associated with lesser ordering, and there were mixed patterns for information seeking and clinical communication tasks. Discussion: Expanded use of telemedicine was associated with significant changes in physician EHR use post-COVID-19 onset. Increased EHR time may suggest a shift in workload, whereas decreased ordering may suggest constraints in virtual care, such as ability to perform physical examination and the reliance on patient-reported symptoms. Institutional differences usage patterns suggest that telemedicine’s impact is context-specific and provides opportunities for understanding how to optimize EHRs to support telemedicine. Conclusion: Telemedicine shifts physician EHR. Supporting physicians through optimized EHR tools, tailored workflows, and team-based interventions is essential for sustainable virtual care delivery without exacerbating EHR burden.