Bryan D. Steitz

Objectives: Characterizing patient portal message content at scale can help target efforts to manage administrative work. We developed and validated a large language model (LLM) pipeline for multi-label classification of messages using an expert-derived topic taxonomy, then characterized topic distribution across a two-year corpus. Materials and Methods: We studied all medical advice request messages sent to ambulatory clinicians at an academic medical center from 2024-2025. We convened an expert panel that derived an 11-category taxonomy through a modified Delphi process. Two annotators labeled 750 randomly selected messages (Cohen kappa 0.80), holding out 500 for evaluation. The pipeline used GPT-4o-mini in a zero-shot prompt. On the held-out set, we measured micro- and macro-averaged precision, recall, and F1, and label stability across runs. We then characterized topic distribution and co-occurrence across the corpus. Results: The pipeline achieved micro- and macro-averaged F1 of 0.89 and 0.86. Labels were identical across runs for 93.6% of messages. Across 2.4 million messages, content concentrated on a few topics. The two most common topics, Problems & Management and Medications & Prescriptions, were present in 67.9% of messages, and the four most common in 93.9%. 51.7% of messages addressed multiple topics. Discussion and Conclusion: The pipeline classified patient message topics accurately and stably across millions of messages. Message content was concentrated within a small number of topics, highlighting opportunities for targeted interventions and enabling more efficient triage, routing, and patient-facing support.

Objectives: Clinical deterioration in hospitalized patients is often preventable, but traditional early warning scores based on structured data are limited by delayed or inconsistent documentation. We developed and evaluated a machine learning pipeline that predicts clinical deterioration using real-time pager messages exchanged between clinicians. Materials and Methods: We conducted a retrospective study of adult non-ICU hospitalizations at Vanderbilt University Medical Center between January 2018 and June 2021. Using the content and frequency of messages, we trained long short-term memory models to predict rapid response activation, unplanned ICU transfer, or cardiac arrest within the next 6, 12, or 24 hours. Model performance was compared with the Epic’s Deterioration Index and with a logistic regression ensemble combining predictions from both models. Results: There were 1 519 445 pages associated with 111 346 hospitalizations among 74 912 patients. Deterioration events were observed in 5114 (4.6%) hospitalizations. The model achieved moderate discriminative ability, with areas under the receiver operating characteristic curve (AUROC) of 0.684 (95% CI, 0.66-0.71), 0.724 (95% CI, 0.71-0.74), and 0.669 (95% CI, 0.65-0.68) for predicting deterioration within 6, 12, and 24 h, respectively. Predictions showed moderate correlation with EDI scores (Pearson’s R, 0.297-0.323) suggesting complementary signals. The ensemble model consistently outperformed either approach alone, achieving an AUROC of 0.797 (95% CI, 0.78-0.82) for 12-h prediction. Discussion and Conclusion: Clinical pages represent an underutilized data source capturing clinicians’ intuition and observations before they appear in formal documentation. Machine learning on pages can augment early warning systems by providing real-time information and intuition without increasing workload.

Background: 21st Century Cures Act information blocking regulations led to many organizations altering policies to electronically release test results to patients immediately upon their availability. Objective: This study aims to understand the prevalence and variation in how often patients view test results in the patient portal before hearing from their health care provider (HCP), are given the option to decide how results are communicated, and understanding of results viewed before hearing from their HCP. Methods: In this cross-sectional analysis, we analyzed data from the 2024 Health Information National Trends Survey on US adults with complete information who received test results via patient portal (n=6045). We examined the share of all patients and “portal users” (accessed their portal in the last year) who viewed test results in their portal, viewed results before hearing from their HCP, and were given the option to decide how results were communicated. We examined patients’ understanding of results viewed before hearing from their HCP and compared it with their understanding of other health information. Finally, we examined differences in weighted proportions of outcomes by individual characteristics using survey-weighted linear regression. Differences were assessed using design-adjusted Wald tests. Results: In 2024, 70% of patients (92% of portal users) viewed results in their patient portal, and 58% (76% of portal users) viewed the results before hearing from their HCP. Overall, 28% of patients and 33% of portal users reported being given the option to decide whether they wanted to receive test results before hearing from their HCP. Only 66% of patients reported that they understood the results they viewed in their patient portal. Provider encouragement, higher education, and digital literacy were most strongly associated with higher rates of viewing test results overall and before hearing from their HCP (eg, 70% of those encouraged viewed results vs 24% not encouraged; P <.001). Associations with age and chronic condition were also observed but were weaker. While higher education and digital literacy, age, and provider encouragement were strongly associated with being given the option to decide how to receive results, only high digital literacy and provider encouragement were strongly, positively associated with understanding results viewed before hearing from their HCP (eg, 77% of patients with high digital literacy understood results compared to 53% of those with low digital literacy; P <.001). Conclusions: This study is the first to provide a national estimate of patients who access immediately released test results and whether patients are given the option to decide how their results are communicated to them. While most patients view results before discussing them with their HCP, rates varied by individual characteristics, and substantially fewer patients report being given the option to decide how results are communicated. Incorporating patient preferences in portal communications may empower patients while preserving immediate access.

BACKGROUND The 21st Century Cures Act information blocking regulations led to many health care providers (HCPs) altering policies to electronically release test results to patients immediately upon their availability. OBJECTIVE To understand how often patients view results in the patient portal before hearing from their HCP, and whether they are given the option to decide how results are communicated. METHODS Using data from the 2024 Health Information National Trends Survey on U.S. adults who received recent test results via patient portal (N=6,045), we examined whether patients viewed test results before hearing from their HCP, were given the option to decide how test results were communicated, and understood results viewed before hearing from an HCP. RESULTS 70% of patients who received results viewed them in their patient portal, most of whom viewed results before hearing from their HCP (58% overall). 28% of patients and 33% of portal users reported being given the option to decide whether they wanted to receive test results before hearing from their HCP. Two-thirds of patients understood results they viewed in their patient portal before hearing from their HCP (66%). CONCLUSIONS While most patients viewed results before discussing with their HCP, only one-third reported being given the option to decide how results would be communicated and two-thirds of patients who viewed immediately released results understood their implications. Clearly presenting the option to decide when test results are communicated and incorporating patient preferences in portal communications could help empower patients and mitigate potential worry.

Importance: Patients have access to test results but often struggle with interpretation due to limited context. Helping patients understand their results is essential for engagement and may reduce unnecessary messages requesting explanation. Objective: To test the hypothesis that releasing test results in a patient-friendly educational format would reduce rates of patient-initiated messaging. Design, Setting, and Participants: This quality improvement study using an interrupted time series design with preintervention and postintervention evaluation was performed at a single academic medical center that implemented patient-friendly presentation of test results through a patient portal for results with the highest message volumes, including basic metabolic panel, comprehensive metabolic panel, complete blood cell count, thyroid panel or thyrotropin level, urine microalbumin tests, and selected microbial, antibody, and polymerase chain reaction (PCR) tests. Participants included adults who received outpatient test results via a patient portal from January 1 to December 31, 2024. Exposures: Educational format messaging. Main Outcomes and Measures: The primary outcome was the proportion of results reviewed by patients followed by a patient-initiated message within 24 hours of result release. An interrupted time series was performed to estimate the association of releasing results with the new educational format with messaging. Fitted models were stratified by whether results were ordered in primary care or flagged as abnormal. Results: A total of 829 902 results were reviewed by 205 139 patients (mean [SD] age, 51.0 [17.8] years; 130 284 [63.5%] female). After introducing educational messaging, nonsignificant decreases in weekly messages (mean [SD], 2769.2 [451.7] vs 2754.0 [529.6]; P = .92) and the proportion of results with a patient-initiated message within 24 hours (87 038 of 497 651 [17.5%] vs 57 585 of 332 251 [17.3%]; P = .22) were observed. When stratified by tests ordered through primary care, a statistically significant decrease was observed in messaging (average marginal effect, −0.8% [95% CI, −1.2% to −0.5%]; P < .001). Messaging was increased for microbial, antibody, and PCR tests ordered outside primary care (average marginal effect, 2.1% [95% CI, 1.0% to 3.1%]; P < .001). Conclusions and Relevance: In this quality improvement study in which test results were released in a patient-friendly educational format, no clinically meaningful change in patient messaging was observed. As organizations navigate information transparency and growing message volumes, it is essential to develop, implement, and study approaches to support information needs without impacting clinical workflow.

Importance: Patients have immediate access to test results in the patient portal, which can cause worry. Identifying behaviors associated with worry while awaiting results may allow health systems to support patients and reduce message volumes.Objective: To describe characteristics of patients who refresh their portal while awaiting test results and measure the association between refresh behavior and patient-initiated messaging.Design, Setting, and Participants: This retrospective cross-sectional study was performed at Vanderbilt University Medical Center, which uses a MyChart-based patient portal. All adult patients (aged ≥18 years) who received outpatient test results between January 1, 2022, and December 31, 2023, were included.Exposures: Use of the patient portal to review test results.Main Outcomes and Measures: The primary outcome was the proportion of patients who refreshed the portal while awaiting test results. Tests were stratified into low-sensitivity (eg, basic metabolic panel) and high-sensitivity (eg, tissue biopsy) groups using validated categories based on each test’s likelihood of being misinterpreted or causing distress. Portal access logs were used to identify refresh behaviors, in which patients accessed the portal seeking new results. Secondary outcomes included the association between patient characteristics and refresh behavior and between refresh behavior and messaging as measured using multivariable logistic regression.Results: A total of 968 774 results were reviewed by 290 349 patients (mean [SD] age, 47.8 [18.0] years; 66.3% female). Patients refreshed their portal for 25.9% of results. Patients more commonly refreshed for high-sensitivity results (25 280 of 64 356 [39.3%]) compared with low sensitivity results (225 516 of 904 418 [24.9%]) (P < .001). Patients who enabled notifications had higher odds of refreshing for high-sensitivity results (odds ratio, 1.11; 95% CI, 1.06-1.17) and low-sensitivity results (odds ratio, 1.54; 95% CI, 1.49-1.59). Each refresh for high-sensitivity results was associated with a lower increase in the probability of messaging within 24 hours (mean marginal effect, 0.41; 95% CI, 0.34-0.49) compared with refreshing for low-sensitivity results (mean marginal effect, 1.03; 95% CI, 0.96-1.11).Conclusion and Relevance: This cross-sectional study found an association of patients’ refresh activity for low-sensitivity results with patient-initiated messaging after review. These findings suggest that refresh behavior, a potential measure of worry, may be a characteristic of patient attitudes and preferences rather than the test.

Patient portals bridge patient and provider communications but exacerbate physician and nursing burnout. Large language models (LLMs) can generate message responses that are viewed favorably by health care professionals/providers (HCPs); however, these studies have not included diverse message types or new prompt-engineering strategies.Our goal is to investigate and compare the quality and precision of GPT-generated message responses versus real doctor responses across the spectrum of message types within a patient portal.We used prompt engineering techniques to craft synthetic provider responses tailored to adult primary care patients. We enrolled a sample of primary care providers in a cross-sectional study to compare authentic with synthetic patient portal message responses generated by GPT-3.5-turbo, July 2023 version (GPT). The survey assessed each response's empathy, relevance, medical accuracy, and readability on a scale from 0 to 5. Respondents were asked to identify responses that were GPT-generated versus provider-generated. Mean scores for all metrics were computed for subsequent analysis.A total of 49 HCPs participated in the survey (59% completion rate), comprising 16 physicians and 32 advanced practice providers (APPs). In comparison to responses generated by real doctors, GPT-generated responses scored statistically significantly higher than doctors in two of the four parameters: empathy (p < 0.05) and readability (p < 0.05). However, no statistically significant difference was observed for relevance and accuracy (p > 0.05). Although readability scores were significantly different, the absolute difference was small, and the clinical significance of this finding remains uncertain.Our findings affirm the potential of GPT-generated message responses to achieve comparable levels of empathy, relevance, and readability to those found in typical responses crafted by HCPs. Additional studies should be done within provider workflows and with careful evaluation of patient attitudes and concerns related to the ethics as well as the quality of generated responses in all settings.

Patient portal use among patients with cancer has increased significantly in recent years. This state-of-the-art review seeks to address and analyse literature involving patient portal use by patients with cancer and their care partners. In this review, we queried articles from PubMed published between January 2018 and April 2024 that describe recent trends and the current presence of portals in cancer care for patients, proxy users and/or care partners. We searched for articles addressing three overarching themes: (1) trends and disparities in portal adoption and use among patients with cancer, (2) use of specific portal components and functions in cancer care and (3) associations between portal use and cancer-related outcomes. Our search identified 278 unique studies, of which 82 were relevant empiric studies that met inclusion criteria and were included in this review. These papers aligned with 12 subthemes, including disparities in patient portal access, growing use of telemedicine via patient portal and patient access to immediately available to electronic health information. Our findings indicate that patient portals play an increasingly important role in helping patients manage their cancer care, despite few disparities that contribute to inequitable use. However, despite consistent growth in use over recent years, there are many areas for improvement in how portals support patients with cancer and a demand for functionality to continually evolve with patient needs.