Maija Holsti

Large language models (LLMs) can improve text analysis efficiency in healthcare. This study explores the application of LLMs to analyze patient perspectives within the exception from informed consent (EFIC) process, which waives consent in emergency research. Our objective is to assess whether LLMs can analyze patient perspectives in EFIC interviews with performance comparable to human reviewers. We analyzed 102 EFIC community interviews from 9 sites, each with 46 questions, as part of the Pediatric Dose Optimization for Seizures in Emergency Medical Services study. We evaluated 5 LLMs, including GPT-4, to assess sentiment polarity on a 5-point scale and classify responses into predefined thematic classes. Three human reviewers conducted parallel analyses, with agreement measured by Cohen’s Kappa and classification accuracy. Polarity scores between LLM and human reviewers showed substantial agreement (Cohen’s kappa: 0.69, 95% CI 0.61–0.76), with major discrepancies in only 4.7% of responses. LLM achieved high thematic classification accuracy (0.868, 95% CI 0.853–0.881), comparable to inter-rater agreement among human reviewers (0.867, 95% CI 0.836–0.901). LLMs enabled large-scale visual analysis, comparing response statistics across sites, questions, and classes. LLMs efficiently analyzed patient perspectives in EFIC interviews, showing substantial sentiment assessment and thematic classification performance. However, occasional underperformance suggests LLMs should complement, not replace, human judgment. Future work should evaluate LLM integration in EFIC to enhance efficiency, reduce subjectivity, and support accurate patient perspective analysis.

We aimed to explore the healthcare team members’ (HTMs) views on social determinants of health (SDH) screening and referral processes in a pediatric emergency department (PED). We conducted a cross-sectional, mixed-methods study to explore PED HTMs’ views on social care practices at a quaternary-level children’s hospital. The survey was created using a goal identification framework. The survey gathered quantitative and qualitative data by assessing SDH screening practices, comfort and personal habits in screening, prioritization of SDH domains, workflows to perform screening, and perceived barriers to screening. Quantitative data were analyzed using descriptive statistics. Qualitative data were analyzed using thematic analysis and multiple reviewers to identify themes in free-text responses. There were 63 HTMs (48% response rate, Table 1), all of whom reported SDH screening efforts should continue in the PED (Fig. 1), yet 36% were unaware of the current SDH screening processes. Participants reported a median comfort level rate of 47.5 out of 100 when asking SDH-related questions (Fig. 1), and the highest reported barriers were “lack of knowledge/skill on resources” (77.3%), “lack of time” (68%), and “patient volume” (59%) (Fig. 2). Regarding the screening process, approximately half of the respondents suggested improvements in “integration of screening” and “variability of screening” (Fig. 4). This explorative analysis demonstrated that HTMs support continuing screening and referring patients for SDH-related needs. In addition, HTM survey respondents suggested improvements in screening and referral processes, opportunities for HTM training, and screening support to address social needs and ultimately improve patient health outcomes.

Background: Health literacy is a growing concern because of its effects on communication and health outcomes. One aspect of this communication is the ability of the health care provider to estimate the health literacy of a patient or their caregiver. The objectives of this study are to quantify misestimation of caregiver health literacy by providers and identify potential descriptive or demographic factors that might be related to those misestimations. Methods: Providers were asked to perceive descriptive factors and estimate the health literacy of caregivers in a pediatric Emergency Department. Then, the health literacy of the caregiver was tested using the Short Assessment of Health Literacy, and cross-tabulated with provider estimates. Results: Providers correctly estimated the health literacy of the caregivers 60% of the time, and misestimates were often underestimates (27.7%) rather than overestimates (12.3%). Providers overestimated the health literacy of 24.1% of fathers and only 9.8% of mothers (P = 0.012). They correctly estimated the health literacy of 63.9% of English-speaking caregivers compared with 30.6% of Spanish-speaking caregivers, and underestimated the health literacy of 50% of Spanish-speaking caregivers and 24.8% of English-speaking caregivers (P < 0.001). Providers correctly estimated the health literacy of 34.4% of racially and ethnically diverse caregivers compared with 71.5% of White/non-Hispanic caregivers. They underestimated the health literacy of 52.1% of these racially and ethnically diverse caregivers and 16.8% of White/non-Hispanic caregivers (P < 0.001). Conclusions: Providers often overestimate and underestimate the health literacy of parents in the pediatric emergency department. Misestimates are related to race, caregiver role, and language spoken by the caregiver. When providers misestimate health literacy, they may use words or phrases that are above or below the health literacy level of the caregiver. These results suggest a need for further health literacy research and interventions in provider education and clinical practice.

Clinical research in academic medical centers can be difficult to conduct and meet enrollment goals. Students under‐represented in medicine (URiM) are also under‐represented in academic leadership positions and as physician‐scientists but are critical to help solve health disparities. Barriers in pursuing medicine as a career may be high for URiM students, therefore it is important to create pre‐medicine opportunities accessible to all students interested in healthcare careers. We describe an undergraduate clinical research platform, the Academic Associate (AcA) program, embedded in the medical system that supports clinical research for academic physician scientists and provides students equitable access to experiences and mentoring opportunities. Students have the opportunity of completing a Pediatric Clinical Research Minor (PCRM) degree. This program satisfies many pre‐medicine opportunities for undergraduate students, including those URiM, and allows access to physician mentors and unique educational experiences for graduate school or employment. Since 2009, 820 students participated in the AcA program (17.5% URiM) and 235 students (18% URiM) completed the PCRM. Of the 820 students, 126 (10% URiM) students matriculated to medical school, 128 (11%URiM) to graduate school, and 85 (16.5% URiM) gained employment in biomedical research fields. Students in our program supported 57 publications and were top‐enrollers for several multicentered studies. The AcA program is cost‐effective and achieves a high level of success enrolling patients into clinical research. Additionally, the AcA program provides equitable access for students URiM to physician mentorship, pre‐medical experiences, and an avenue to early immersion in academic medicine.

Communication gaps between the healthcare team and caregivers of pediatric patients can result in negative consequences. This study aims to identify specific words and phrases used in a pediatric emergency department (ED) that are unclear or confusing to caregivers. Research assistants at the Primary Children’s Hospital recorded caregivers’ responses to the question, “ What words or phrases have been used during this visit that are unclear or don’t make sense to you?” Across all steps in the care process, 62 of 220 participants (28.2%) reported unclear words and phrases used by the healthcare team. Responses recorded after the discharge step had the highest proportion of communication problems, followed by the initial evaluation and then the update step ( χ² [2, N = 220] = 6.30, P = .043). Themes among responses included ED logistics, signs/symptoms, the diagnostic process, treatment/procedures, general confusion, and language barriers. These results provide feedback to pediatric emergency medicine providers about potential communication gaps and point to a need for further efforts to train providers in the practice of high-quality communication.

Problem: American Indian/Alaska Native (AI/AN) populations are facing multiple health crises, including limited access to care, high rates of chronic disease, and early mortality that is far worse than other underrepresented minorities in the United States. According to the Association of American Indian Physicians, AI/AN people represent 2.0% of the U.S. population but only 0.2% of medical students and 0.1% of full-time faculty at MD-granting institutions. Increasing the number of AI/AN clinicians and scientists is one strategy to improve health outcomes in the AI/AN population and address these crises. Approach: In 2010, the University of Utah partnered with research, cultural, and professional mentors to create a 10-week summer Native American Research Internship (NARI) program for AI/AN college students across the United States who are interested in pursuing biomedical careers. NARI attracts and supports AI/AN students by offering mentored summer research internships in an innovative, culturally aware framework that adapts to observed challenges to optimize educational experiences and support biomedical career aspirations. Outcomes: During the first decade of the NARI program, 128 students from 22 U.S. states, representing 46 tribal nations and 57 colleges and universities, participated. Of those 128 students, 113 (88%) have completed a bachelor’s degree and the remaining 15 (12%) are currently working toward a bachelor’s degree. No NARI student has dropped out of college. Twenty-six (20%) NARI alumni have matriculated to medical school and 30 (23%) to graduate school. Eight (6%) participants have completed medical school, and 3 (2%) are pursuing a PhD in science. An additional 36 (28%) have gained employment in biomedical research fields. Next Steps: The NARI program has increased the participation of AI/AN students in medicine and the biomedical sciences. The innovative, culturally aware, and adaptive framework is a model for other programs for AI/AN students and students in other underrepresented communities.

Background: The US federal regulation “Exception from Informed Consent for Emergency Research,” 21 Code of Federal Regulations 50.24, permits emergency research without informed consent under limited conditions. Additional safeguards to protect human subjects include requirements for community consultation and public disclosure prior to starting the research. Because the regulations are vague about these requirements, Institutional Review Boards determine the adequacy of these activities at a local level. Thus, there is potential for broad interpretation and practice variation. Aim: To describe the variation of community consultation and public disclosure activities approved by Institutional Review Boards, and the effectiveness of this process for a multi-center, Exception from Informed Consent, pediatric status epilepticus clinical research trial. Methods: Community consultation and public disclosure activities were analyzed for each of the 15 participating sites. Surveys were conducted with participants enrolled in the status epilepticus trial to assess the effectiveness of public disclosure dissemination prior to study enrollment. Results: Every Institutional Review Board, among the 15 participating sites, had a varied interpretation of Exception from Informed Consent regulations for community consultation and public disclosure activities. Institutional Review Boards required various combinations of focus groups, interviews, surveys, and meetings for community consultation, and news releases, mailings, and public service announcements for public disclosure. At least 4335 patients received information about the study from these efforts. In all, 158 chose to be included in the “Opt Out” list. Of the 304 participants who were enrolled under Exception from Informed Consent, 12 (5%) had heard about the study through community consultation or public disclosure activities. The activities reaching the highest number of participants were surveys and focus groups associated with existing meetings. Public disclosure activities were more efficient and cost-effective if they were part of an in-hospital resource for patients and families. Conclusion: There is substantial variation in Institutional Review Boards’ interpretations of the federal regulations for community consultation and public disclosure. One of the goals of community consultation and public disclosure efforts for emergency research is to provide community members an opportunity to opt out of Exception from Informed Consent research; however, rarely do patients or their legally authorized representatives report having learned about a study prior to enrollment.