Larry F. Chu

Without strategic intervention to support medical education research in anesthesiology, the training of future anesthesiologists may be compromised. As a result, the Anesthesia Research Council convened a working group of education leaders and charged them with mapping the future of education research in anesthesiology. At present, no comprehensive strategy exists for developing education research into a robust and self-sustaining discipline within anesthesiology. Few anesthesiologists are qualified to mentor the next generation of rigorous researchers in education who will advance both the theory and practice of medical education in our specialty and throughout medicine. This working group sought to understand the current state of anesthesiology education research funding and to identify key initiatives to consider in generating research excellence in this field. This paper explores three key aspects of developing a path forward for medical education research; understanding the current state of anesthesiology education research funding and publications as markers of impact; integrating educational frameworks that support diversity, equity, and inclusion (DEI) to ensure that anesthesiology training reflects the diverse needs of learners and patients; and recommendations for improving medical education research in anesthesiology to help democratize the resources necessary for research, cultivate collaboration, and advance faculty development. A qualitative thematic analysis of relevant literature was completed. Recommendations for expanding the current education frameworks, measures and methods include addressing: (1) Diversifying the Physician Workforce (2) Teaching Styles and Learning Environments (3) Quality and Accountability of Graduate Medical Education; (4) Cultural Competency and Intersectionality; (5) Advocacy. Identifying high priority, immediate and future actions that key stakeholders should undertake to generate centers of excellence and establish early career development programs in medical education research will likely have a national impact in anesthesiology and in medicine.

Academic anesthesiology has increasingly used grant dollars, citation impact factor, and publication numbers to measure faculty productivity. While they were originally designed as proxies for quality, they now risk becoming the mission driving the specialty. Academic anesthesiology departments are facing increasing faculty burnout, leaving departments bottom-heavy and lacking creativity, mentorship, and leadership. This has consequences for anesthesiology and patient care. Self-determination theory helps explain this pattern by showing how controlling metrics erode autonomy, competence, and relatedness, the core drivers of motivation. To protect the vitality of the specialty, departments must realign their missions with values that metrics cannot easily capture: teaching, mentorship, integrity, and community engagement. Diversified portfolios and balanced scorecards demonstrate feasible ways to support these values within financial constraints. Aligning systems with principles rather than proxies will allow departments to cultivate innovation and restore medicine’s moral foundation. Metrics may guide us, but they must never define us.

Large language models (LLMs) can answer expert-level questions in medicine but are prone to hallucinations and arithmetic errors. Early evidence suggests LLMs cannot reliably perform clinical calculations, limiting their potential integration into clinical workflows. We evaluated ChatGPT’s performance across 48 medical calculation tasks, finding incorrect responses in one-third of trials (n = 212). We then assessed three forms of agentic augmentation: retrieval-augmented generation, a code interpreter tool, and a set of task-specific calculation tools (OpenMedCalc) across 10,000 trials. Models with access to task-specific tools showed the greatest improvement, with LLaMa and GPT-based models demonstrating a 5.5-fold (88% vs 16%) and 13-fold (64% vs 4.8%) reduction in incorrect responses, respectively, compared to the unimproved models. Our findings suggest that integration of machine-readable, task-specific tools may help overcome LLMs’ limitations in medical calculations.

A bstract: Prior work has shown that large language models (LLMs) have the ability to answer expert-level multiple choice questions in medicine, but are limited by both their tendency to hallucinate knowledge and their inherent inadequacy in performing basic mathematical operations. Unsurprisingly, early evidence suggests that LLMs perform poorly when asked to execute common clinical calculations. Recently, it has been demonstrated that LLMs have the capability of interacting with external programs and tools, presenting a possible remedy for this limitation. In this study, we explore the ability of ChatGPT (GPT-4, November 2023) to perform medical calculations, evaluating its performance across 48 diverse clinical calculation tasks. Our findings indicate that ChatGPT is an unreliable clinical calculator, delivering inaccurate responses in one-third of trials (n=212). To address this, we developed an open-source clinical calculation API ( openmedcalc.org ), which we then integrated with ChatGPT. We subsequently evaluated the performance of this augmented model by comparing it against standard ChatGPT using 75 clinical vignettes in three common clinical calculation tasks: Caprini VTE Risk, Wells DVT Criteria, and MELD-Na. The augmented model demonstrated a marked improvement in accuracy over unimproved ChatGPT. Our findings suggest that integration of machine-usable, clinician-informed tools can help alleviate the reliability limitations observed in medical LLMs.

Background: The design of personal protective equipment (PPE) may affect well-being and clinical work. PPE as an integrated item may improve usability and increase adherence by healthcare professionals. Human factors design and safety may reduce occupational-acquired diseases. As an integrated PPE, a lightweight protective air-purifying respirator (L-PAPR) could be used during health procedures where healthcare professionals are exposed to airborne pathogens. The human factors affecting the implementation of alternative PPE such as L-PAPR have not been thoroughly studied. The population of interest is health care professionals, the intervention is the performance by PPE during tasks across the three PPE types 1.) N95 respirators and face shields, 2.)traditional powered air-purifying respirator(PAPR), and 3.) L-PAPR. The outcomes are user error, communications, safety, and end-user preferences. Objective: This study will assess whether the L-PAPR improves health care professionals’ comfort in terms of perceived workload and physical and psychological burden during direct patient care when compared with the traditional PAPR or N95 and face shield. This study also aims to evaluate human factors during the comparison of the use of L-PAPR with a combination of N95 respirators plus face shields or the traditional PAPRs. Methods: This is an interventional randomized crossover quality improvement feasibility study consisting of a 3-site simulation phase with 10 participants per site and subsequent field testing in 2 sites with 30 participants at each site. The 3 types of respiratory PPE will be compared across medical tasks and while donning and doffing. We will evaluate the user’s perceived workload, usability, usage errors, and heart rate. We will conduct semistructured interviews to identify barriers and enablers to implementation across each PPE type over a single continuous wear episode and observe interpersonal communications across conditions and PPE types. Results: We expect the research may highlight communication challenges and differences in usability and convenience across PPE types along with error frequency during PPE use across PPE types, tasks, and time. Conclusions: The design of PPE may affect overall well-being and hinder or facilitate clinical work. Combining 2 pieces of PPE into a single integrated item may improve usability and reduce occupational-acquired diseases. The human factors affecting the implementation of an alternative PPE such as L-PAPR or PAPR have not been thoroughly studied. International Registered Report Identifier (IRRID): PRR1-10.2196/36549

Background: Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) are self-reporting tools that can measure important information about patients, such as health priorities, experience, and perception of outcome. The use of traditional objective measures such as vital signs and lab values can be supplemented with these self-reported patient measures to provide a more complete picture of a patient’s health status. Machine learning, the use of computer algorithms that improve automatically through experience, is a powerful tool in health care that often does not use subjective information shared by patients. However, machine learning has largely been based on objective measures and has been developed without patient or public input. Algorithms often do not have access to critical information from patients and may be missing priorities and measures that matter to patients. Combining objective measures with patient-reported measures can improve the ability of machine learning algorithms to assess patients’ health status and improve the delivery of health care. Objective: The objective of this scoping review is to identify gaps and benefits in the way machine learning is integrated with patient-reported outcomes for the development of improved public and patient partnerships in research and health care. Methods: We reviewed the following 3 questions to learn from existing literature about the reported gaps and best methods for combining machine learning and patient-reported outcomes: (1) How are the public engaged as involved partners in the development of artificial intelligence in medicine? (2) What examples of good practice can we identify for the integration of PROMs into machine learning algorithms? (3) How has value-based health care influenced the development of artificial intelligence in health care? We searched Ovid MEDLINE(R), Embase, PsycINFO, Science Citation Index, Cochrane Library, and Database of Abstracts of Reviews of Effects in addition to PROSPERO and the ClinicalTrials website. The authors will use Covidence to screen titles and abstracts and to conduct the review. We will include systematic reviews and overviews published in any language and may explore additional study types. Quantitative, qualitative, and mixed methods studies are included in the reviews. Results: The search is completed, and Covidence software will be used to work collaboratively. We will report the review using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and Critical Appraisal Skills Programme for systematic reviews. Conclusions: Findings from our review will help us identify examples of good practice for how to involve the public in the development of machine learning systems as well as interventions and outcomes that have used PROMs and PREMs. International Registered Report Identifier (IRRID): DERR1-10.2196/36395