Daiju Ueda

Large language models (LLMs) show promise in supporting drug recommendations both within and outside clinical settings, but they may also amplify systematic biases affecting equity and cost. We examined the commercial neutrality of LLM-generated product name recommendations among US Food and Drug Administration (FDA)-approved products containing the same active ingredient. We evaluated product name recommendations from five LLMs using 177,120 synthetic clinical vignettes spanning 123 disease–drug pairs and 1,440 sociodemographic profiles. Each vignette specified a disease and an active ingredient and asked the model to recommend FDA-approved products. Across disease–drug pairs and models, the most frequently recommended product accounted for a mean of 97.5% of first-choice recommendations, and the models showed almost perfect agreement on which products to recommend. LLM-recommended product distributions differed from real-world prescribing patterns and often yielded higher projected acquisition costs. Recommendations inconsistent with regulatory information varied across LLMs and sociodemographic factors. Models more often recommended products with greater online visibility (Google search counts, odds ratio (OR) 1.70; a dedicated Wikipedia page, OR 3.00) and favoured originator products (OR 20.5), whereas products with later FDA approval and those more commonly prescribed in practice were recommended less often (ORs 0.76 and 0.87, respectively). Although modified prompts shifted recommendations in the intended direction, LLMs still favoured a narrow set of products, which could have potential implications for drug costs and pharmaceutical markets. Rigorous evaluation and targeted mitigation are needed before widespread use.

Background: Large language models (LLMs) are increasingly being integrated into clinical decision support, ranging from documentation aids to triage and patient communication. However, their behavior in ethically ambiguous medical situations is poorly characterized. Variability in how different LLMs balance approval of against refusal of a proposed action could materially shape the recommendations clinicians receive, yet no widely shared benchmark systematically quantifies this variability across contemporary model families. Objective: This study aimed to develop a standardized case-based audit set of medical ethics dilemmas and quantify variation in recommendation direction, model-reported confidence, within-model stability, and between-model agreement across 5 selected contemporary LLMs. Methods: We constructed 60 original medical ethics dilemmas spanning 4 domains. Three researchers independently annotated case difficulty and ethical principles before reaching consensus. Five primary models—GPT-5, Claude Sonnet 4.5, Gemini 2.5 Pro, full-size DeepSeek-R1, and Meta Llama-3-8B-Instruct—were each evaluated 3 times per case using a fixed prompt and temperature 0. DeepSeek-R1-Distill-Qwen-1.5B was evaluated separately as an exploratory small-model comparator. The primary outcome was the binary yes or no answer; model-reported confidence and reason were also collected. Model differences were assessed using a case-clustered generalized estimating equation (GEE), and between-model agreement and within-model stability were summarized. Results: Of 900 planned primary panel outputs, 897 (99.7%) had valid binary answers. Trial-level yes rates were 44.4% (80/180) for DeepSeek-R1, 42.4% (75/177) for Claude Sonnet 4.5, 38.9% (70/180) for GPT-5, 38.9% (70/180) for Gemini 2.5 Pro, and 16.7% (30/180) for Llama-3-8B. The overall model effect was significant (Wald χ24=23.34; P<.001); after Holm adjustment, only comparisons involving Llama-3-8B were significant. Across 59 complete-panel cases, raw agreement was 79.0% (466/590 model-pair comparisons; 95% bootstrap CI 72.9%-84.7%), and Fleiss κ was 0.55 (95% bootstrap CI 0.43-0.66). Within-model response stability across 3 runs ranged from 85.0% (51/60) to 100.0% (59/59 and 60/60). Conclusions: Selected contemporary LLMs showed model-dependent variation but moderate overall agreement in binary recommendations for medical ethics cases. These results characterize recommendation behavior under a fixed, time-specific protocol and do not establish ethical correctness or the effects on clinicians or patients. Reporting model version, recommendation tendency, confidence, and repeatability may improve transparency when such systems are evaluated for ethically sensitive applications.

Despite growing clinical adoption of artificial intelligence in radiology, fewer than one in five commercially available models are cleared for paediatric use. Existing systems are trained predominantly on adult data and generalize poorly to neonates, yet efficient strategies for repurposing them without data sharing or large-scale local data curation are lacking. Here we present an international retrospective study of neonatal chest radiographs from 30 neonatal intensive care units across 20 countries and territories, evaluating continual learning as a privacy-preserving approach to adapt a deep learning model for adult endotracheal tube assessment to neonatal care. Each hospital contributed 50 radiographs for sequential model retraining without centralized data pooling. Continual learning achieved lower anatomical landmark localization error (0.67 cm versus 1.29 cm for hospital-specific fine-tuning and 4.30 cm for adult-only models) at 29 of 30 sites, with improved detection of deep intubations at 27 of 30 sites. These findings demonstrate that privacy-preserving collaborative learning can address critical gaps in paediatric AI deployment.

Purpose: Generative artificial intelligence (AI) models are increasingly evaluated for diagnostic tasks in radiology, yet accuracy, study designs, endpoints, and comparators vary widely. The purpose was to synthesize diagnostic accuracy of generative AI for radiology and compare performance with physicians. Materials and methods: A systematic review and meta-analysis was prospectively registered in PROSPERO (CRD420251040000) and conducted in accordance with PRISMA-DTA guidance. Searches of Medline, Scopus, Web of Science, Cochrane Central, and medRxiv (June 2018–March 2025) identified studies validating generative AI on diagnostic tasks in radiology. Two reviewers independently screened, extracted data, and assessed risk of bias with PROBAST+AI, with disagreements resolved by a third reviewer. Multilevel random-effects meta-regression was performed to compare AI performance with physician performance and identify sources of heterogeneity, with study-clustered robust inference to account for multiple estimates per study. Results: In total, 48 studies met inclusion criteria. Pooled diagnostic accuracy of generative AI was 42.9% (95% CI, 35.8–50.1%) for free-text tasks and 58.1% (95% CI, 50.0–66.2%) for choice tasks. Generative AI overall showed significantly lower accuracy than expert physicians (difference in accuracy [physicians minus AI], +13.0 percentage points [95% CI, 0.9–25.2]; P = .038). Text-only input was associated with higher accuracy than image-only input (difference in accuracy [image-only minus text-only], −25.9 percentage points [95% CI, −39.1 to −12.8]; P = .001) and text-and-image input (− 10.6 percentage points [95% CI, −20.8 to −0.3]; P = .046). Conclusion: Generative AI remained less accurate than expert physicians on diagnostic tasks in radiology. Text-oriented assistive use warrants further evaluation, but the observed accuracy difference may reflect task difficulty and information content rather than the input modality itself. Standardized, transparently reported, adequately powered evaluations are warranted before clinical deployment.

Background: Assessment of amyloid‐β (Aβ) burden and associated iron deposition and neurodegeneration is important for Alzheimer's disease (AD) management. Although quantitative susceptibility mapping (QSM) detects iron and myelin changes, conventional metrics suffer from signal cancelation between paramagnetic and diamagnetic components. Consequently, spatial interactions between these independent susceptibility sources and Aβ burden remain unestablished. Purpose: To investigate spatial associations between regional Aβ burden and independent paramagnetic and diamagnetic susceptibility sources using χ‐separation. Study Type: Prospective. Population: A total of 77 participants with mild cognitive impairment or dementia (mean age, 74 years; women, 50) examined using QSM and amyloid positron emission tomography (PET) ( ¹⁸ F‐flutemetamol or ¹⁸ F‐florbetapir). Field Strength/Sequence: 3 T/QSM (3‐dimensional multi‐echo gradient‐echo sequence). Assessment: χ‐separation decomposed QSM data into paramagnetic (χ‐para) and diamagnetic (χ‐dia) components. Centiloid‐scale maps standardized PET quantification for voxel‐wise analysis. The spatial relationship between Aβ load and susceptibility metrics was analyzed using voxel‐wise correlations within AD‐signature cortical regions. Statistical Tests: Voxel‐wise spatial associations between Centiloid values and QSM metrics were evaluated using Pearson's coefficients. Spatial association was considered statistically significant if the 99% confidence interval (CI) did not cross zero. Results: Conventional QSM showed near‐zero cortical susceptibility and obscured pathological details owing to physiological cancelation. χ‐separation unmasked susceptibility sources. Local Aβ deposition was consistently associated with reduced absolute χ‐dia across cortical regions (mean r range: −0.267 to −0.098; all 99% CIs did not cross zero), consistent with alterations in net diamagnetic susceptibility sources, potentially involving myelin‐related components. χ‐para showed weaker and regionally heterogeneous associations with Aβ burden, with a positive association most evident in the precuneus (mean r = 0.072, 99% CI: 0.052–0.092). Data Conclusion: χ‐separation reveals region‐specific cortical alteration associated with regional Aβ pathology. By separating these susceptibility sources, this method may provide complementary information to conventional QSM, offering a refined, noninvasive approach for characterizing amyloid‐associated tissue changes in AD. Evidence Level: 2. Technical Efficacy: Stage 2.

Objective: Though subgroup performance reporting helps ensure the safety of artificial intelligence (AI) products, the extent of this reporting remains unclear. This scoping review identifies studies validating commercially available AI-based products and reports the trends in performance reporting across sex, age, and race/ethnicity demographic subgroups. Materials and methods: Peer-reviewed validation studies of commercially available products published after 2010 were collected from the Health AI Register and PubMed on 29 November 2024. Study trends in the reporting of sex, age, and race/ethnicity were mapped with regression analysis. We apply the Wilson confidence interval equation to estimate which tuberculosis detection studies are underpowered for subgroup meta-analysis. Results: Three hundred ninety-two of 545 studies validating 252 products reported subgroup demographic data for any of the three groups. Only 77 of these presented subgroup performance results. Skeletal (20/88) and lung (30/139) studies, and those utilizing chest (24/79) or bone (19/63) radiographs, most often presented subgroup performance data. We found no evidence that more recent studies (OR: 1.039 [95% CI: 0.959–1.127]) or company sponsorship (OR: 1.010 [95% CI: 0.492–1.920]) led to increased subgroup reporting. We show that 14/21 tuberculosis datasets may be underpowered for post-hoc subgroup meta-analysis. Conclusion: This scoping review quantifies how fragmented the commercial validation landscape is, showing that reporting for both the demographics and per-subgroup performance is inadequate for estimating subgroup bias. This systemic problem requires effort from all stakeholders, from researchers to regulatory agencies, encouraging thorough reporting and commercial product validation to support physician and patient trust in medical AI products. Key Points: Question The number of studies validating the performance of each commercially available radiology AI product for minority subgroup bias is unclear . Findings The currently available commercial AI validation studies often neglect to describe demographic subgroup data, and fewer provide performance results per subgroup, prohibiting algorithmic bias meta-analysis . Clinical relevance Physician and patient trust in the medical AI already used clinically must be built on peer-reviewed literature and meta-analysis. The current literature is insufficient for determining the safety and performance of these products for demographic minorities .

Purpose: Large language models (LLMs) are increasingly integrated into radiology workflows, but their demographic biases have not been evaluated in diagnostic radiology. This study aimed to investigate racial and sex biases in the diagnostic performance of LLMs (text-only and vision models) in radiology and to evaluate whether prompting strategies mitigate these biases. Materials and methods: This retrospective study included consecutive Diagnosis Please cases published in Radiology from April 1998 to October 2024, excluding cases with sex-specific diseases. For each case, eight race-sex scenarios were generated by altering four race/ethnicity categories (Asian, Black, Hispanic, White) and two sex categories (male, female). Three LLMs (GPT-5, Claude Sonnet 4.5, Gemini 2.5 Flash) were evaluated as text-only models (medical history and imaging findings) and vision models (medical history and images) using three prompting strategies (basic, self-consistency, chain-of-thought prompting). Generalized estimating equations were used to compare diagnostic accuracy across race/ethnicity, sex, and prompting strategies. Results: A total of 286 cases were included. Across models and conditions, ten significant race-related and four significant sex-related differences in diagnostic accuracy were observed. Among the four race/ethnicity groups, Black patients were most likely to have significantly lower accuracy (four of the ten statistically significant race-related comparisons [40%]) and least likely to have significantly higher accuracy (1/10 [10%]). Vision models with female patients under basic prompting showed a higher number of significant race-related differences (6/10 [60%]) than vision models with male patients and all text-only models. Text-only and vision models showed no statistically significant differences in diagnostic accuracy across prompting strategies ( p = 0.78 and 0.95, respectively); basic and self-consistency prompting produced ten and four significant race- or sex-related differences, respectively, whereas no significant differences were observed with chain-of-thought prompting. Conclusion: Large language models exhibited racial and sex biases in diagnostic radiology, and chain-of-thought prompting may help mitigate these biases.

BACKGROUND Large language models (LLMs) are increasingly being integrated into clinical decision support, ranging from documentation aids to triage and patient communication. However, their behavior in ethically ambiguous medical situations is poorly characterized. Variability in how different LLMs balance approval of against refusal of a proposed action could materially shape the recommendations clinicians receive, yet no widely shared benchmark systematically quantifies this variability across contemporary model families. OBJECTIVE This study aimed to develop a standardized case-based audit set of medical ethics dilemmas and quantify variation in recommendation direction, model-reported confidence, within-model stability, and between-model agreement across 5 selected contemporary LLMs. METHODS We constructed 60 original medical ethics dilemmas spanning 4 domains. Three researchers independently annotated case difficulty and ethical principles before reaching consensus. Five primary models—GPT-5, Claude Sonnet 4.5, Gemini 2.5 Pro, full-size DeepSeek-R1, and Meta Llama-3-8B-Instruct—were each evaluated 3 times per case using a fixed prompt and temperature 0. DeepSeek-R1-Distill-Qwen-1.5B was evaluated separately as an exploratory small-model comparator. The primary outcome was the binary yes or no answer; model-reported confidence and reason were also collected. Model differences were assessed using a case-clustered generalized estimating equation (GEE), and between-model agreement and within-model stability were summarized. RESULTS Of 900 planned primary panel outputs, 897 (99.7%) had valid binary answers. Trial-level yes rates were 44.4% (80/180) for DeepSeek-R1, 42.4% (75/177) for Claude Sonnet 4.5, 38.9% (70/180) for GPT-5, 38.9% (70/180) for Gemini 2.5 Pro, and 16.7% (30/180) for Llama-3-8B. The overall model effect was significant (Wald χ24=23.34; P<.001); after Holm adjustment, only comparisons involving Llama-3-8B were significant. Across 59 complete-panel cases, raw agreement was 79.0% (466/590 model-pair comparisons; 95% bootstrap CI 72.9%-84.7%), and Fleiss κ was 0.55 (95% bootstrap CI 0.43-0.66). Within-model response stability across 3 runs ranged from 85.0% (51/60) to 100.0% (59/59 and 60/60). CONCLUSIONS Selected contemporary LLMs showed model-dependent variation but moderate overall agreement in binary recommendations for medical ethics cases. These results characterize recommendation behavior under a fixed, time-specific protocol and do not establish ethical correctness or the effects on clinicians or patients. Reporting model version, recommendation tendency, confidence, and repeatability may improve transparency when such systems are evaluated for ethically sensitive applications.