Manuel Corpas

Agentic large language models are increasingly used across the genomic workflow, from variant calling to clinical interpretation, yet they are evaluated by accuracy alone, a single figure that cannot say whether a system is safe or where in the workflow a failure originates. We present ClawBench, a framework that attributes each outcome to the architectural layer that produced it across both halves of the canonical pipeline. Two design choices remove the confounds that make agentic genomics hard to evaluate: a temporally blinded truth set, in which every scored ClinVar label first became available only after the training cutoff of every model tested, and a fail-closed evidence contract that blocks evidence circular with the truth label. We score validity, safety, provenance and reproducibility, not accuracy alone, under a constraint gradient that relocates correctness from a model’s prior into executed, validated code. We show three things. First, dangerous misclassification is rare and model-invariant, a controlled precondition of the executed architecture rather than a frontier, while fabricated evidence is measurable and is neutralised by execution. Second, different variant classes are rate-limited by different layers: loss-of-function variants by the deterministic combiner threshold, and rare missense by evidence formation, where evidence acquisition is asymmetric and capped and strength assignment is a recoverable layer that naive strength-licensing prompts confound. Third, for variant calling the arms separate not on whether a model can plan a pipeline, which all do, but on trust properties, pinning, provenance, auditability and reproducibility, which climb monotonically toward validated execution; and a local open-weight model reproduces the safety result yet meets the structured-output and provenance contract far less often than frontier models, a conformance gap rather than a capability or safety gap. An end-to-end join attributes failures across the whole workflow, separating a missed call from a propagated genotype error from a correctly called but misinterpreted variant. ClawBench shows that apparently identical outcomes arise from distinct, independently measurable failure modes, and that trustworthiness in agentic genomics is a property of the pipeline architecture rather than of the model, providing a portable, contamination-resistant unit of attribution for the field.

Summary: Genomics is adopting autonomous AI agents that interpret genomes from natural-language instructions faster than it is building the means to trust them. We report the first large-scale controlled evaluation of where, in an agentic genomic pipeline, correctness must reside for the system to be trustworthy at clinical scale. Using pharmacogenomics, a domain where errors are measurable and sometimes lethal, we benchmarked nine frontier large language models across 44,550 scored evaluations on 110 pharmacogenomic cases, and tested model interpretation of real star-allele diplotypes from more than 7,000 individuals in three ancestrally diverse populations. Trustworthiness proved to be a property of pipeline architecture, not of the model. Letting the model reason was stochastic and unsafe, and grounding it in the correct guidelines by retrieval paradoxically increased lethal-class errors. Encoding the validated decision logic as a versioned skill and executing it as code made the pharmacogenomic mapping exact, auditable and identical across models, confining all residual error to a single input-interpretation step. On individual genomes, unguarded model interpretation degraded along an ancestry gradient; execution removes this gradient from the clinical mapping, relocating it to the auditable completeness of the input caller. This establishes a generalisable, auditable architecture for trustworthy agentic genome interpretation at scale. Highlights: Correctness must be executed, not reasoned or retrieved, to be trustworthy Retrieval raises phenotype accuracy yet increases lethal-class errors; skills do not Execution makes the clinical mapping exact and model-invariant; error stays at input A deterministic input caller is the predicted route to all-correct emitted answers In brief: Corpas and colleagues show that trustworthy agentic genome interpretation comes not from making language models reason correctly about biology, but from confining them to interpreting input while versioned, validated skills do the reasoning as executed code. Across nine large language models and 110 pharmacogenomics cases, executing the skill makes the clinical mapping deterministic, auditable and model-invariant. Significance: Genomics is adopting autonomous, language-model-mediated agents faster than it is building the standards needed to trust them. On a pharmacogenomic benchmark with lethal-class consequences, we show that an agent’s trustworthiness is not a property of the model but of how the agent is constrained: correctness must be moved out of the stochastic model into a versioned skill executed as code, with the model confined to interpreting heterogeneous input. This gives the field a transferable architecture for trustworthy agentic genome interpretation, a predicted route to deploying it so that every emitted answer is correct (execute the validated skill, call the input deterministically, and abstain on the irreducible residual), and a way to develop genomic skills as validated, executable, versioned units rather than prompts. Following a validation framework described elsewhere, we use clinical-grade to mean determinism, auditability, traceability to versioned components and population-invariant performance, all achieved under skill-constrained execution. We distinguish two senses of population performance: the executed clinical mapping is population-invariant by construction, verified across European, Latin American and East African origin individuals, whereas the model’s interpretation of real, ancestrally diverse diplotypes is not, degrading along an ancestry gradient, which is precisely why the mapping must be executed rather than reasoned. We do not claim full clinical validation, which would additionally require non-canonical inputs, real-world genomic and clinical data, human comparators and multi-site concordance.

Biobank-scale datasets such as the UK Biobank have become foundational resources for advancing biomedical discovery. Yet the complexity and heterogeneity of these resources, spanning genomics, imaging, clinical records, and metadata, pose substantial barriers to access and interpretation. Large Language Models (LLMs) offer a promising avenue for making such datasets more navigable through natural language interfaces. However, the extent to which current general-purpose LLMs can retrieve and synthesize biobank-specific insights has not yet been systematically evaluated. In this study, we present a reproducible, multi-metric evaluation framework to benchmark the capabilities of leading LLMs. We evaluated six leading large language models: Gemini 3 Pro, Claude Opus 4.5, Claude Sonnet 4, GPT-5.2, Mistral Large, and DeepSeek V3, on four benchmark tasks designed to assess biobank-related knowledge retrieval. We evaluate model performance across six dimensions (coverage, semantic accuracy, factual correctness, domain knowledge, reasoning quality, and biobank specificity) and assessed output consistency using curated UK Biobank references and a robust random baseline. All models outperformed the baseline by 16× to 25 × , with strong statistical separation (p < 0.001), confirming meaningful biobank-specific knowledge retrieval. Gemini 3 Pro achieved the highest overall accuracy across tasks such as keyword synthesis, institution recognition, and topic inference, while Claude Sonnet 4 demonstrated the most uniform performance across evaluation dimensions. Our benchmark provides a rigorous framework for evaluating LLMs in biomedical settings. Using the UK Biobank as a real-world testbed, we highlight both the capabilities and limitations of current models, measuring their capacity to recall structured biomedical knowledge consistent with authoritative biobank metadata.

Large language model (LLM) agents are typically deployed as clones: identical copies of a single configuration with no mechanism for heritable variation or population-level dynamics. Here we introduce Genomebook, a designed evolutionary system that encodes 26 behavioural traits across 60 diploid loci using additive, dominant, and recessive inheritance models. Twenty founder agents, each defined by a personality profile (SOUL.md) and a compiled genome (DNA.md), reproduce sexually via Mendelian segregation with de novo mutation at a base rate of 0.1% per locus per generation (3× at cognitive, immune, and metabolic hotspots). A registry of 20 synthetic conditions with defined penetrance and fitness costs introduces centrally imposed selective pressure through a compatibility scoring function. Over a single run of eight generations (626 agents, 792 social network posts), we observe trait trajectories consistent with the encoded selection rules: leadership rose from 0.525 to 0.710 under dominant inheritance, obsessive focus fell from 0.775 to 0.601 under fitness cost penalisation, and longevity declined from 0.463 to 0.209. Vocabulary diversity declined from 0.42 to 0.19, and topic inheritance between parents and offspring reached 83%, though these patterns may reflect prompt conditioning rather than purely genetic causation. Agents referenced family members spontaneously, consistent with kinship information present in the DNA.md system prompt and the LLM’s capacity for narrative elaboration. Replicated simulations (20 independent runs across three conditions) confirm that trait trajectories are consistent across seeds under standard selection, while a random mating ablation produces attenuated trends with wider variance. A non-genetic baseline (random trait assignment, no inheritance) produces flat trajectories converging to population means, confirming that the observed dynamics require genetic architecture and cannot be reproduced by unstructured parameter variation. These results establish genetic architecture as a structured, auditable, and heritable parameterisation layer for LLM agent behaviour.

Electronic health record (EHR)-linked biobanks are transforming biomedical research, enabling population-scale studies that integrate genomic, clinical, and phenotypic data. Yet as these resources proliferate, it remains unclear how their research outputs reflect global health priorities. This article presents a comprehensive review of five globally established EHR-linked biobanks: UK Biobank, the Million Veteran Program, FinnGen, the All of Us Research Program, and the Estonian Biobank. Drawing on 14,142 peer-reviewed publications from 2000 to 2024, we show how each biobank displays a distinct thematic profile, shaped by institutional mandates, population focus, and methodological design. We further evaluate alignment with global disease burden by mapping biobank-linked publications to 25 high-priority disease areas using World Health Organization disability-adjusted life years data. Our burden-adjusted gap scores and opportunity indices reveal striking underrepresentation of conditions such as malaria, tuberculosis, and diarrheal diseases when comparing biobank research output against high-priority diseases.

Global health inequities are widely documented in outcomes. However, the research systems that generate knowledge, trials, and discovery have rarely been evaluated as an integrated structure. We introduce the Health Equity Informative Metrics (HEIM) framework, a three-dimensional audit of discovery (biobank output), translation (clinical trial activity), and knowledge (semantic organisation of the scientific literature). Analysing 70 international biobanks, 563,725 registered clinical trials, 13.1 million PubMed abstracts, and 175 Global Burden of Disease categories, we demonstrate that exclusion compounds systematically for diseases that primarily burden the Global South. No WHO-classified neglected tropical disease has generated a publication from these 70 biobanks. Clinical trial sites concentrate 2.5-fold in high-income countries relative to disease burden. Diseases disproportionately affecting low-and middle-income regions are 44% more semantically isolated from mainstream biomedical research than other conditions (P < 0.0001, Cohen’s d = 1.80), limiting cross-disciplinary integration. Nine of the ten most neglected diseases across all dimensions disproportionately affect the Global South, and these disparities show no improvement over 26 years. By contrast, the trajectory of HIV/AIDS demonstrates that sustained, coordinated investment can reverse semantic isolation and integrate a once-marginalised disease into mainstream biomedical networks. HEIM reframes research inequity as a measurable, multi-stage enterprise and establishes a framework for health data accountability.

Kidney disease disproportionately affects populations of African ancestry, yet most genetic studies have focused on Europeans. Here, we present a three-stage genome-wide association study meta-analysis of estimated glomerular filtration rate in ~26,000 individuals across Eastern, Western, and Southern Africa and ~81,000 African-ancestry individuals in the diaspora. Continental African meta-analysis identifies four independent genome-wide significant loci, including two previously unreported loci. Pan-African meta-analysis identifies 19 independent loci, including three previously unreported loci. Fine-mapping reveals four loci with high causality probability, and phenome-wide analyses demonstrate pleiotropic effects on cardiometabolic and immunological traits. Notably, APOL1 high-risk variants strongly associated with kidney disease in African Americans show markedly lower frequency and attenuated effects in continental Africa, indicating potential distinct genetic architectures. Polygenic scores from genetically similar populations significantly outperformed those from distant cohorts. These findings demonstrate the necessity of conducting genomic research across diverse African populations to enable equitable health outcomes.

Background: Despite representing approximately 85% of the global population, low- and middle-income countries (LMICs) contribute less than 10% of participants to genome-wide association studies and biobank research. This disparity has profound implications for the generalisability of precision medicine. However, no standardised framework exists to quantify research equity at the biobank level or track progress over time. Methods: We developed the Health Equity Informative Metrics (HEIM) framework to quantify alignment between biobank research output and global disease burden. We analysed 75,356 PubMed-indexed publications (2000-2025) from 27 biobanks across 19 countries, mapping each to 179 disease categories from the Global Burden of Disease Study 2021. We calculated disease-specific Gap Scores measuring the mismatch between burden (disability-adjusted life years, DALYs) and research attention, biobank-level Equity Alignment Scores (EAS), and regional equity ratios comparing high-income (HIC) to LMIC research intensity. Findings: Within our 27-biobank sample, HIC biobanks produced substantially higher research output per DALY compared to LMIC biobanks (ratio: 322:1; sensitivity analyses: >100:1 across methodological variations). Regional concentration was marked: the Americas and Europe accounted for 97.8% of publications, while Africa, Eastern Mediterranean, and South-East Asia combined contributed <1%. Of 179 disease categories, 23 (13%) exhibited critical or high-severity research gaps despite substantial global burden. Only 4 of 27 biobanks (15%) achieved ‘Strong’ or ‘Moderate’ equity alignment scores; 19 (70%) were rated ‘Poor’. Six disease categories showed critical gaps, including drowning (15.7 million DALYs, 0 mapped publications) and iodine deficiency (2.3 million DALYs, 10 publications). Interpretation: The HEIM framework reveals substantial disparities in how biobank research capacity is distributed relative to global disease burden. While the precise equity ratio varies with sample selection and methodology, the fundamental pattern of profound inequity is robust across reasonable analytical choices. These findings provide baseline measurements for tracking progress toward more equitable genomic research and identify high-priority targets for intervention.