Grounding Health AI: Architecture and Evaluation of a Domain-Expert Metabolic Health Agent
Preprint
Abstract
General-purpose language models generate fluent health reports that can fabricate derived clinical met-rics. In an illustrative comparison on identical two-week CGM and meal data, leading foundation models produced reports with invented MAGE values, inflated meal counts, and unreferenced complication-risk projections — failures invisible to non-expert readers and plausible enough to mislead clinicians. We describe the HPP Personal Health Agent (PHA), a metabolic health agent that grounds generation in four layers: the Human Phenotype Project (HPP), a deep-phenotyped cohort of 13,000+ participants sup-plying population references and trained predictive models; 21 domain-expert tools and trained-model wrappers that compute clinical metrics and risk predictions; declarative behavioural skills that constrain what the model may claim; and 21 automated evals across 8 categories developed via a test-driven cycle in which each eval encodes a failure mode discovered during iterative development. In a 210-report ma-trix (14 participants × 3 prompts × 5 system conditions), the gains are largest on the system’s primary use case — meal-grounded metabolic reports, the report it was designed for — where the full system raises a deterministic form/provenance score from 0.37 (the same foundation model with no tools or skills) to 0.91; this score measures structural completeness, numerical accuracy, tool grounding, and clinical-language compliance — a necessary condition for trustworthy health reporting, with clinical quality as a complementary axis examined qualitatively. A skills-vs-tools decomposition shows the two layers act on different axes: tools drive numerical accuracy (≈14% → 90% of reported metrics correct), while the declarative skills add most of the remaining gain in citations, completeness, and structure (tools alone recover only part of the gap, 0.49 from the same 0.37 baseline). The lift generalises beyond the primary use case — to a second metabolic prompt (0.72) and a cardiovascular extension (0.70), each from a 0.37–0.39 baseline. The architecture extends across clinical domains: adding a SCORE2 cardiovascular risk tool and a corresponding skill — with no changes to orchestration, eval harness, or existing tools — produced a cardiovascular risk report from the same system. Trustworthy domain-specialised health AI is a systems design problem: deep-phenotyped cohort data, domain-expert tools and models, and eval-driven development together form a replicable pattern.Keywords
This publication has 19 references indexed in Scilit:
- A foundation model for continuous glucose monitoring dataNature, 2026
- Transforming wearable data into personal health insights using large language model agentsNature Communications, 2026
- AgentMD: Empowering language agents for risk prediction with large-scale clinical tool learningNature Communications, 2025
- CGMacros: a pilot scientific dataset for personalized nutrition and diet monitoringScientific Data, 2025
- A personal health large language model for sleep and fitness coachingNature Medicine, 2025
- Deep phenotyping of health–disease continuum in the Human Phenotype ProjectNature Medicine, 2025
- Dietaryindex: a user-friendly and versatile R package for standardizing dietary pattern analysis in epidemiological and clinical studiesThe American Journal of Clinical Nutrition, 2024
- Large language models encode clinical knowledgeNature, 2023
- CGMap: Characterizing continuous glucose monitor data in thousands of non-diabetic individualsCell Metabolism, 2023
- The UN Decade of Nutrition, the NOVA food classification and the trouble with ultra-processingPublic Health Nutrition, 2017