gRely: Relyability for genome trained sequence-to-expression models
Preprint
Abstract
Sequence-to-function (S2F) models predict molecular phenotypes from DNA sequence and are increasingly applied to variant effect prediction (VEP), where the goal is to quantify how genetic variants alter gene expression. However, S2F model predictions are not uniformly reliable: accuracy varies substantially across variants, genes, and tissues, and current practice relies on crude magnitude thresholding to enrich for trustworthy predictions, which discards the majority of variants where S2F models could still provide signal. We developed gRely, a meta-modeling framework that estimates the probability that a given Borzoi VEP correctly predicts eQTL direction, using 1,121 features derived from the target variant, gene, and model outputs. On held-out tissues, gRely achieves a mean average precision of 0.885 (random baseline 0.744). Critically, within the low-magnitude regime where thresholding fails entirely, gRely identifies a high-confidence subset with 76% accuracy compared to a 58% baseline, recovering reliable predictions that magnitude filtering would discard. Interpretation via SHAP reveals that in this low-magnitude regime, gene expression level and cross-replicate signal concentration replace VEP magnitude as the primary discriminators of reliability. gRely is the first framework to provide per-prediction confidence scores for S2F model VEPs, and generalizes across architectures, producing consistent improvements on AlphaGenome predictions. By making reliability quantifiable, gRely enables principled filtering rather than blanket thresholding, and marks a step toward trustworthy deployment of S2F models in genomic research and clinical applications.Keywords
This publication has 41 references indexed in Scilit:
- Advancing regulatory variant effect prediction with AlphaGenomeNature, 2026
- Predicting gene expression from DNA sequence using deep learning modelsNature Reviews Genetics, 2025
- Machine-guided design of cell-type-targeting cis-regulatory elementsNature, 2024
- Hold out the genome: a roadmap to solving the cis-regulatory codeNature, 2023
- A sequence-based global map of regulatory activity for deciphering human geneticsNature Genetics, 2022
- A compendium of uniformly processed human gene expression and splicing quantitative trait lociNature Genetics, 2021
- Highly accurate protein structure prediction with AlphaFoldNature, 2021
- Leveraging supervised learning for functionally informed fine-mapping of cis-eQTLs identifies an additional 20,913 putative causal eQTLsNature Communications, 2021
- Regulatory genomic circuitry of human disease loci by integrative epigenomicsNature, 2021
- Predicting effects of noncoding variants with deep learning–based sequence modelNature Methods, 2015