Learning the limits of protein structure models with NMR chemical shifts
Preprint
Abstract
Protein foundation models are intended to implicitly learn the chemical rules underlying protein function. However, systematically evaluating what biochemistry these models have learned remains difficult, as most downstream tasks lack experimental data fine-grained enough to expose their failure modes. NMR chemical shifts fill this gap: as direct per-atom reporters of local chemical environment, they offer a sensitive probe of structure-based representations. In this work, we present a systematically curated chemical shift dataset designed to minimize experimental confounders. We use it to benchmark diverse protein foundation models in their ability to predict backbone 15 N chemical shifts, atoms that are known to be sensitive to many effects. The largest error across all models is in core residues with evidence of fast-timescale dynamics, which we investigate mechanistically using molecular dynamics simulations. Given the biological relevance of structured dynamic regions, we trained a new model based on the best of these embeddings, SLAE, to predict its own error. “SLAE-ERR” achieves a mean absolute error of 0.76 ppm for predicting error. Together, we hope that the datasets, evaluation tasks, and models presented here are sources of guidance for protein design model improvement.This publication has 84 references indexed in Scilit:
- DSSP 4: FAIR annotation of protein secondary structureProtein Science, 2025
- Effect of solvent and protein dynamics in ligand recognition and inhibition of aminoglycoside adenyltransferase 2″‐IaProtein Science, 2017
- Improved side‐chain torsion potentials for the Amber ff99SB protein force fieldProteins: Structure, Function, and Bioinformatics, 2010
- Probing electric fields in proteins in solution by NMR spectroscopyProteins: Structure, Function, and Bioinformatics, 2008
- Linear analysis of carbon-13 chemical shift differences and its application to the detection and correction of errors in referencing and spin system identificationsJournal of Biomolecular NMR, 2005
- A simple method to adjust inconsistently referenced 13C and 15N chemical shift assignments of proteinsJournal of Biomolecular NMR, 2005
- Helix cappingProtein Science, 1998
- 1H, 13C and 15N chemical shift referencing in biomolecular NMRJournal of Biomolecular NMR, 1995
- 1H, 13C and 15N random coil NMR chemical shifts of the common amino acids. I. Investigations of nearest-neighbor effectsJournal of Biomolecular NMR, 1995
- Dictionary of protein secondary structure: Pattern recognition of hydrogen‐bonded and geometrical featuresBiopolymers, 1983