Uncovering Features of ProteinMPNN with Sparse Autoencoders
Preprint
Abstract
Customization and modification of machine learning derived sequences can be achieved by understanding the mechanics behind state-of-the-art protein design models such as ProteinMPNN. Uninterpretability is hypothesized to be due largely to polysemanticity where neural networks store more features than they have dimensions. Here, we introduce sparse autoencoders (SAEs) into ProteinMPNN which expand its latent space and uncovers correlations between the activations in sparse dimensions and the presence of protein features such as secondary structure, residue volume, residue shape, disulfide bridges, and coordination to ligands. Additionally, we show that sparse activations can be manipulated to produce a variant of the short chain variable fragment Brolucizumab that is more thermostable than variants produced by traditional machine learning methods based on high temperature molecular dynamics simulations. Thus, SAEs provide both mechanistic understanding of protein design models and allow customizable manipulation of sequence prediction.This publication has 36 references indexed in Scilit:
- Engineering scFv immunotherapies: From CAR T cells to bispecific antibodiesInternational Immunopharmacology, 2025
- Determining key residues of engineered scFv antibody variants with improved MMP-9 binding using deep sequencing and machine learningComputational and Structural Biotechnology Journal, 2024
- Thermostability and binding properties of single‐chained Fv fragments derived from therapeutic antibodiesProtein Science, 2024
- Explanation in artificial intelligence: Insights from the social sciencesArtificial Intelligence, 2019
- Visualizing and Understanding Convolutional Networks2014
- MDAnalysis: A toolkit for the analysis of molecular dynamics simulationsJournal of Computational Chemistry, 2011
- Scalable molecular dynamics with NAMDJournal of Computational Chemistry, 2005
- Stability engineering of antibody single-chain Fv fragmentsJournal of Molecular Biology, 2001
- VMD: Visual molecular dynamicsJournal of Molecular Graphics, 1996
- Basic Local Alignment Search ToolJournal of Molecular Biology, 1990