Leveraging ancestral sequence reconstruction for protein representation learning
Preprint
Abstract
Protein language models (PLMs) convert amino acid sequences into the numerical representations required to train machine learning (ML) models. Many PLMs are large (>600 M parameters) and trained on a broad span of protein sequence space. However, these models have limitations in terms of predictive accuracy and computational cost. Here, we use multiplexed Ancestral Sequence Reconstruction (mASR) to generate small but focused functional protein sequence datasets for PLM training. Compared to large PLMs, this local ancestral sequence embedding (LASE) produces representations 10-fold faster and with higher predictive accuracy. We show that due to the evolutionary nature of the ASR data, LASE produces smoother fitness landscapes in which protein variants that are closer in fitness value become numerically closer in representation space. This work contributes to the implementation of ML-based protein design in real-world settings, where data is sparse and computational resources are limited.Keywords
This publication has 77 references indexed in Scilit:
- Benchmarking of protein descriptor sets in proteochemometric modeling (part 1): comparative study of 13 amino acid descriptor setsJournal of Cheminformatics, 2013
- CD-HIT: accelerated for clustering the next-generation sequencing dataBioinformatics, 2012
- Reconstructing a Missing Link in the Evolution of a Recently Diverged Phosphotriesterase by Active-Site Loop RemodelingBiochemistry, 2012
- Diminishing returns and tradeoffs constrain the laboratory optimization of an enzymeNature Communications, 2012
- Pervasive Cryptic Epistasis in Molecular EvolutionPLOS Genetics, 2010
- Conformational sampling, catalysis, and evolution of the bacterial phosphotriesteraseProceedings of the National Academy of Sciences, 2009
- An epistatic ratchet constrains the direction of glucocorticoid receptor evolutionNature, 2009
- Interpretable Numerical Descriptors of Amino Acid SpaceJournal of Computational Biology, 2009
- Combinatorial LandscapesSIAM Review, 2002
- New Chemical Descriptors Relevant for the Design of Biologically Active Peptides. A Multivariate Characterization of 87 Amino AcidsJournal of Medicinal Chemistry, 1998