So ManyFolds, So Little Time: Efficient Protein Structure Prediction With pLMs and MSAs
Preprint
Abstract
In recent years, machine learning approaches for de novo protein structure prediction have made significant progress, culminating in AlphaFold which approaches experimental accuracies in certain settings and heralds the possibility of rapid in silico protein modelling and design. However, such applications can be challenging in practice due to the significant compute required for training and inference of such models, and their strong reliance on the evolutionary information contained in multiple sequence alignments (MSAs), which may not be available for certain targets of interest. Here, we first present a streamlined AlphaFold architecture and training pipeline that still provides good performance with significantly reduced computational burden. Aligned with recent approaches such as OmegaFold and ESMFold, our model is initially trained to predict structure from sequences alone by leveraging embeddings from the pretrained ESM-2 protein language model (pLM). We then compare this approach to an equivalent model trained on MSA-profile information only, and find that the latter still provides a performance boost – suggesting that even state-of-the-art pLMs cannot yet easily replace the evolutionary information of homologous sequences. Finally, we train a model that can make predictions from either the combination, or only one, of pLM and MSA inputs. Ultimately, we obtain accuracies in any of these three input modes similar to models trained uniquely in that setting, whilst also demonstrating that these modalities are complimentary, each regularly outperforming the other.Keywords
This publication has 18 references indexed in Scilit:
- Accurate prediction of protein structures and interactions using a three-track neural networkScience, 2021
- Highly accurate protein structure prediction with AlphaFoldNature, 2021
- Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequencesProceedings of the National Academy of Sciences of the United States of America, 2021
- Learning the language of viral evolution and escapebioRxiv, 2020
- BERTology Meets Biology: Interpreting Attention in Protein Language ModelsbioRxiv, 2020
- UniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searchesBioinformatics, 2014
- lDDT: a local superposition-free score for comparing protein structures and models using distance difference testsBioinformatics, 2013
- Scoring function for automated assessment of protein structure template qualityProteins: Structure, Function, and Bioinformatics, 2004
- The Protein Data BankNucleic Acids Research, 2000
- Principles that Govern the Folding of Protein ChainsScience, 1973