S‐PLM: Structure‐Aware Protein Language Model via Contrastive Learning Between Sequence and Structure
Open Access
- 12 December 2024
- journal article
- research article
- Published by Wiley in Advanced Science
- Vol. 12 (5) , e2404212
Abstract
Proteins play an essential role in various biological and engineering processes. Large protein language models (PLMs) present excellent potential to reshape protein research by accelerating the determination of protein functions and the design of proteins with the desired functions. The prediction and design capacity of PLMs relies on the representation gained from the protein sequences. However, the lack of crucial 3D structure information in most PLMs restricts the prediction capacity of PLMs in various applications, especially those heavily dependent on 3D structures. To address this issue, S‐PLM is introduced as a 3D structure‐aware PLM that utilizes multi‐view contrastive learning to align the sequence and 3D structure of a protein in a coordinated latent space. S‐PLM applies Swin‐Transformer on AlphaFold‐predicted protein structures to embed the structural information and fuses it into sequence‐based embedding from ESM2. Additionally, a library of lightweight tuning tools is provided to adapt S‐PLM for diverse downstream protein prediction tasks. The results demonstrate S‐PLM's superior performance over sequence‐only PLMs on all protein clustering and classification tasks, achieving competitiveness comparable to state‐of‐the‐art methods requiring both sequence and structure inputs. S‐PLM and its lightweight tuning tools are available at https://github.com/duolinwang/S-PLM/.Keywords
All Related Versions
Funding Information
- National Institutes of Health (R01LM014510)
- National Science Foundation (#2138259)
- National Centre for Supercomputing Applications (CIS230053)
- Foundation for the National Institutes of Health (R35GM126985)
This publication has 30 references indexed in Scilit:
- Discovery of deaminase functions by structure-based protein clusteringCell, 2023
- Deep metric learning for accurate protein secondary structure predictionKnowledge-Based Systems, 2022
- Learning the protein language: Evolution, structure, and functionCell Systems, 2021
- Structure-based protein function prediction using graph convolutional networksNature Communications, 2021
- The language of proteins: NLP, machine learning & protein sequencesComputational and Structural Biotechnology Journal, 2021
- Contrastive Multiview Coding2020
- GPS 5.0: An Update on the Prediction of Kinase-Specific Phosphorylation Sites in ProteinsGenomics, Proteomics & Bioinformatics, 2020
- Continual lifelong learning with neural networks: A reviewNeural Networks, 2019
- NetSurfP‐2.0: Improved prediction of protein structural features by integrated deep learningProteins: Structure, Function, and Bioinformatics, 2019
- Evaluation and improvement of multiple sequence methods for protein secondary structure predictionProteins: Structure, Function, and Bioinformatics, 1999