Transformers in Protein: A Survey
Authors:
- Xiaowen LingXLXiaowen Ling
- Zhiqiang LiZLZhiqiang Li
- Yanbin WangYWYanbin Wang
- Zhuhong YouZhuhong You
Preprint
Abstract
As protein informatics advances rapidly, the demand for enhanced predictive accuracy, structural analysis, and functional understanding has intensified. Transformer models, as powerful deep learning architectures, have demonstrated unprecedented potential in addressing diverse challenges across protein research. However, a comprehensive review of Transformer applications in this field remains lacking. This paper bridges this gap by surveying over 100 studies, offering an in-depth analysis of practical implementations and research progress of Transformers in protein-related tasks. Our review systematically covers critical domains, including protein structure prediction, function prediction, protein-protein interaction analysis, functional annotation, and drug discovery/target identification. To contextualize these advancements across various protein domains, we adopt a domain-oriented classification system. We first introduce foundational concepts: the Transformer architecture and attention mechanisms, categorize Transformer variants tailored for protein science, and summarize essential protein knowledge. For each research domain, we outline its objectives and background, critically evaluate prior methods and their limitations, and highlight transformative contributions enabled by Transformer models. We also curate and summarize pivotal datasets and open-source code resources to facilitate reproducibility and benchmarking. Finally, we discuss persistent challenges in applying Transformers to protein informatics and propose future research directions. This review aims to provide a consolidated foundation for the synergistic integration of Transformer and protein informatics, fostering further innovation and expanded applications in the field.Keywords
All Related Versions
This publication has 56 references indexed in Scilit:
- TransConv: convolution-infused transformer for protein secondary structure predictionJournal of Molecular Modeling, 2025
- Transformers and Large Language Models for Chemistry and Drug Discovery2024
- Machine learning: its challenges and opportunities in plant system biologyApplied Microbiology and Biotechnology, 2022
- Machine learning solutions for predicting protein–protein interactionsWIREs Computational Molecular Science, 2022
- Protein tertiary structure prediction and refinement using deep learning and Rosetta in CASP14Proteins: Structure, Function, and Bioinformatics, 2021
- Deep Learning in ProteomicsProteomics, 2020
- GOAP: A Generalized Orientation-Dependent, All-Atom Statistical Potential for Protein Structure PredictionBiophysical Journal, 2011
- MOLMOL: A program for display and analysis of macromolecular structuresJournal of Molecular Graphics, 1996
- Prediction of Protein Secondary Structure at Better than 70% AccuracyJournal of Molecular Biology, 1993
- Analysis of the accuracy and implications of simple methods for predicting the secondary structure of globular proteinsJournal of Molecular Biology, 1978