CD-GPT As a Biological Foundation Model Bridging the Gap between Molecular Sequences Through Central Dogma
Preprint
Abstract
The central dogma serves as a fundamental framework for understanding the flow and expression of genetic information within living organisms, facilitating the connection of diverse biological sequences across molecule types. In this study, we present CD-GPT (Central Dogma Generative Pretrained Transformer), a generative biological foundation model with 1 billion parameters, aiming to capture the sequence relationships between DNA, RNA, and proteins. We model sequences in a unified representational space and employ a shared, multi-molecule vocabulary to narrow their distances in the embedding space effectively. Through extensive pretraining on nucleotide and amino acid sequence data, CD-GPT exhibits exceptional performance in a wide range of predictive and generative downstream tasks, including mono-molecular and multi-molecular analyses. Notably, CD-GPT excels in tasks such as genomic element detection, protein property prediction, RNA-protein interaction identification and also generative tasks like protein generation and reverse translation. The versatility of CD-GPT opens up promising avenues for advanced multi-omics analysis.Keywords
This publication has 68 references indexed in Scilit:
- RoFormer: Enhanced transformer with Rotary Position EmbeddingNeurocomputing, 2023
- Using machine learning approaches for multi-omics data analysis: A reviewBiotechnology Advances, 2021
- How RNA-Binding Proteins Interact with RNA: Molecules and MechanismsMolecular Cell, 2020
- NetSurfP‐2.0: Improved prediction of protein structural features by integrated deep learningProteins: Structure, Function, and Bioinformatics, 2019
- The coming of age of de novo protein designNature, 2016
- Understanding splicing regulation through RNA splicing mapsTrends in Genetics, 2011
- Codon optimization can improve expression of human genes in Escherichia coli: A multi-gene studyProtein Expression and Purification, 2008
- RNA‐binding proteins and post‐transcriptional gene regulationFEBS Letters, 2008
- Evaluation and improvement of multiple sequence methods for protein secondary structure predictionProteins: Structure, Function, and Bioinformatics, 1999
- Central Dogma of Molecular BiologyNature, 1970