Chunyuan Li
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Chunyuan Li has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Hehuan Ma
The University of Texas at Arlington
1 shared publication - Jinyu Yang
Department of Computer Science and Engineering, University of Texas at Arlington, Arlington, TX 76019, USA
1 shared publication - Junzhou Huang
The University of Texas at Arlington
1 shared publication - Weizhi An
Department of Computer Science and Engineering, University of Texas at Arlington, Arlington, TX 76019, USA
1 shared publication - Yatao Bian
Tencent AI Lab, Shenzhen 518000, China
1 shared publication - Yuzhi Guo
The University of Texas at Arlington
1 shared publication
Latest Publications
- Open Access
- 12 June 2024
- Published by MDPI AG in BioMedInformatics
- Vol. 4 (2) , 1556-1571
Advancing DNA Language Models through Motif-Oriented Pre-Training with MoDNA
Acquiring meaningful representations of gene expression is essential for the accurate prediction of downstream regulatory tasks, such as identifying promoters and transcription factor binding sites. However, the current dependency on supervised learning, constrained by the limited availability of labeled genomic data, impedes the ability to develop robust predictive models with broad generalization capabilities. In response, recent advancements have pivoted towards the application of self-supervised training for DNA sequence modeling, enabling the adaptation of pre-trained genomic representations to a variety of downstream tasks. Departing from the straightforward application of masked language learning techniques to DNA sequences, approaches such as MoDNA enrich genome language modeling with prior biological knowledge. In this study, we advance DNA language models by utilizing the Motif-oriented DNA (MoDNA) pre-training framework, which is established for self-supervised learning at the pre-training stage and is flexible enough for application across different downstream tasks. MoDNA distinguishes itself by efficiently learning semantic-level genomic representations from an extensive corpus of unlabeled genome data, offering a significant improvement in computational efficiency over previous approaches. The framework is pre-trained on a comprehensive human genome dataset and fine-tuned for targeted downstream tasks. Our enhanced analysis and evaluation in promoter prediction and transcription factor binding site prediction have further validated MoDNA’s exceptional capabilities, emphasizing its contribution to advancements in genomic predictive modeling.
- 12 June 2024
- Published by MDPI AG in BioMedInformatics
- Vol. 4 (2) , 1556-1571
Open AccessAdvancing DNA Language Models through Motif-Oriented Pre-Training with MoDNA
Acquiring meaningful representations of gene expression is essential for the accurate prediction of downstream regulatory tasks, such as identifying promoters and transcription factor binding sites. However, the current dependency on supervised learning, constrained by the limited availability of labeled genomic data, impedes the ability to develop robust predictive models with broad generalization capabilities. In response, recent advancements have pivoted towards the application of self-supervised training for DNA sequence modeling, enabling the adaptation of pre-trained genomic representations to a variety of downstream tasks. Departing from the straightforward application of masked language learning techniques to DNA sequences, approaches such as MoDNA enrich genome language modeling with prior biological knowledge. In this study, we advance DNA language models by utilizing the Motif-oriented DNA (MoDNA) pre-training framework, which is established for self-supervised learning at the pre-training stage and is flexible enough for application across different downstream tasks. MoDNA distinguishes itself by efficiently learning semantic-level genomic representations from an extensive corpus of unlabeled genome data, offering a significant improvement in computational efficiency over previous approaches. The framework is pre-trained on a comprehensive human genome dataset and fine-tuned for targeted downstream tasks. Our enhanced analysis and evaluation in promoter prediction and transcription factor binding site prediction have further validated MoDNA’s exceptional capabilities, emphasizing its contribution to advancements in genomic predictive modeling.