A lightweight codon-based DNA Transformer for Regulatory Region Identification in the Genome
Authors:
Preprint
Abstract
We developed a lightweight codon-based DNA Transformer equipped with multi-head self-attention and an adaptive classifier head, which achieves exon intron classification with high accuracy and also has moderate accuracy in CDS classification and splice site recognition. We named this model as ExIT (Exon-Intron Transformer). We have implemented codon tokenization for this model. This has been validated on the human genome with external validation from the chimpanzee genome. Further benchmarking has implied that our model is better than the existing models in the above tasks.Keywords
This publication has 7 references indexed in Scilit:
- Transformers significantly improve splice site predictionCommunications Biology, 2024
- Nucleotide Transformer: building and evaluating robust foundation models for human genomicsNature Methods, 2024
- Human exons and introns classification using pre-trained Resnet-50 and GoogleNet models and 13-layers CNN modelJournal of Genetic Engineering and Biotechnology, 2024
- Attention Is All You NeedarXiv, 2023
- SpliceVault predicts the precise nature of variant-associated mis-splicingNature Genetics, 2023
- DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genomeBioinformatics, 2021
- DNA-Level Splice Junction Prediction using Deep Recurrent Neural NetworksarXiv, 2015