DSDongjing Shan

Dongjing Shan

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Dongjing Shan has a total of 21 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    M3Site: multiclass multimodal learning for protein active site identification and classification

    by Song Ouyang, Yong Luo, Huiyu Cai, Kehua Su, Fei Liao, Na Zhan, Huangxuan Zhao, Tailang Yin, Lin Zhao, Dongjing Shan

    Accurately identifying and classifying protein active sites is crucial for understanding protein mechanisms, drug design, and synthetic biology. Current methods often rely on binary classification and single-modal data, limiting their scope. To address these limitations, we propose M$^{3}$Site, a multimodal framework that integrates protein sequence embeddings, structural graph representations, and functional text annotations for residue-level, multiclass active site prediction. Built upon a curated dataset of 25 883 proteins sourced from UniProt and AlphaFold2, M$^{3}$Site leverages pretrained protein language models, equivariant graph neural networks, and biomedical language models for feature extraction. The function informed cross-attention module enables cross-modal feature fusion, while the adaptive weighted fusion mechanism balances modality contributions. A compound loss function tackles class imbalance, ensuring robust performance. Experimental results show M$^{3}$Site significantly outperforms existing models, and an interactive application has been developed to enhance its practical utility for predictions and visualizations. The dataset, source code for experiments, and interactive application are publicly available at https://github.com/Gift-OYS/M3Site.


  • Open Access

    Graph-based vision transformer with sparsity for training on small datasets from scratch

    by Peng Li, Lu Huang, Jin Li, Haiyan Yan, Dongjing Shan

    Vision Transformers (ViTs) have achieved impressive results in large-scale image classification. However, when training from scratch on small datasets, there is still a significant performance gap between ViTs and Convolutional Neural Networks (CNNs), which is attributed to the lack of inductive bias. To address this issue, we propose a Graph-based Vision Transformer (GvT) that utilizes graph convolutional projection and graph-pooling. In each block, queries and keys are calculated through graph convolutional projection based on the spatial adjacency matrix, while dot-product attention is used in another graph convolution to generate values. When using more attention heads, the queries and keys become lower-dimensional, making their dot product an uninformative matching function. To overcome this low-rank bottleneck in attention heads, we employ talking-heads technology based on bilinear pooled features and sparse selection of attention tensors. This allows interaction among filtered attention scores and enables each attention mechanism to depend on all queries and keys. Additionally, we apply graph-pooling between two intermediate blocks to reduce the number of tokens and aggregate semantic information more effectively. Our experimental results show that GvT produces comparable or superior outcomes to deep convolutional networks and surpasses vision transformers without pre-training on large datasets.


  • IUPAC-GPT: an IUPAC-based large-scale molecular pre-trained model for property prediction and molecule generation

    by Jiashun Mao, Tang Sui, Kwang-Hwi Cho, Kyoung Tai No, Jianmin Wang, Dongjing Shan

    The international union of pure and applied chemistry (IUPAC) name nomenclature constitutes a universally recognized standard naming system for allocating names to chemical compounds and is a human-friendly, substructure molecular language. Simplified molecular input line entry system (SMILES) string is currently the most popular molecular representation language and is a computer-friendly, atomic-level molecular language. Considering the readability of IUPAC name and the advantages of SMILES string, it becomes significant to investigate the distinctions of these two molecular languages in term of molecular generation and regression/classification tasks. Thus, we have developed a chemical language model named IUPAC-GPT. Besides molecular generation, we have also incorporated the freezing of IUPAC-GPT model parameters and the attachment of trainable lightweight networks for fine-tuning regression/classification tasks. The results indicate that pre-trained IUPAC-GPT can grasp general knowledge that can be effectively transferred to downstream tasks such as molecular generation, binary classification, and property regression prediction. Furthermore, when utilizing the same configuration, IUPAC-GPT exhibited superior performance compared to the smilesGPT model in term of some property prediction tasks. Overall, transformer-like language models pretrained on IUPAC corpora emerge as promising alternatives, offering improved performance in terms of interpretability and semantic abstraction (chemical groups and modifications) when compared to models pretrained on SMILES corpora.


  • Open Access

    Bridging the Gap with Convolutional networks: A Graph-based Vision Transformer with Sparsity for Training on Small Datasets from Scratch

    by Dongjing Shan, Jin Li, Lisha Zhong, Biao Qu, Qi Han

    Vision Transformers (ViTs) have achieved impressive results in large-scale image classification. However, when training from scratch on small datasets, there is still a significant performance gap between ViTs and Convolutional Neural Networks (CNNs), which is attributed to the lack of inductive bias. To address this issue, we propose a Graph-based Vision Transformer (GvT) that utilizes graph convolutional projection and graphpooling. In each block, queries and keys are calculated through graph convolutional projection based on the spatial adjacency matrix, while dot-product attention is used in another graph convolution to generate values. When using more attention heads, the queries and keys become lower-dimensional, making their dot product an uninformative matching function. To overcome this low-rank bottleneck in attention heads, we employ talkingheads technology based on bilinear pooled features and sparse selection of attention tensors. This allows interaction among filtered attention scores and enables each attention mechanism to depend on all queries and keys. Additionally, we apply graphpooling between two intermediate blocks to reduce the number of tokens and aggregate semantic information more effectively. Our experimental results show that GvT produces comparable or superior outcomes to deep convolutional networks and surpasses vision transformers without pre-training on large datasets. The code for our proposed model is publicly available on the website: GitHub/GvT.