Mengxue Pang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Mengxue Pang has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Chris Nugent
School of Computing, Ulster University, Belfast, UK
1 shared publication - Jun Yang
School of Mathematics Science, Liaocheng University, Liaocheng, China
1 shared publication - Lishan Qiao
School of Computer Science and Technology, Shandong Jianzhu University, Jinan, China
1 shared publication - Mingxia Liu
University of North Carolina at Chapel Hill
1 shared publication - Shuai Zhang
Ulster University - Belfast Campus, Belfast, United Kingdom of Great Britain and Northern Ireland
1 shared publication - Yaru Li
Shandong Jianzhu University
1 shared publication
Latest Publications
- 21 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 29 (11) , 8460-8473
Miniformer: A Minimalist Transformer for Brain Functional Networks Analysis
Learning to estimate and classify brain functional networks (BFNs) has become an increasingly important way of predicting neurological or mental disorders at their early stages. The traditional methods conduct BFN estimation and classification in two separate steps, thus preventing the interaction and joint optimization. In contrast, Transformer provides a natural architecture to learn BFNs with downstream tasks in an end-to-end manner. Despite their great potential, Transformer-based methods involve a large number of parameters that need to be learnt from big data and often lead to poor model interpretability. Considering the challenge in acquiring data and the high demand for model interpretability in medical scenarios, in this paper, we propose a minimalist Transformer architecture, referred to as Miniformer, by simplifying the projection matrices in the self-attention module into a single diagonal matrix, which greatly reduces the number of parameters, alleviates the risk of overfitting, and improves the interpretability. Additionally, the clear physical meaning of parameters in Miniformer makes the integration of domain knowledge or prior easier and more natural. Therefore, we further develop two variants of Miniformer by incorporating sparsity for removing potentially noisy time points from fMRI signals, and smoothness for capturing the temporal correlations in fMRI signals, respectively. To evaluate the effectiveness of the proposed methods, we perform brain disease diagnosis experiments on three public datasets. The results show that Miniformer and its variants tend to achieve higher classification performance than comparison methods with good interpretability.
- 21 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 29 (11) , 8460-8473
Miniformer: A Minimalist Transformer for Brain Functional Networks Analysis
Learning to estimate and classify brain functional networks (BFNs) has become an increasingly important way of predicting neurological or mental disorders at their early stages. The traditional methods conduct BFN estimation and classification in two separate steps, thus preventing the interaction and joint optimization. In contrast, Transformer provides a natural architecture to learn BFNs with downstream tasks in an end-to-end manner. Despite their great potential, Transformer-based methods involve a large number of parameters that need to be learnt from big data and often lead to poor model interpretability. Considering the challenge in acquiring data and the high demand for model interpretability in medical scenarios, in this paper, we propose a minimalist Transformer architecture, referred to as Miniformer, by simplifying the projection matrices in the self-attention module into a single diagonal matrix, which greatly reduces the number of parameters, alleviates the risk of overfitting, and improves the interpretability. Additionally, the clear physical meaning of parameters in Miniformer makes the integration of domain knowledge or prior easier and more natural. Therefore, we further develop two variants of Miniformer by incorporating sparsity for removing potentially noisy time points from fMRI signals, and smoothness for capturing the temporal correlations in fMRI signals, respectively. To evaluate the effectiveness of the proposed methods, we perform brain disease diagnosis experiments on three public datasets. The results show that Miniformer and its variants tend to achieve higher classification performance than comparison methods with good interpretability.