Jing Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Jing Zhang has a total of 27 co-authors. Here are the top co-authors by number of shared publications:
- Aiping Liu
University of Science and Technology of China
7 shared publications - Xun Chen
University of Science and Technology of China
5 shared publications - Yu Liu
Department of Biomedical Engineering, Hefei University of Technology, Hefei, China
4 shared publications - Lubin Gan
University of Science and Technology of China, Hefei, China
2 shared publications - Min Gao
Department of Electrocardiogram The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China Hefei Anhui China
2 shared publications Qingguo Xie
School of Information Science and Technology, University of Science and Technology of China, Hefei, China
2 shared publications
Latest Publications
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 3635-3644
MeDKCoOp: Dual Knowledge-guided Graph Prompt Learning for Biomedical Vision-Language Models
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 3635-3644
Open AccessMeDKCoOp: Dual Knowledge-guided Graph Prompt Learning for Biomedical Vision-Language Models
- 27 October 2025
- Published by Elsevier in Expert Systems with Applications
- Vol. 300
Enhancing zero-shot brain tumor subtype classification via fine-grained patch-text alignment
- 27 October 2025
- Published by Elsevier in Expert Systems with Applications
- Vol. 300
Enhancing zero-shot brain tumor subtype classification via fine-grained patch-text alignment
- 1 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 24 (12) , 19428-19441
Medical Image Fusion Based on Multilevel Bidirectional Feature Interaction Network
Medical image fusion integrates multi-modal images with complementary information to enhance the image quality in clinical diagnosis. This process typically involves three steps including feature extraction, feature fusion, and image reconstruction. However, most image fusion methods have limitations in simultaneously extracting specific and shared information from different-modal images while considering cross-modal interaction, leading to incomplete feature extraction and fusion. Besides, the multi-level features interaction of most existing methods is insufficient, leading to defective utilization of fused information under different receptive fields. To address these issues, we propose an end-to-end medical image fusion network based on multi-level bidirectional feature interaction. The bidirectional feature interaction is manifested vertically and horizontally. Firstly, we design a cross-modal interactive feature extraction module based on two modality-independent branches and a modality-shared branch, achieving vertical feature interaction between different-modal features at the same level. Besides, we proposed a forward and backward interaction module (FBIM) to achieve multi-level horizontal feature interaction, which can bridge the gap of multi-level fused features and promote the utilization of fused information under different receptive fields. Additionally, we design a fusion module to generate two weight maps based on the modality-shared branch to fuse two modality-independent features. Qualitative and quantitative experiments demonstrate the superiority of our method over the state-of-the-art fusion methods.
- 1 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 24 (12) , 19428-19441
Medical Image Fusion Based on Multilevel Bidirectional Feature Interaction Network
Medical image fusion integrates multi-modal images with complementary information to enhance the image quality in clinical diagnosis. This process typically involves three steps including feature extraction, feature fusion, and image reconstruction. However, most image fusion methods have limitations in simultaneously extracting specific and shared information from different-modal images while considering cross-modal interaction, leading to incomplete feature extraction and fusion. Besides, the multi-level features interaction of most existing methods is insufficient, leading to defective utilization of fused information under different receptive fields. To address these issues, we propose an end-to-end medical image fusion network based on multi-level bidirectional feature interaction. The bidirectional feature interaction is manifested vertically and horizontally. Firstly, we design a cross-modal interactive feature extraction module based on two modality-independent branches and a modality-shared branch, achieving vertical feature interaction between different-modal features at the same level. Besides, we proposed a forward and backward interaction module (FBIM) to achieve multi-level horizontal feature interaction, which can bridge the gap of multi-level fused features and promote the utilization of fused information under different receptive fields. Additionally, we design a fusion module to generate two weight maps based on the modality-shared branch to fuse two modality-independent features. Qualitative and quantitative experiments demonstrate the superiority of our method over the state-of-the-art fusion methods.
- 18 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 24 (8) , 12806-12818
LeGFusion: Locally Enhanced Global Learning for Multimodal Image Fusion
Multi-modal image fusion (MMIF) can provide more comprehensive scene characteristics by synthesizing a single image from multi-sensor images of the same scene, which works out the limitation of single-type hardwares. To handle MMIF tasks, current deep learning-based methods usually employ convolutional neural networks or combine transformer to extract local and global contextual information of source images. However, none of existing works fully explore contextual information both across modalities and within single modalities, leading to limited fusion results. To this end, we propose a new MMIF method via locally-enhanced global learning, termed as LeGFusion. Specifically, the network of LeGFusion is devised based on locally-enhanced transformer block (LETB), which can capture long-range dependencies benefiting from non-overlapping window-based self-attention while capturing useful local context with the utilization of the convolution operator into transformer. On the one hand, several LETBs are deployed to extract global contexts from each modality while emphasizing its local information. On the other hand, the fusion module that also consists of LETBs is designed to integrate multi-modal features by perceiving cross-modal local and global interactions. Powered by these intra-modal and inter-modal contextual information exploration, the proposed LeGFusion enjoys a high capability in capturing significant complementary information for image fusion. Extensive experiments are conducted on two types of MMIF tasks, including infrared-visible image fusion and medical image fusion. The qualitative and quantitative evaluation results demonstrate the superiority of our LeGFusion over state-of-the-art methods. Furthermore, we validate the generalization ability of LeGFusion without fine-tuning, and achieve fantastic results.
- 18 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 24 (8) , 12806-12818
LeGFusion: Locally Enhanced Global Learning for Multimodal Image Fusion
Multi-modal image fusion (MMIF) can provide more comprehensive scene characteristics by synthesizing a single image from multi-sensor images of the same scene, which works out the limitation of single-type hardwares. To handle MMIF tasks, current deep learning-based methods usually employ convolutional neural networks or combine transformer to extract local and global contextual information of source images. However, none of existing works fully explore contextual information both across modalities and within single modalities, leading to limited fusion results. To this end, we propose a new MMIF method via locally-enhanced global learning, termed as LeGFusion. Specifically, the network of LeGFusion is devised based on locally-enhanced transformer block (LETB), which can capture long-range dependencies benefiting from non-overlapping window-based self-attention while capturing useful local context with the utilization of the convolution operator into transformer. On the one hand, several LETBs are deployed to extract global contexts from each modality while emphasizing its local information. On the other hand, the fusion module that also consists of LETBs is designed to integrate multi-modal features by perceiving cross-modal local and global interactions. Powered by these intra-modal and inter-modal contextual information exploration, the proposed LeGFusion enjoys a high capability in capturing significant complementary information for image fusion. Extensive experiments are conducted on two types of MMIF tasks, including infrared-visible image fusion and medical image fusion. The qualitative and quantitative evaluation results demonstrate the superiority of our LeGFusion over state-of-the-art methods. Furthermore, we validate the generalization ability of LeGFusion without fine-tuning, and achieve fantastic results.
- 4 April 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 23 (9) , 9796-9807
Multimodal Image Fusion via Self-Supervised Transformer
Multi-modal image fusion (MMIF) aims to synthesize one more informative image from complementary multi-modal images. Due to the absence of ground truth, most MMIF works utilize unsupervised loss functions to preserve specific source information into the fused image, suffering from subjective information preservation and high demand for a large training set. Self-supervised learning is a feasible technology to avoid these issues. Current self-supervised MMIF works pretrain the network using a large natural image set via the vanilla image reconstruction task, then implement image fusion by integrating fusion rules into the well-trained network. However, they ignore the domain discrepancy between training data (i.e., the natural image set) and test data (i.e., the MMIF set). In addition, existing MMIF methods are mostly built based on traditional convolutional neural networks, leading to the inability of modelling long-range contextual relationships of source modalities. To this end, we propose a self-supervised transformer for multi-modal image fusion. Under the self-supervised framework, we design random image super-resolution (RSR) as the pretext task to train the network. Both RSR and MMIF have similar multi-resolution inputs and common goals: image restoration and enhancement, which are beneficial for diminishing the domain discrepancy between the train and test data. Moreover, we present a transformer-based fusion network for capturing both local and global information of source modalities. The proposed method is validated on multiple MMIF datasets. Qualitative and quantitative experimental results demonstrate that the proposed method can yield superior fusion quality compared to the state-of-the-art image fusion methods.
- 4 April 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 23 (9) , 9796-9807
Multimodal Image Fusion via Self-Supervised Transformer
Multi-modal image fusion (MMIF) aims to synthesize one more informative image from complementary multi-modal images. Due to the absence of ground truth, most MMIF works utilize unsupervised loss functions to preserve specific source information into the fused image, suffering from subjective information preservation and high demand for a large training set. Self-supervised learning is a feasible technology to avoid these issues. Current self-supervised MMIF works pretrain the network using a large natural image set via the vanilla image reconstruction task, then implement image fusion by integrating fusion rules into the well-trained network. However, they ignore the domain discrepancy between training data (i.e., the natural image set) and test data (i.e., the MMIF set). In addition, existing MMIF methods are mostly built based on traditional convolutional neural networks, leading to the inability of modelling long-range contextual relationships of source modalities. To this end, we propose a self-supervised transformer for multi-modal image fusion. Under the self-supervised framework, we design random image super-resolution (RSR) as the pretext task to train the network. Both RSR and MMIF have similar multi-resolution inputs and common goals: image restoration and enhancement, which are beneficial for diminishing the domain discrepancy between the train and test data. Moreover, we present a transformer-based fusion network for capturing both local and global information of source modalities. The proposed method is validated on multiple MMIF datasets. Qualitative and quantitative experimental results demonstrate that the proposed method can yield superior fusion quality compared to the state-of-the-art image fusion methods.
- Open Access
- 1 January 2023
- Published by Wiley in Journal of Healthcare Engineering
- Vol. 2023 (1) , 1755121
ECG‐Based Multiclass Arrhythmia Classification Using Beat‐Level Fusion Network
Cardiovascular disease (CVD) is one of the most severe diseases threatening human life. Electrocardiogram (ECG) is an effective way to detect CVD. In recent years, many methods have been proposed to detect arrhythmia using 12‐lead ECG. In particular, deep learning methods have been proven to be effective and have been widely used. The attention mechanism has attracted extensive attention in many fields in a series of deep learning methods. Off‐the‐shelf solutions based on deep learning and attention mechanism for ECG classification mostly give weights to time points. None of the existing methods were considered using the attention mechanism dealing with ECG signals at the level of heartbeats. In this paper, we propose a beat‐level fusion net (BLF‐Net) for multiclass arrhythmia classification by assigning weights at the heartbeat level, according to the contribution of the heartbeat to diagnostic results. This algorithm consists of three steps: (1) segmenting the long ECG signal into short beats; (2) using a neural network to extract features from heartbeats; and (3) assigning weights to features extracted from heartbeats using an attention mechanism. We test our algorithm on the PTB‐XL database and have superiority over state‐of‐the‐art performance on six classification tasks. Besides, the principle of this architecture is clarified by visualizing the weight of the attention mechanism. The proposed BLF‐Net is shown to be useful and automatically provides an effective network structure for arrhythmia classification, which is capable of aiding cardiologists in arrhythmia diagnosis.
- 1 January 2023
- Published by Wiley in Journal of Healthcare Engineering
- Vol. 2023 (1) , 1755121
Open AccessECG‐Based Multiclass Arrhythmia Classification Using Beat‐Level Fusion Network
Cardiovascular disease (CVD) is one of the most severe diseases threatening human life. Electrocardiogram (ECG) is an effective way to detect CVD. In recent years, many methods have been proposed to detect arrhythmia using 12‐lead ECG. In particular, deep learning methods have been proven to be effective and have been widely used. The attention mechanism has attracted extensive attention in many fields in a series of deep learning methods. Off‐the‐shelf solutions based on deep learning and attention mechanism for ECG classification mostly give weights to time points. None of the existing methods were considered using the attention mechanism dealing with ECG signals at the level of heartbeats. In this paper, we propose a beat‐level fusion net (BLF‐Net) for multiclass arrhythmia classification by assigning weights at the heartbeat level, according to the contribution of the heartbeat to diagnostic results. This algorithm consists of three steps: (1) segmenting the long ECG signal into short beats; (2) using a neural network to extract features from heartbeats; and (3) assigning weights to features extracted from heartbeats using an attention mechanism. We test our algorithm on the PTB‐XL database and have superiority over state‐of‐the‐art performance on six classification tasks. Besides, the principle of this architecture is clarified by visualizing the weight of the attention mechanism. The proposed BLF‐Net is shown to be useful and automatically provides an effective network structure for arrhythmia classification, which is capable of aiding cardiologists in arrhythmia diagnosis.
- 22 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 71 (00189456) , 1-11
Transformer-Based End-to-End Anatomical and Functional Image Fusion
Medical image fusion aims to derive complementary information from medical images with different modalities and is becoming increasingly important in clinical applications. The design of fusion strategy plays a key role in achieving high-quality fusion results. Existing methods usually employ handcrafted fusion rules or convolution-based networks to fuse multimodal medical images. However, these fusion strategies are insufficiently fine-grained and cannot capture global information of multimodal images effectively. Moreover, for deep learning-based fusion methods, they always concatenate source images/deep features of different modalities to input into the neural network, easily leading to inadequate information utilization from source images. To address these problems, we propose a transformer-based end-to-end framework for medical image fusion, termed TransFusion. The proposed TransFusion introduces the transformer as the fusion strategy, which utilizes its self-attention mechanism to incorporate the global contextual information of multimodal features and fuse them adequately. Besides, unlike traditional parallel multibranch architectures or shared networks used for multiple inputs, we design branch networks that interact through fusion transformers at multiple scales to utilize the information of different modalities more adequately. A natural advantage of our design is the ability to aggregate global multimodal features through self-attention. Both qualitative and quantitative experiments demonstrate the superiority of our method over the state-of-the-art fusion methods.
- 22 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 71 (00189456) , 1-11
Transformer-Based End-to-End Anatomical and Functional Image Fusion
Medical image fusion aims to derive complementary information from medical images with different modalities and is becoming increasingly important in clinical applications. The design of fusion strategy plays a key role in achieving high-quality fusion results. Existing methods usually employ handcrafted fusion rules or convolution-based networks to fuse multimodal medical images. However, these fusion strategies are insufficiently fine-grained and cannot capture global information of multimodal images effectively. Moreover, for deep learning-based fusion methods, they always concatenate source images/deep features of different modalities to input into the neural network, easily leading to inadequate information utilization from source images. To address these problems, we propose a transformer-based end-to-end framework for medical image fusion, termed TransFusion. The proposed TransFusion introduces the transformer as the fusion strategy, which utilizes its self-attention mechanism to incorporate the global contextual information of multimodal features and fuse them adequately. Besides, unlike traditional parallel multibranch architectures or shared networks used for multiple inputs, we design branch networks that interact through fusion transformers at multiple scales to utilize the information of different modalities more adequately. A natural advantage of our design is the ability to aggregate global multimodal features through self-attention. Both qualitative and quantitative experiments demonstrate the superiority of our method over the state-of-the-art fusion methods.
- Open Access
- 29 May 2021
- Published by Wiley in Journal of Healthcare Engineering
- Vol. 2021, 1-11
Interpatient ECG Heartbeat Classification with an Adversarial Convolutional Neural Network
Discovering shared, invariant feature representations across subjects in electrocardiogram (ECG) classification tasks is crucial for improving the generalization of models to unknown patients. Although deep neural networks have recently been emerging in extracting generalizable ECG features, they usually rely on labeled samples from a large number of subjects to guarantee generalization. Extracting invariant representations to intersubject variabilities from a small number of subjects is still a challenge today due to individual physical differences. To address this problem, we propose an adversarial deep neural network framework for interpatient heartbeat classification by integrating adversarial learning into a convolutional neural network to learn subject-invariant, class-discriminative features. The proposed method was evaluated on the MIT-BIH arrhythmia database which is a publicly available ECG dataset collected from 47 patients. Compared with the state-of-the-art methods, the proposed method achieves the highest performance for detecting supraventricular ectopic beats (SVEBs), which are very challenging to identify, and also gains comparable performance on the detection of ventricular ectopic beats (VEBs). The sensitivities of SVEBs and VEBs are 78.8% and 92.5%, respectively. The precisions of SVEBs and VEBs are 90.8% and 94.3%, respectively. With high performance in the detection of pathological classes (i.e., SVEBs and VEBs), this work provides a promising method for ECG classification tasks when the number of patients is limited.
- 29 May 2021
- Published by Wiley in Journal of Healthcare Engineering
- Vol. 2021, 1-11
Open AccessInterpatient ECG Heartbeat Classification with an Adversarial Convolutional Neural Network
Discovering shared, invariant feature representations across subjects in electrocardiogram (ECG) classification tasks is crucial for improving the generalization of models to unknown patients. Although deep neural networks have recently been emerging in extracting generalizable ECG features, they usually rely on labeled samples from a large number of subjects to guarantee generalization. Extracting invariant representations to intersubject variabilities from a small number of subjects is still a challenge today due to individual physical differences. To address this problem, we propose an adversarial deep neural network framework for interpatient heartbeat classification by integrating adversarial learning into a convolutional neural network to learn subject-invariant, class-discriminative features. The proposed method was evaluated on the MIT-BIH arrhythmia database which is a publicly available ECG dataset collected from 47 patients. Compared with the state-of-the-art methods, the proposed method achieves the highest performance for detecting supraventricular ectopic beats (SVEBs), which are very challenging to identify, and also gains comparable performance on the detection of ventricular ectopic beats (VEBs). The sensitivities of SVEBs and VEBs are 78.8% and 92.5%, respectively. The precisions of SVEBs and VEBs are 90.8% and 94.3%, respectively. With high performance in the detection of pathological classes (i.e., SVEBs and VEBs), this work provides a promising method for ECG classification tasks when the number of patients is limited.
- Open Access
- 9 March 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Translational Engineering in Health and Medicine
- Vol. 9, 1-11
MLBF-Net: A Multi-Lead-Branch Fusion Network for Multi-Class Arrhythmia Classification Using 12-Lead ECG
Automatic arrhythmia detection using 12-lead electrocardiogram (ECG) signal plays a critical role in early prevention and diagnosis of cardiovascular diseases. In the previous studies on automatic arrhythmia detection, most methods concatenated 12 leads of ECG into a matrix, and then input the matrix to a variety of feature extractors or deep neural networks for extracting useful information. Under such frameworks, these methods had the ability to extract comprehensive features (known as integrity) of 12-lead ECG since the information of each lead interacts with each other during training. However, the diverse lead-specific features (known as diversity) among 12 leads were neglected, causing inadequate information learning for 12-lead ECG. To maximize the information learning of multi-lead ECG, the information fusion of comprehensive features with integrity and lead-specific features with diversity should be taken into account. In this paper, we propose a novel Multi-Lead-Branch Fusion Network (MLBF-Net) architecture for arrhythmia classification by integrating multi-loss optimization to jointly learning diversity and integrity of multi-lead ECG. MLBF-Net is composed of three components: 1) multiple lead-specific branches for learning the diversity of multi-lead ECG; 2) cross-lead features fusion by concatenating the output feature maps of all branches for learning the integrity of multi-lead ECG; 3) multi-loss co-optimization for all the individual branches and the concatenated network. We demonstrate our MLBF-Net on China Physiological Signal Challenge 2018 which is an open 12-lead ECG dataset. The experimental results show that MLBF-Net obtains an average $F_{1}$ score of 0.855, reaching the highest arrhythmia classification performance. The proposed method provides a promising solution for multi-lead ECG analysis from an information fusion perspective.
- 9 March 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Translational Engineering in Health and Medicine
- Vol. 9, 1-11
Open AccessMLBF-Net: A Multi-Lead-Branch Fusion Network for Multi-Class Arrhythmia Classification Using 12-Lead ECG
Automatic arrhythmia detection using 12-lead electrocardiogram (ECG) signal plays a critical role in early prevention and diagnosis of cardiovascular diseases. In the previous studies on automatic arrhythmia detection, most methods concatenated 12 leads of ECG into a matrix, and then input the matrix to a variety of feature extractors or deep neural networks for extracting useful information. Under such frameworks, these methods had the ability to extract comprehensive features (known as integrity) of 12-lead ECG since the information of each lead interacts with each other during training. However, the diverse lead-specific features (known as diversity) among 12 leads were neglected, causing inadequate information learning for 12-lead ECG. To maximize the information learning of multi-lead ECG, the information fusion of comprehensive features with integrity and lead-specific features with diversity should be taken into account. In this paper, we propose a novel Multi-Lead-Branch Fusion Network (MLBF-Net) architecture for arrhythmia classification by integrating multi-loss optimization to jointly learning diversity and integrity of multi-lead ECG. MLBF-Net is composed of three components: 1) multiple lead-specific branches for learning the diversity of multi-lead ECG; 2) cross-lead features fusion by concatenating the output feature maps of all branches for learning the integrity of multi-lead ECG; 3) multi-loss co-optimization for all the individual branches and the concatenated network. We demonstrate our MLBF-Net on China Physiological Signal Challenge 2018 which is an open 12-lead ECG dataset. The experimental results show that MLBF-Net obtains an average $F_{1}$ score of 0.855, reaching the highest arrhythmia classification performance. The proposed method provides a promising solution for multi-lead ECG analysis from an information fusion perspective.