Lei Zhu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Lei Zhu has a total of 102 co-authors. Here are the top co-authors by number of shared publications:
- Yanye Lu
Peking University
22 shared publications - Hangzhou He
Peking University
6 shared publications - Lujia Jin
Peking University
6 shared publications Hangzhou He
4 shared publications- Kaiwen Li
Peking University, Beijing, China
4 shared publications Qian Chen
Peking University
4 shared publications
Latest Publications
- 1 September 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 30 (9) , 7694-7707
Multi-Level Asymmetric Contrastive Learning for Medical Image Segmentation Pre-Training
Medical image segmentation is a fundamental yet challenging task due to the arduous process of acquiring large volumes of high-quality labeled data from experts. Contrastive learning offers a promising but still problematic solution to this dilemma. Firstly existing medical contrastive learning strategies focus on extracting image-level representation, which ignores abundant multi-level representations. Furthermore they underutilize the decoder either by random initialization or separate pre-training from the encoder, thereby neglecting the potential collaboration between the encoder and decoder. To address these issues, we propose a novel multi-level asymmetric contrastive learning framework named MACL for enhancing medical image segmentation. Specifically, we design an asymmetric contrastive learning structure to pre-train encoder and decoder simultaneously to provide better initialization for segmentation models. Moreover, we develop a multi-level contrastive learning strategy that integrates correspondences across feature-level, image-level, and pixel-level representations to ensure the encoder and decoder capture comprehensive details from representations of varying scales and granularities during the pre-training phase. Finally, experiments on 8 medical image datasets indicate our MACL framework outperforms existing 11 contrastive learning strategies. i.e. Our MACL achieves a superior performance with more precise predictions from visualization figures and 1.72%, 7.87%, 2.49% and 1.48% Dice higher than previous best results on ACDC, MMWHS, HVSMR and CHAOS with 10% labeled data, respectively. And our MACL also has a strong generalization ability among 5 variant U-Net backbones.
- 1 September 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 30 (9) , 7694-7707
Multi-Level Asymmetric Contrastive Learning for Medical Image Segmentation Pre-Training
Medical image segmentation is a fundamental yet challenging task due to the arduous process of acquiring large volumes of high-quality labeled data from experts. Contrastive learning offers a promising but still problematic solution to this dilemma. Firstly existing medical contrastive learning strategies focus on extracting image-level representation, which ignores abundant multi-level representations. Furthermore they underutilize the decoder either by random initialization or separate pre-training from the encoder, thereby neglecting the potential collaboration between the encoder and decoder. To address these issues, we propose a novel multi-level asymmetric contrastive learning framework named MACL for enhancing medical image segmentation. Specifically, we design an asymmetric contrastive learning structure to pre-train encoder and decoder simultaneously to provide better initialization for segmentation models. Moreover, we develop a multi-level contrastive learning strategy that integrates correspondences across feature-level, image-level, and pixel-level representations to ensure the encoder and decoder capture comprehensive details from representations of varying scales and granularities during the pre-training phase. Finally, experiments on 8 medical image datasets indicate our MACL framework outperforms existing 11 contrastive learning strategies. i.e. Our MACL achieves a superior performance with more precise predictions from visualization figures and 1.72%, 7.87%, 2.49% and 1.48% Dice higher than previous best results on ACDC, MMWHS, HVSMR and CHAOS with 10% labeled data, respectively. And our MACL also has a strong generalization ability among 5 variant U-Net backbones.
- 9 April 2026
- Published by Optica Publishing Group in Biomedical Optics Express
- Vol. 17 (5) , 2262-2281
WeakMitoSAM: competitive prompt aggregation for point-supervised mitochondria segmentation in electron microscopy images
Mitochondrial morphology is a critical indicator of cellular metabolic status and disease pathogenesis, requiring high-resolution visualization and precise segmentation in electron microscopy (EM) images. While fully supervised deep learning models have achieved significant progress, their reliance on dense pixel-wise annotations presents a major bottleneck due to the labor-intensive labeling process and expert variability near the optical diffraction limit. Existing weakly supervised methods, primarily designed for densely packed instances, often fail to generalize to the sparse distribution of mitochondria in EM data. In this paper, we propose WeakMitoSAM, a novel weakly supervised framework for high-precision mitochondria segmentation using sparse point annotations. Our approach introduces the competitive aggregation of multiple prompts strategy, which employs a Bias-augmented Softmax mechanism to reconcile semantic ambiguities and suppress background noise, effectively converting sparse priors into high-fidelity pseudo-labels. Subsequently, segment anything model is specialized for mitochondrial ultrastructures via low-rank adaptation, ensuring parameter-efficient domain adaptation. Experimental results across four public EM datasets demonstrate that WeakMitoSAM achieves state-of-the-art performance in point-supervised scenarios and even outperforms several fully supervised benchmarks, providing an efficient and robust solution for large-scale mitochondrial morphofunctional analysis.
- 9 April 2026
- Published by Optica Publishing Group in Biomedical Optics Express
- Vol. 17 (5) , 2262-2281
WeakMitoSAM: competitive prompt aggregation for point-supervised mitochondria segmentation in electron microscopy images
Mitochondrial morphology is a critical indicator of cellular metabolic status and disease pathogenesis, requiring high-resolution visualization and precise segmentation in electron microscopy (EM) images. While fully supervised deep learning models have achieved significant progress, their reliance on dense pixel-wise annotations presents a major bottleneck due to the labor-intensive labeling process and expert variability near the optical diffraction limit. Existing weakly supervised methods, primarily designed for densely packed instances, often fail to generalize to the sparse distribution of mitochondria in EM data. In this paper, we propose WeakMitoSAM, a novel weakly supervised framework for high-precision mitochondria segmentation using sparse point annotations. Our approach introduces the competitive aggregation of multiple prompts strategy, which employs a Bias-augmented Softmax mechanism to reconcile semantic ambiguities and suppress background noise, effectively converting sparse priors into high-fidelity pseudo-labels. Subsequently, segment anything model is specialized for mitochondrial ultrastructures via low-rank adaptation, ensuring parameter-efficient domain adaptation. Experimental results across four public EM datasets demonstrate that WeakMitoSAM achieves state-of-the-art performance in point-supervised scenarios and even outperforms several fully supervised benchmarks, providing an efficient and robust solution for large-scale mitochondrial morphofunctional analysis.
- 3 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35 (10577149) , 1636-1651
SuperCL: Superpixel Guided Contrastive Learning for Medical Image Segmentation Pre-Training
Medical image segmentation is a critical yet challenging task, primarily due to the difficulty of obtaining extensive datasets of high-quality, expert-annotated images. Contrastive learning presents a potential but still problematic solution to this issue. Because most existing methods focus on extracting instance-level or pixel-to-pixel representation, which ignores the characteristics between intra-image similar pixel groups. Moreover, when considering contrastive pairs generation, most SOTA methods mainly rely on manually setting thresholds, which requires a large number of gradient experiments and lacks efficiency and generalization. To address these issues, we propose a novel contrastive learning approach named SuperCL for medical image segmentation pre-training. Specifically, our SuperCL exploits the structural prior and pixel correlation of images by introducing two novel contrastive pairs generation strategies: Intra-image Local Contrastive Pairs (ILCP) Generation and Inter-image Global Contrastive Pairs (IGCP) Generation. Considering superpixel cluster aligns well with the concept of contrastive pairs generation, we utilize the superpixel map to generate pseudo masks for both ILCP and IGCP to guide supervised contrastive learning. Moreover, we also propose two modules named Average SuperPixel Feature Map Generation (ASP) and Connected Components Label Generation (CCL) to better exploit the prior structural information for IGCP. Finally, experiments on 8 medical image datasets indicate our SuperCL outperforms existing 12 methods. i.e. Our SuperCL achieves a superior performance with more precise predictions from visualization figures and 3.15%, 5.44%, 7.89% DSC higher than the previous best results on MMWHS, CHAOS, Spleen with 10% annotations. Our code is released at https://github.com/stevezs315/SuperCL.
- 3 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35 (10577149) , 1636-1651
SuperCL: Superpixel Guided Contrastive Learning for Medical Image Segmentation Pre-Training
Medical image segmentation is a critical yet challenging task, primarily due to the difficulty of obtaining extensive datasets of high-quality, expert-annotated images. Contrastive learning presents a potential but still problematic solution to this issue. Because most existing methods focus on extracting instance-level or pixel-to-pixel representation, which ignores the characteristics between intra-image similar pixel groups. Moreover, when considering contrastive pairs generation, most SOTA methods mainly rely on manually setting thresholds, which requires a large number of gradient experiments and lacks efficiency and generalization. To address these issues, we propose a novel contrastive learning approach named SuperCL for medical image segmentation pre-training. Specifically, our SuperCL exploits the structural prior and pixel correlation of images by introducing two novel contrastive pairs generation strategies: Intra-image Local Contrastive Pairs (ILCP) Generation and Inter-image Global Contrastive Pairs (IGCP) Generation. Considering superpixel cluster aligns well with the concept of contrastive pairs generation, we utilize the superpixel map to generate pseudo masks for both ILCP and IGCP to guide supervised contrastive learning. Moreover, we also propose two modules named Average SuperPixel Feature Map Generation (ASP) and Connected Components Label Generation (CCL) to better exploit the prior structural information for IGCP. Finally, experiments on 8 medical image datasets indicate our SuperCL outperforms existing 12 methods. i.e. Our SuperCL achieves a superior performance with more precise predictions from visualization figures and 3.15%, 5.44%, 7.89% DSC higher than the previous best results on MMWHS, CHAOS, Spleen with 10% annotations. Our code is released at https://github.com/stevezs315/SuperCL.
- 16 December 2025
- Published by Elsevier in Expert Systems with Applications
- Vol. 305
Improve retinal artery/vein classification via channel coupling
- 16 December 2025
- Published by Elsevier in Expert Systems with Applications
- Vol. 305
Improve retinal artery/vein classification via channel coupling
- 20 September 2025
- Published by Springer Nature
- p. 638-648
Training-Free Test-Time Improvement for Explainable Medical Image Classification
- 20 September 2025
- Published by Springer Nature
- p. 638-648
Training-Free Test-Time Improvement for Explainable Medical Image Classification
- 7 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 44 (12) , 5038-5050
Points-Supervised Fundus Vessel Segmentation via Shape Priors and Contrastive Learning
The performance of fully supervised methods for fundus vessel segmentation highly relies on a large number of full labels which are laborious and time-consuming to obtain. Although weak annotations relax the requirement for pixel-wise labeling, they pose challenges in learning comprehensive information about the target. Some methods use pseudo labels generated from network predictions for extra supervision, but false positive predictions in these labels may harm training. In this paper, to tackle this problem and to balance the annotation cost and supervision information, we introduce point annotations to fundus vessel segmentation and propose a novel method, called Points-based Vessel segmentation Network (PVN), to enhance the segmentation accuracy. In PVN, to avoid noise in pseudo labels, by combining proposed Point Activation Maps, shape priors of vessels are learned and used as soft supervision. Additionally, to further leverage the annotated vessel and background points, we design a novel contrastive learning method in a pixels-and-regions-mixed manner, which helps learn discriminative features by distinguishing between pixel and region samples of vessels and background. The performance of PVN is evaluated on laser speckle contrast imaging fundus images, 548 nm fundus images, and three public datasets, where PVN outperforms other point-supervised methods. Even with only 1% annotated pixels, PVN still achieves excellent performance. Our method is also flexible and easy to be combined with other frameworks. To the best of our knowledge, we are the first to propose and demonstrate the effectiveness of point annotations for fundus vessel segmentation. Our code is available at: https://github.com/kaiwenli325/PVN.
- 7 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 44 (12) , 5038-5050
Points-Supervised Fundus Vessel Segmentation via Shape Priors and Contrastive Learning
The performance of fully supervised methods for fundus vessel segmentation highly relies on a large number of full labels which are laborious and time-consuming to obtain. Although weak annotations relax the requirement for pixel-wise labeling, they pose challenges in learning comprehensive information about the target. Some methods use pseudo labels generated from network predictions for extra supervision, but false positive predictions in these labels may harm training. In this paper, to tackle this problem and to balance the annotation cost and supervision information, we introduce point annotations to fundus vessel segmentation and propose a novel method, called Points-based Vessel segmentation Network (PVN), to enhance the segmentation accuracy. In PVN, to avoid noise in pseudo labels, by combining proposed Point Activation Maps, shape priors of vessels are learned and used as soft supervision. Additionally, to further leverage the annotated vessel and background points, we design a novel contrastive learning method in a pixels-and-regions-mixed manner, which helps learn discriminative features by distinguishing between pixel and region samples of vessels and background. The performance of PVN is evaluated on laser speckle contrast imaging fundus images, 548 nm fundus images, and three public datasets, where PVN outperforms other point-supervised methods. Even with only 1% annotated pixels, PVN still achieves excellent performance. Our method is also flexible and easy to be combined with other frameworks. To the best of our knowledge, we are the first to propose and demonstrate the effectiveness of point annotations for fundus vessel segmentation. Our code is available at: https://github.com/kaiwenli325/PVN.
- Open Access
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (3) , 3401-3409
V2C-CBM: Building Concept Bottlenecks with Vision-to-Concept Tokenizer
Concept Bottleneck Models (CBMs) offer inherent interpretability by initially translating images into human-comprehensible concepts, followed by a linear combination of these concepts for classification. However, the annotation of concepts for visual recognition tasks requires extensive expert knowledge and labor, constraining the broad adoption of CBMs. Recent approaches have leveraged the knowledge of large language models to construct concept bottlenecks, with multimodal models like CLIP subsequently mapping image features into the concept feature space for classification. Despite this, the concepts produced by language models can be verbose and may introduce non-visual attributes, which hurts accuracy and interpretability. In this study, we investigate to avoid these issues by constructing CBMs directly from multimodal models. To this end, we adopt common words as base concept vocabulary and leverage auxiliary unlabeled images to construct a Vision-to-Concept (V2C) tokenizer that can explicitly quantize images into their most relevant visual concepts, thus creating a vision-oriented concept bottleneck tightly coupled with the multimodal model. This leads to our V2C-CBM which is training efficient and interpretable with high accuracy. Our V2C-CBM has matched or outperformed LLM-supervised CBMs on various visual classification benchmarks, validating the efficacy of our approach.
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (3) , 3401-3409
Open AccessV2C-CBM: Building Concept Bottlenecks with Vision-to-Concept Tokenizer
Concept Bottleneck Models (CBMs) offer inherent interpretability by initially translating images into human-comprehensible concepts, followed by a linear combination of these concepts for classification. However, the annotation of concepts for visual recognition tasks requires extensive expert knowledge and labor, constraining the broad adoption of CBMs. Recent approaches have leveraged the knowledge of large language models to construct concept bottlenecks, with multimodal models like CLIP subsequently mapping image features into the concept feature space for classification. Despite this, the concepts produced by language models can be verbose and may introduce non-visual attributes, which hurts accuracy and interpretability. In this study, we investigate to avoid these issues by constructing CBMs directly from multimodal models. To this end, we adopt common words as base concept vocabulary and leverage auxiliary unlabeled images to construct a Vision-to-Concept (V2C) tokenizer that can explicitly quantize images into their most relevant visual concepts, thus creating a vision-oriented concept bottleneck tightly coupled with the multimodal model. This leads to our V2C-CBM which is training efficient and interpretable with high accuracy. Our V2C-CBM has matched or outperformed LLM-supervised CBMs on various visual classification benchmarks, validating the efficacy of our approach.
- Open Access
V2C-CBM: Building Concept Bottlenecks with Vision-to-Concept Tokenizer
Concept Bottleneck Models (CBMs) offer inherent interpretability by initially translating images into human-comprehensible concepts, followed by a linear combination of these concepts for classification. However, the annotation of concepts for visual recognition tasks requires extensive expert knowledge and labor, constraining the broad adoption of CBMs. Recent approaches have leveraged the knowledge of large language models to construct concept bottlenecks, with multimodal models like CLIP subsequently mapping image features into the concept feature space for classification. Despite this, the concepts produced by language models can be verbose and may introduce non-visual attributes, which hurts accuracy and interpretability. In this study, we investigate to avoid these issues by constructing CBMs directly from multimodal models. To this end, we adopt common words as base concept vocabulary and leverage auxiliary unlabeled images to construct a Vision-to-Concept (V2C) tokenizer that can explicitly quantize images into their most relevant visual concepts, thus creating a vision-oriented concept bottleneck tightly coupled with the multimodal model. This leads to our V2C-CBM which is training efficient and interpretable with high accuracy. Our V2C-CBM has matched or outperformed LLM-supervised CBMs on various visual classification benchmarks, validating the efficacy of our approach.
Open AccessV2C-CBM: Building Concept Bottlenecks with Vision-to-Concept Tokenizer
Concept Bottleneck Models (CBMs) offer inherent interpretability by initially translating images into human-comprehensible concepts, followed by a linear combination of these concepts for classification. However, the annotation of concepts for visual recognition tasks requires extensive expert knowledge and labor, constraining the broad adoption of CBMs. Recent approaches have leveraged the knowledge of large language models to construct concept bottlenecks, with multimodal models like CLIP subsequently mapping image features into the concept feature space for classification. Despite this, the concepts produced by language models can be verbose and may introduce non-visual attributes, which hurts accuracy and interpretability. In this study, we investigate to avoid these issues by constructing CBMs directly from multimodal models. To this end, we adopt common words as base concept vocabulary and leverage auxiliary unlabeled images to construct a Vision-to-Concept (V2C) tokenizer that can explicitly quantize images into their most relevant visual concepts, thus creating a vision-oriented concept bottleneck tightly coupled with the multimodal model. This leads to our V2C-CBM which is training efficient and interpretable with high accuracy. Our V2C-CBM has matched or outperformed LLM-supervised CBMs on various visual classification benchmarks, validating the efficacy of our approach.
- 14 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (6) , 11493-11506
Branches Mutual Promotion for End-to-End Weakly Supervised Semantic Segmentation
End-to-end weakly supervised semantic segmentation (E2E-WSSS) aims at optimizing a segmentation model in a single-stage training process based on only image annotations. Existing methods adopt an online-trained classification branch to provide pseudo annotations for supervising the segmentation branch. However, this strategy makes the classification branch dominate the whole concurrent training process, hindering these two branches from assisting each other. In our work, we treat these two branches equally by viewing them as diverse ways to generate the segmentation map, and add interactions on both their supervision and operation to achieve mutual promotion. For this purpose, a bidirectional supervision mechanism is elaborated to force the consistency between the outputs of these two branches. Thus, the segmentation branch can also give feedback to the classification branch to enhance the quality of localization seeds. Moreover, our method also designs interaction operations between these two branches to exchange their knowledge to assist each other. Experiments indicate our work outperforms existing end-to-end weakly supervised segmentation methods. Codes are available at https://github.com/zh460045050/BMP-WSSS.
- 14 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (6) , 11493-11506
Branches Mutual Promotion for End-to-End Weakly Supervised Semantic Segmentation
End-to-end weakly supervised semantic segmentation (E2E-WSSS) aims at optimizing a segmentation model in a single-stage training process based on only image annotations. Existing methods adopt an online-trained classification branch to provide pseudo annotations for supervising the segmentation branch. However, this strategy makes the classification branch dominate the whole concurrent training process, hindering these two branches from assisting each other. In our work, we treat these two branches equally by viewing them as diverse ways to generate the segmentation map, and add interactions on both their supervision and operation to achieve mutual promotion. For this purpose, a bidirectional supervision mechanism is elaborated to force the consistency between the outputs of these two branches. Thus, the segmentation branch can also give feedback to the classification branch to enhance the quality of localization seeds. Moreover, our method also designs interaction operations between these two branches to exchange their knowledge to assist each other. Experiments indicate our work outperforms existing end-to-end weakly supervised segmentation methods. Codes are available at https://github.com/zh460045050/BMP-WSSS.
- 7 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 46 (12) , 8680-8695
Boosting Weakly Supervised Object Localization and Segmentation With Domain Adaption
Weakly supervised object localization (WSOL), adopting only image-level annotations to learn the pixel-level localization model, can release human resources in the annotation process. Most one-stage WSOL methods learn the localization model with multi-instance learning, making them only activate discriminative object parts rather than the whole object. In our work, we attribute this problem to the domain shift between the training and test process of WSOL and provide a novel perspective that views WSOL as a domain adaption (DA) task. Under this perspective, a DA-WSOL pipeline is elaborated to better assist WSOL with DA approaches by considering the specificities for the adaption of WSOL. Our DA-WSOL pipeline can discern the source-related and the Universum samples from other target samples based on a proposed target sampling strategy and then utilize them to solve the sample unbalancing and label unmatching between the source and target domain of WSOL. Experiments show that our pipeline outperforms SOTA methods on three WSOL benchmarks and can improve the performance of downstream weakly supervised semantic segmentation tasks.
- 7 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 46 (12) , 8680-8695
Boosting Weakly Supervised Object Localization and Segmentation With Domain Adaption
Weakly supervised object localization (WSOL), adopting only image-level annotations to learn the pixel-level localization model, can release human resources in the annotation process. Most one-stage WSOL methods learn the localization model with multi-instance learning, making them only activate discriminative object parts rather than the whole object. In our work, we attribute this problem to the domain shift between the training and test process of WSOL and provide a novel perspective that views WSOL as a domain adaption (DA) task. Under this perspective, a DA-WSOL pipeline is elaborated to better assist WSOL with DA approaches by considering the specificities for the adaption of WSOL. Our DA-WSOL pipeline can discern the source-related and the Universum samples from other target samples based on a proposed target sampling strategy and then utilize them to solve the sample unbalancing and label unmatching between the source and target domain of WSOL. Experiments show that our pipeline outperforms SOTA methods on three WSOL benchmarks and can improve the performance of downstream weakly supervised semantic segmentation tasks.