Zhe Cao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Zhe Cao has a total of 45 co-authors. Here are the top co-authors by number of shared publications:
- Ruiheng Zhang
Beijing Institute of Technology
12 shared publications - Lixin Xu
State Key Laboratory of Electromechanical Dynamic Control, School of Mechatronical Engineering, Beijing Institute of Technology, Beijing, China
8 shared publications - Biwen Yang
Beijing Institute of Technology
2 shared publications - Fuchun Sun
Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
2 shared publications Haoyang Sun
State Key Lab oratory of Electromechanical Dynamic Control, the School of Mechatronical Engineering, Beijing Institute of Technology, Beijing, China
2 shared publications- Jin Zhang
Beijing Institute of Technology
2 shared publications
Latest Publications
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 25 (11) , 21872-21883
DIHARNet: A Temporal-Doppler-Spatial Domain Fusion Method for Direction Insensitive Human Activity Recognition Based on mmWave Radar
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 25 (11) , 21872-21883
DIHARNet: A Temporal-Doppler-Spatial Domain Fusion Method for Direction Insensitive Human Activity Recognition Based on mmWave Radar
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Consumer Electronics
- Vol. 72 (4) , 11556-11567
TrustCardio: A Multimodal Large Language Model With Chain-of-Thought Explainability for Personalized Vascular Age Prediction via Edge-Enabled Wearable Devices
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Consumer Electronics
- Vol. 72 (4) , 11556-11567
TrustCardio: A Multimodal Large Language Model With Chain-of-Thought Explainability for Personalized Vascular Age Prediction via Edge-Enabled Wearable Devices
- 21 October 2026
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 134 (11) , 486
Learning Complex Scene Segmentation from Noisy Label
- 21 October 2026
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 134 (11) , 486
Learning Complex Scene Segmentation from Noisy Label
- 11 September 2026
- Published by Springer Nature
- p. 115-132
Thermo-JEPA: Learning a Geometry-Grounded Thermal World Model via Cross-Modal Privileged Masking
- 11 September 2026
- Published by Springer Nature
- p. 115-132
Thermo-JEPA: Learning a Geometry-Grounded Thermal World Model via Cross-Modal Privileged Masking
- 28 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. PP (15361233) , 1-14
Co-D 2 : Collaborative Detection and De-ghosting with Radar-guided Infrared Perception
Infrared (IR) imaging is vital for perception under low-light and adverse weather conditions, yet its reliability is often undermined by ghost artifacts caused by specular reflections. These artifacts lead to false detections and hinder accurate scene understanding, particularly in safety-critical applications such as autonomous driving. Millimeter-wave (mmWave) radar offers motion-aware and geometry-consistent priors that hold significant potential for mitigating such artifacts. However, existing fusion methods struggle to reconcile the conflicting goals of object detection and ghost suppression. To address this challenge, we propose Co-D2, a Collaborative Detection and De-ghosting framework that integrates radar-derived structural cues into infrared perception. The framework employs a Radar informed Cross-task Feature Separator (RCFS) to disentangle features for detection and restoration, coupled with a dual-branch architecture and an uncertainty-aware loss to balance the two tasks. We further establish the first benchmark for infrared ghost removal. Extensive experiments on real-world datasets demonstrate that Co-D2 achieves superior performance in both detection and artifact suppression, highlighting its effectiveness for robust multimodal perception in complex environments.
- 28 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. PP (15361233) , 1-14
Co-D 2 : Collaborative Detection and De-ghosting with Radar-guided Infrared Perception
Infrared (IR) imaging is vital for perception under low-light and adverse weather conditions, yet its reliability is often undermined by ghost artifacts caused by specular reflections. These artifacts lead to false detections and hinder accurate scene understanding, particularly in safety-critical applications such as autonomous driving. Millimeter-wave (mmWave) radar offers motion-aware and geometry-consistent priors that hold significant potential for mitigating such artifacts. However, existing fusion methods struggle to reconcile the conflicting goals of object detection and ghost suppression. To address this challenge, we propose Co-D2, a Collaborative Detection and De-ghosting framework that integrates radar-derived structural cues into infrared perception. The framework employs a Radar informed Cross-task Feature Separator (RCFS) to disentangle features for detection and restoration, coupled with a dual-branch architecture and an uncertainty-aware loss to balance the two tasks. We further establish the first benchmark for infrared ghost removal. Extensive experiments on real-world datasets demonstrate that Co-D2 achieves superior performance in both detection and artifact suppression, highlighting its effectiveness for robust multimodal perception in complex environments.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28, 11782-11793
When Infrared Meets Text: Frequency-Enhanced Alignment With Uncertainty Quantification for Infrared Image-Text Retrieval
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28, 11782-11793
When Infrared Meets Text: Frequency-Enhanced Alignment With Uncertainty Quantification for Infrared Image-Text Retrieval
- 11 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Consumer Electronics
- Vol. 72 (1) , 692-704
ADP: Graph Adaptive Pooling Based on Edge Understanding With Graph Pooling Information Bottleneck
Graph pooling plays a crucial role in achieving effective local information aggregation. When dealing with graph, data in non-Euclidean space, a major challenge lies in the uncertainty of the number of nodes. Graph pooling with self-adaptivity can capture the optimal information region to aggregate shared information within this region. Existing methods rely on hard thresholds or parameter settings, making it hard to achieve real adaptiveness and often compromising structural integrity. Inspired by the special relationship between edges and nodes, we propose a real ADaptive Pooling (ADP) method, requiring no hyper-parameter tuning. Specifically, we obtain the graph’s main structure by filtering the most significant edges of each node. Subsequently, we utilize the naturally formed connected regions of n-edge graphs to define information regions. Finally, we integrate the features of nodes within these regions and establish new connections between the nodes. We hold these regions’ structural integrity and stability with triplet loss and information bottleneck compression loss. Our method achieves adaptive pooling while preserving graph structural features, and it has demonstrated state-of-the-art performance on 12 graph datasets in 4 types.
- 11 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Consumer Electronics
- Vol. 72 (1) , 692-704
ADP: Graph Adaptive Pooling Based on Edge Understanding With Graph Pooling Information Bottleneck
Graph pooling plays a crucial role in achieving effective local information aggregation. When dealing with graph, data in non-Euclidean space, a major challenge lies in the uncertainty of the number of nodes. Graph pooling with self-adaptivity can capture the optimal information region to aggregate shared information within this region. Existing methods rely on hard thresholds or parameter settings, making it hard to achieve real adaptiveness and often compromising structural integrity. Inspired by the special relationship between edges and nodes, we propose a real ADaptive Pooling (ADP) method, requiring no hyper-parameter tuning. Specifically, we obtain the graph’s main structure by filtering the most significant edges of each node. Subsequently, we utilize the naturally formed connected regions of n-edge graphs to define information regions. Finally, we integrate the features of nodes within these regions and establish new connections between the nodes. We hold these regions’ structural integrity and stability with triplet loss and information bottleneck compression loss. Our method achieves adaptive pooling while preserving graph structural features, and it has demonstrated state-of-the-art performance on 12 graph datasets in 4 types.
- Open Access
IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark
Real-world infrared imagery presents unique challenges for vision-language models due to the scarcity of aligned text data and domain-specific characteristics. Although existing methods have advanced the field, their reliance on synthetic infrared images generated through style transfer from visible images, which limits their ability to capture the unique characteristics of the infrared modality. To address this, we propose IRGPT, the first multi-modal large language model for real-world infrared images, built upon a large-scale InfraRed-Text Dataset (IR-TD) comprising over 260K authentic image-text pairs. The proposed IR-TD dataset contains real infrared images paired with meticulously handcrafted texts, where the initial drafts originated from two complementary processes: (1) LLM-generated descriptions of visible images, and (2) rule-based descriptions of annotations. Furthermore, we introduce a bi-cross-modal curriculum transfer learning strategy that systematically transfers knowledge from visible to infrared domains by considering the difficulty scores of both infrared-visible and infrared-text. Evaluated on a benchmark of 9 tasks (e.g., recognition, grounding), IRGPT achieves state-of-the-art performance even compared with larger-scale models.
Open AccessIRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark
Real-world infrared imagery presents unique challenges for vision-language models due to the scarcity of aligned text data and domain-specific characteristics. Although existing methods have advanced the field, their reliance on synthetic infrared images generated through style transfer from visible images, which limits their ability to capture the unique characteristics of the infrared modality. To address this, we propose IRGPT, the first multi-modal large language model for real-world infrared images, built upon a large-scale InfraRed-Text Dataset (IR-TD) comprising over 260K authentic image-text pairs. The proposed IR-TD dataset contains real infrared images paired with meticulously handcrafted texts, where the initial drafts originated from two complementary processes: (1) LLM-generated descriptions of visible images, and (2) rule-based descriptions of annotations. Furthermore, we introduce a bi-cross-modal curriculum transfer learning strategy that systematically transfers knowledge from visible to infrared domains by considering the difficulty scores of both infrared-visible and infrared-text. Evaluated on a benchmark of 9 tasks (e.g., recognition, grounding), IRGPT achieves state-of-the-art performance even compared with larger-scale models.
- Open Access
- 30 April 2025
- Published by Springer Nature in Visual Intelligence
- Vol. 3 (1) , 1-12
DBDB: de-bimodal defocus blur in joint infrared-visible imaging
Despite the recent accomplishments in joint infrared-visible imaging, the bimodal defocus blur (BDB) phenomenon received scant attention. Our analysis reveals that BDB is predominantly attributable to disparities in optical parameters between cameras, resulting in two primary challenges: incomplete single-modal information and difficulty in cross-modal information interaction. With regard to the former, the infrared modality is the primary victim, as the deblurring networks’ bias toward high-frequency results in erroneous low-frequency reconstruction (e.g., over-sharpening). In the latter case, the relative nature of the blur effect can lead to ambiguity in determining which modality’s information should be prioritized for guidance, and conflicts may arise between the clear components of the blurred image and the blurry components of the clear image. To address these issues, we propose the first de-bimodal defocus blur (DBDB) method, which consists of a low-frequency semantic hold (LSH) module with a pre-trained infrared model and a cross-modal complementary feature induction (CCFI) module driven by a max-min blur entropy loss. LSH is designed to ensure that the low-frequency information captured by the infrared modality does not contain any misleading data, while CCFI facilitates the acquisition of accurate information by means of adaptive adjustment and the loss function. The experimental results of deblurring and downstream tasks on two synthetic datasets demonstrate the superiority of our method.
- 30 April 2025
- Published by Springer Nature in Visual Intelligence
- Vol. 3 (1) , 1-12
Open AccessDBDB: de-bimodal defocus blur in joint infrared-visible imaging
Despite the recent accomplishments in joint infrared-visible imaging, the bimodal defocus blur (BDB) phenomenon received scant attention. Our analysis reveals that BDB is predominantly attributable to disparities in optical parameters between cameras, resulting in two primary challenges: incomplete single-modal information and difficulty in cross-modal information interaction. With regard to the former, the infrared modality is the primary victim, as the deblurring networks’ bias toward high-frequency results in erroneous low-frequency reconstruction (e.g., over-sharpening). In the latter case, the relative nature of the blur effect can lead to ambiguity in determining which modality’s information should be prioritized for guidance, and conflicts may arise between the clear components of the blurred image and the blurry components of the clear image. To address these issues, we propose the first de-bimodal defocus blur (DBDB) method, which consists of a low-frequency semantic hold (LSH) module with a pre-trained infrared model and a cross-modal complementary feature induction (CCFI) module driven by a max-min blur entropy loss. LSH is designed to ensure that the low-frequency information captured by the infrared modality does not contain any misleading data, while CCFI facilitates the acquisition of accurate information by means of adaptive adjustment and the loss function. The experimental results of deblurring and downstream tasks on two synthetic datasets demonstrate the superiority of our method.
- Top Cited Papers
- 6 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 35 (5) , 4857-4869
Visible-Infrared Person Re-Identification With Real-World Label Noise
In recent years, growing needs for advanced security and traffic management have significantly heightened the prominence of the visible-infrared person re-identification community (VI-ReID), garnering considerable attention. A critical challenge in VI-ReID is the performance degradation attributable to label noise, an issue that becomes even more pronounced in cross-modal scenarios due to an increased likelihood of data confusion. While previous methods have achieved notable successes, they often overlook the complexities of instance-dependent and real-world noise, creating a disconnect from the practical applications of person re-identification. To bridge this gap, our research analyzes the primary sources of label noise in real-world settings, which include a) instantiated identities, b) blurry infrared images, and c) annotators’ errors. In response to these challenges, we develop a Robust Hybrid Loss function (RHL) that enables targeted recognition and retrieval optimization through a more fine-grained division of the noisy dataset. The proposed method categorises data into three sets: clean, obviously noisy, and indistinguishably noisy, with bespoke loss calculations for each category. The identification loss is structured to address the varied nature of these sets specifically. For the retrieval sub-task, we utilize an enhanced triplet loss, adept at handling noisy correspondences. Furthermore, to empirically validate our method, we have re-annotated a real-world dataset, SYSU-Real. Our experiments on SYSU-MM01 and RegDB, conducted under various noise ratios of random and instance-dependent label noise, demonstrate the generalized robustness and effectiveness of our proposed approach.
- 6 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 35 (5) , 4857-4869
Top Cited PapersVisible-Infrared Person Re-Identification With Real-World Label Noise
In recent years, growing needs for advanced security and traffic management have significantly heightened the prominence of the visible-infrared person re-identification community (VI-ReID), garnering considerable attention. A critical challenge in VI-ReID is the performance degradation attributable to label noise, an issue that becomes even more pronounced in cross-modal scenarios due to an increased likelihood of data confusion. While previous methods have achieved notable successes, they often overlook the complexities of instance-dependent and real-world noise, creating a disconnect from the practical applications of person re-identification. To bridge this gap, our research analyzes the primary sources of label noise in real-world settings, which include a) instantiated identities, b) blurry infrared images, and c) annotators’ errors. In response to these challenges, we develop a Robust Hybrid Loss function (RHL) that enables targeted recognition and retrieval optimization through a more fine-grained division of the noisy dataset. The proposed method categorises data into three sets: clean, obviously noisy, and indistinguishably noisy, with bespoke loss calculations for each category. The identification loss is structured to address the varied nature of these sets specifically. For the retrieval sub-task, we utilize an enhanced triplet loss, adept at handling noisy correspondences. Furthermore, to empirically validate our method, we have re-annotated a real-world dataset, SYSU-Real. Our experiments on SYSU-MM01 and RegDB, conducted under various noise ratios of random and instance-dependent label noise, demonstrate the generalized robustness and effectiveness of our proposed approach.