Mining Cross-Modality Implicit Semantic Association for Unsupervised Visible-Infrared Person Re-Identification
- 17 December 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 21 (15566013) , 697-709
Abstract
Unsupervised visible-infrared person reidentification (US-VI-ReID) seeks to learn a cross-modality retrieval model without relying on manual annotations, thereby reducing the high cost associated with labeling. Recent large-scale vision-language pre-training models, such as CLIP, have shown significant potential in enhancing pure-vision-based person re-identification. However, existing CLIP-based US-VI-ReID methods focus on independently learning semantic information within the visible and infrared modalities. These methods overlook the mismatch between the pre-training data of CLIP and the downstream cross-modality data, resulting in substantial cross-modal semantic differences. Such inconsistent semantic information, which exhibits modality discrepancies, cannot ensure the accuracy of cross-modality associations and thus hampers the performance of cross-modality learning. To address these challenges and further explore the generalizable semantic representation across modalities in CLIP, we propose a novel framework named Mining Cross-Modality Implicit Semantic Association (MCSA), which focuses on learning a modality-invariant implicit semantic space to enhance cross-modality associations and feature learning. The proposed method comprises two key modules: Modality-invariant Prompt Learning and GCNs-Driven Collaboration Alignment. Specifically, to enable CLIP to learn modality-invariant semantics, we integrate a random color augmentation branch into the visible stream for joint contrastive learning for mining generalizable semantic representations. This ensures the color generalization of the constructed implicit semantic prompts. Moreover, within the cross-modal invariant implicit semantic space, we utilize Graph Convolutional Networks (GCNs) to uncover more reliable cross-modal associations. By integrating information from images and semantic graphs, we jointly refine cross-modal correspondences, enabling the model to perform precise cross-modal feature learning. Extensive experiments conducted on the SYSU-MM01 and RegDB datasets demonstrate the effectiveness of the proposed MCSA. The source code will be released.Keywords
Funding Information
- National Natural Science Foundation of China (62501428)
- National Natural Science Foundation of China (62302351)
- National Natural Science Foundation of China (62225113)
This publication has 56 references indexed in Scilit:
- Transformer for Object Re-identification: A SurveyInternational Journal of Computer Vision, 2024
- Multi-memory Matching for Unsupervised Visible-Infrared Person Re-identification2024
- Visible–infrared person re-identification via patch-mixed cross-modality learningPattern Recognition, 2024
- Enhanced visible–infrared person re-identification based on cross-attention multiscale residual vision transformerPattern Recognition, 2024
- MIMR: Modality-Invariance Modeling and Refinement for unsupervised visible-infrared person re-identificationKnowledge-Based Systems, 2023
- Cluster Contrast for Unsupervised Person Re-identification2023
- Optimal Transport for Label-Efficient Visible-Infrared Person Re-Identification2022
- Learning with Twin Noisy Labels for Visible-Infrared Person Re-Identification2022
- FMCNet: Feature-Level Modality Compensation for Visible-Infrared Person Re-Identification2022
- Discover Cross-Modality Nuances for Visible-Infrared Person Re-Identification2021