Balanced and Essential Modality-Specific and Modality-Shared Representations for Visible-Infrared Person Re-Identification
- 25 January 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 31 (10709908) , 491-495
Abstract
Retrieving and matching individual images for Visible-Infrared Person Re-identification is a challenging task due to the huge modality gap between daytime color and nighttime infrared images from different modalities. Existing approaches rely on inefficient data augmentation and/or biased modality characteristics, limiting their potential for performance improvement. To solve these problems, we propose a novel balanced approach between Modality-Specific and Modality-Shared method including efficient transform for diversity of instances. First, we propose the Informative Weighted Gray transform (IWG), which aims to maximize the diversity of instances by generating the distinct combination of RGB colors. Second, introducing the Customized Modality-Specific Enhanced Module (CMSpEM) provides enhanced feature maps using attention mechanism between pre-pooling and post-pooling features and reinforces specific features for the modality-shared based network with an extremely small number of parameters. Third, we introduce the Pseudo Label-oriented Modality-Specific (PLMSp) Loss, which provides effective representation learning explicitly to reduce the modality gap using pseudo labels as anchors. We compare our Balanced and Essential Modality-Specific and Modality-Shared Network (BEMSSNet) and various existing methods for the mAP and Rank-1 performances on the SYSU-MM01 and RegDB datasets. Experimental results demonstrate that our proposed model outperforms the existing state-of-the-art methods.Keywords
Funding Information
- National Research Foundation of Korea
- Korea government (RS-2023-00244355)
This publication has 35 references indexed in Scilit:
- Hetero-Center loss for cross-modality person Re-identificationNeurocomputing, 2020
- HSME: Hypersphere Manifold Embedding for Visible Thermal Person Re-IdentificationProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Bi-Directional Center-Constrained Top-Ranking for Visible Thermal Person Re-IdentificationIEEE Transactions on Information Forensics and Security, 2019
- Learning to Reduce Dual-Level Discrepancy for Infrared-Visible Person Re-Identification2019
- Multi-level Factorisation Net for Person Re-identification2018
- Human Semantic Parsing for Person Re-identification2018
- A Discriminatively Learned CNN Embedding for Person ReidentificationACM Transactions on Multimedia Computing, Communications, and Applications, 2017
- RGB-Infrared Cross-Modality Person Re-identification2017
- Person Recognition System Based on a Combination of Body Images from Visible Light and Thermal CamerasSensors, 2017
- Deep Residual Learning for Image Recognition2016