Visible-Infrared Person Re-Identification via Homogeneous Augmented Tri-Modal Learning
- 11 June 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 16 (15566013) , 728-739
Abstract
Matching person images between the daytime visible modality and night-time infrared modality (VI-ReID) is a challenging cross-modality pedestrian retrieval problem. Existing methods usually learn the multi-modality features in raw image, ignoring the image-level discrepancy. Some methods apply GAN technique to generate the cross-modality images, but it destroys the local structure and introduces unavoidable noise. In this paper, we propose a Homogeneous Augmented Tri-Modal (HAT) learning method for VI-ReID, where an auxiliary grayscale modality is generated from their homogeneous visible images, without additional training process. It preserves the structure information of visible images and approximates the image style of infrared modality. Learning with the grayscale visible images enforces the network to mine structure relations across multiple modalities, making it robust to color variations. Specifically, we solve the tri-modal feature learning from both multi-modal classification and multi-view retrieval perspectives. For multi-modal classification, we learn a multi-modality sharing identity classifier with a parameter-sharing network, trained with a homogeneous and heterogeneous identification loss. For multi-view retrieval, we develop a weighted tri-directional ranking loss to optimize the relative distance across multiple modalities. Incorporated with two invariant regularizers, HAT simultaneously minimizes multiple modality variations. In-depth analysis demonstrates the homogeneous grayscale augmentation significantly outperforms the current state-of-the-art by a large margin.Keywords
This publication has 62 references indexed in Scilit:
- MARS: A Video Benchmark for Large-Scale Person Re-Identification2016
- Deep Perceptual Mapping for Cross-Modal Face RecognitionInternational Journal of Computer Vision, 2016
- Hierarchical Gaussian Descriptor for Person Re-identification2016
- Cross-Modal Retrieval With CNN Visual Features: A New BaselineIEEE Transactions on Cybernetics, 2016
- Efficient PSD Constrained Asymmetric Metric Learning for Person Re-Identification2015
- Scalable Person Re-identification: A Benchmark2015
- A Spatio-Temporal Appearance Representation for Video-Based Pedestrian Re-Identification2015
- Multiple Representations-Based Face Sketch–Photo SynthesisIEEE Transactions on Neural Networks and Learning Systems, 2015
- Person re-identification by Local Maximal Occurrence representation and metric learning2015
- Deep Metric Learning for Person Re-identification2014