Deep Multi-Patch Matching Network for Visible Thermal Person Re-Identification

Authors:
Publisher website
Google Scholar
Abstract
\textit{Visible Thermal Person Re-Identification} (VTReID) is a cross-modality retrieval problem in computer vision. Accurate VTReID is very challenging due to large modality discrepancies. In this work, we design a novel \textit{Multi-Patch Matching Network} (MPMN) framework to simultaneously mitigate the heterogeneity of coarse-grained and fine-grained visual semantics. In view of cross-modality matching, we verify that aligning modality distributions of the original features is likely to suffer from the selective alignment behavior, \ie, only focuses on easiest dimensions or subspaces. Inspired by adversarial learning, we propose a new \textit{Multi-Patch Modality Alignment} (MPMA) loss to jointly balance and reduce the modality discrepancies of multi-patch features by mining hard subspaces and abandoning easy subspaces. Since multi-patch features are potentially complementary to each other, the semantic correlations between different patches should be exploited during training. Motivated by knowledge distillation, we put forward a new \textit{Cross-Patch Correlation Distillation} (CPCD) loss to transfer the semantic knowledges across different patches. To balance multi-patch tasks, an effective \textit{Patch-Aware Priority Attention} (PAPA) method is further introduced to dynamically prioritize hard patch tasks during training. This paper experimentally demonstrates the effectiveness of the proposed methods, achieving superior performance over the state-of-the-art methods on RegDB and SYSU-MM01 datasets.
Funding Information
  • National Natural Science Foundation of China (61532018)
  • National Natural Science Foundation of China (U1931202)

This publication has 37 references indexed in Scilit: