Joint Identity-Aware Mixstyle and Graph-Enhanced Prototype for Clothes-Changing Person Re-Identification
- 1 September 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 26 (15209210) , 3457-3468
Abstract
In recent years, considerable progress has been witnessed in the person re-identification (Re-ID). However, in a more realistic long-term scenario, the appearance shift arising from the clothes-changing inevitably deteriorates the conventional methods that heavily depend on the clothing color. Although the current clothes-changing person Re-ID methods introduce external human knowledge (i.e, contour, mask) and sophisticated feature decoupling strategy to alleviate the clothing shift, they still face the risk of overfitting to clothing due to the limited clothing diversity of training set. To more efficiently and effectively promote the clothes-irrelevant feature learning, we present a novel joint Identity-aware Mixstyle and Graph-enhanced Prototype method for clothes-changing person Re-ID. Specifically, by treating the cloth-changing as fine-grained domain/style shift, the identity-aware mixstyle (IMS) is proposed from the perspective of domain generalization, which mixes the instance-level feature statistics of samples within each identity to synthesize novel and diverse clothing styles, while retaining the correspondence between synthesized samples and latent label space. By incorporating the IMS module, the more diverse styles can be exploited to train a clothing-shift robust model. To further reduce the feature discrepancy caused by clothing variations, the graph-enhanced prototype constraint (GEP) module is proposed to explore the graph similarity structure of style-augmented samples across memory bank to build informative and robust prototypes, which serve as powerful exemplars for better clothing-irrelevant metric learning. The two modules are integrated into a joint learning framework and benefit each other. The extensive experiments conducted on clothes-changing person Re-ID datasets validate the superiority and effectiveness of our method. In addition, our method also shows good universality and corruption robustness on other Re-ID tasks.Keywords
Funding Information
- National Natural Science Foundation of China (62272430)
This publication has 48 references indexed in Scilit:
- Patching Your Clothes: Semantic-Aware Learning for Cloth-Changed Person Re-Identification2022
- Long-Term Cloth-Changing Person Re-identification2021
- Identity-Guided Human Semantic Parsing for Person Re-identification2020
- Faster Person Re-identification2020
- Rethinking the Distribution Gap of Person Re-identification with Camera-Based Batch Normalization2020
- Beyond Part Models: Person Retrieval with Refined Part Pooling (and A Strong Convolutional Baseline)2018
- Domain Generalization with Adversarial Feature Learning2018
- A Discriminative Feature Learning Approach for Deep Face Recognition2016
- Empirical Risk Minimization2016
- Deep Residual Learning for Image Recognition2016