Image-Specific Information Suppression and Implicit Local Alignment for Text-Based Person Search
Top Cited Papers
- 15 September 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (12) , 17973-17986
Abstract
Text-based person search (TBPS) is a challenging task that aims to search pedestrian images with the same identity from an image gallery given a query text. In recent years, TBPS has made remarkable progress, and state-of-the-art (SOTA) methods achieve superior performance by learning local fine-grained correspondence between images and texts. However, most existing methods rely on explicitly generated local parts to model fine-grained correspondence between modalities, which is unreliable due to the lack of contextual information or the potential introduction of noise. Moreover, the existing methods seldom consider the information inequality problem between modalities caused by image-specific information. To address these limitations, we propose an efficient joint multilevel alignment network (MANet) for TBPS, which can learn aligned image/text feature representations between modalities at multiple levels, and realize fast and effective person search. Specifically, we first design an image-specific information suppression (ISS) module, which suppresses image background and environmental factors by relation-guided localization (RGL) and channel attention filtration (CAF), respectively. This module effectively alleviates the information inequality problem and realizes the alignment of information volume between images and texts. Second, we propose an implicit local alignment (ILA) module to adaptively aggregate all pixel/word features of image/text to a set of modality-shared semantic topic centers and implicitly learn the local fine-grained correspondence between modalities without additional supervision and cross-modal interactions. Also, a global alignment (GA) is introduced as a supplement to the local perspective. The cooperation of global and local alignment modules enables better semantic alignment between modalities. Extensive experiments on multiple databases demonstrate the effectiveness and superiority of our MANet.Keywords
All Related Versions
Funding Information
- National Key Research and Development Program of China (2018AAA0102002)
- National Natural Science Foundation of China (62172212, 61932020)
This publication has 73 references indexed in Scilit:
- Participation-Contributed Temporal Dynamic Model for Group Activity Recognition2018
- Improving Deep Visual Representation for Person Re-identification by Global and Local Image-language Association2018
- Stacked Cross Attention for Image-Text Matching2018
- Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net2018
- Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization2017
- Identity-Aware Textual-Visual Matching with Latent Co-attention2017
- Person Search with Natural Language Description2017
- Learning Deep Context-Aware Features over Body and Latent Parts for Person Re-identification2017
- Query-Driven Approach to Face Clustering and TaggingIEEE Transactions on Image Processing, 2016
- Deep Residual Learning for Image Recognition2016