Attention Map Guided Transformer Pruning for Occluded Person Re-Identification on Edge Device
- 6 April 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 25 (15209210) , 1592-1599
Abstract
Due to its significant capability of modeling long-range dependencies, vision transformer (ViT) has achieved promising success in both holistic and occluded person re-identification (Re-ID) tasks. However, the inherent problems of transformers such as the huge computational cost and memory footprint are still two unsolved issues that will block the deployment of ViT based person Re-ID models on resource-limited edge devices. Our goal is to reduce both the inference complexity and model size without sacrificing the comparable accuracy on person Re-ID, especially for tasks with occlusion. To this end, we propose a novel attention map guided (AMG) transformer pruning method, which removes both redundant tokens and heads with the guidance of the attention map in a hardware-friendly way. We first calculate the entropy in the key dimension and sum it up for the whole map, and the corresponding head parameters of maps with high entropy will be removed for model size reduction. Then we combine the similarity and first-order gradients of key tokens along the query dimension for token importance estimation and remove redundant key and value tokens to further reduce the inference complexity. Comprehensive experiments on Occluded DukeMTMC and Market-1501 demonstrate the effectiveness of our proposals. For example, our proposed pruning strategy on ViT-Base enjoys 29.4% FLOPs savings with 0.2% drop on Rank-1 and 0.4% improvement on mAP, respectively. Code and models have been made available at https://github.com/NUST-Machine-Intelligence-Laboratory/AMG .Keywords
Funding Information
- National Natural Science Foundation of China (62102182, 62202227)
- Natural Science Foundation of Jiangsu Province (BK20210327, BK20220938)
- China Postdoctoral Science Foundation (2022M721628)
- Jiangsu Funding Program for Excellent Postdoctoral Talent (2022ZB271)
This publication has 36 references indexed in Scilit:
- STNReID: Deep Convolutional Networks With Pairwise Spatial Transformer Networks for Partial Person Re-IdentificationIEEE Transactions on Multimedia, 2020
- Importance Estimation for Neural Network Pruning2019
- Beyond Part Models: Person Retrieval with Refined Part Pooling (and A Strong Convolutional Baseline)2018
- GLAD: Global–Local-Alignment Descriptor for Scalable Person Re-IdentificationIEEE Transactions on Multimedia, 2018
- Occluded Person Re-Identification2018
- Exploiting Web Images for Dataset Construction: A Domain Robust ApproachIEEE Transactions on Multimedia, 2017
- An Optimal Resource Allocation for Superposition Coding-Based Hybrid Digital–Analog SystemIEEE Internet of Things Journal, 2017
- Joint Compression of Near-Duplicate VideosIEEE Transactions on Multimedia, 2016
- Scalable Person Re-identification: A Benchmark2015
- ImageNet: A large-scale hierarchical image database2009