TransPose: Keypoint Localization via Transformer
Authors:
Top Cited Papers
- 1 October 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 11782-11792
Abstract
While CNN-based models have made remarkable progress on human pose estimation, what spatial dependencies they capture to localize keypoints remains unclear. In this work, we propose a model called Trans-Pose, which introduces Transformer for human pose estimation. The attention layers built in Transformer enable our model to capture long-range relationships efficiently and also can reveal what dependencies the predicted key-points rely on. To predict keypoint heatmaps, the last attention layer acts as an aggregator, which collects contributions from image clues and forms maximum positions of keypoints. Such a heatmap-based localization approach via Transformer conforms to the principle of Activation Maximization [19]. And the revealed dependencies are image-specific and fine-grained, which also can provide evidence of how the model handles special cases, e.g., occlusion. The experiments show that TransPose achieves 75.8 AP and 75.0 AP on COCO validation and test-dev sets, while being more lightweight and faster than mainstream CNN architectures. The TransPose model also transfers very well on MPII benchmark, achieving superior performance on the test set when fine-tuned with small training costs. Code and pre-trained models are publicly available 1 .Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China
This publication has 32 references indexed in Scilit:
- Hybrid computing using a neural network with dynamic external memoryNature, 2016
- Learning Deep Features for Discriminative Localization2016
- Convolutional Pose Machines2016
- Flowing ConvNets for Human Pose Estimation in Videos2015
- On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance PropagationPLOS ONE, 2015
- Fully convolutional networks for semantic segmentation2015
- Heterogeneous Multi-task Learning for Human Pose Estimation with Deep Convolutional Neural NetworkInternational Journal of Computer Vision, 2014
- DeepPose: Human Pose Estimation via Deep Neural Networks2014
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Long Short-Term MemoryNeural Computation, 1997