MatPose: A 2D Human Pose Estimation Model with Hybrid Mamba-Transformer
Top Cited Papers
Open Access
- 13 January 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 22 (1) , 1-21
Abstract
Recently, Mamba has gained widespread attention due to its ability to model long-range dependencies with linear computational complexity. To explore the application of Mamba in 2D human pose estimation, we propose MatPose, a Mamba-Transformer hybrid model specifically designed for efficient 2D human pose estimation. The model aims to combine Mamba’s efficient modeling of long-range dependencies with the powerful global context modeling capabilities of the Transformer to effectively extract human pose keypoints. Initially, to address the lack of local features when Mamba is applied to computer vision tasks, we design a Cross-Stage Multi-Scale Convolution (CSMSC) module by integrating multi-scale convolution, cross-stage feature fusion, and spatial attention mechanisms to effectively extract local features. Then, to mitigate the long-range forgetting issue inherent in Mamba, we shorten the sequence length using the Conv-Reduce operation. In addition, we design a Channel Selection Attention (CSA) mechanism to compensate for the feature loss caused by the Conv-Reduce operation. Finally, to explore a suitable integration method for the Mamba-Transformer hybrid model in 2D human pose estimation, we conduct a comprehensive ablation study on the feasibility of integrating Mamba and Transformer models. Experimental results show that the proposed method, compared to the baseline model, improves performance while reducing computational overhead. On the COCO val2017 dataset, MatPose achieves an AP of 74.6 with only 5.18 GFLOPs, outperforming most existing human pose estimation models.Keywords
Funding Information
- National Natural Science Foundation of China (62277014, 62262056)
This publication has 23 references indexed in Scilit:
- SMGNFORMER: Fusion Mamba‐graph transformer network for human pose estimationIET Computer Vision, 2024
- YOLOv8: A Novel Object Detection Algorithm with Enhanced Performance and Robustness2024
- Large Separable Kernel Attention: Rethinking the Large Kernel Attention design in CNNExpert Systems with Applications, 2023
- Deep High-Resolution Representation Learning for Human Pose Estimation2019
- CrowdPose: Efficient Crowded Scenes Pose Estimation and a New Benchmark2019
- CBAM: Convolutional Block Attention Module2018
- Simple Baselines for Human Pose Estimation and Tracking2018
- Squeeze-and-Excitation Networks2018
- Cascaded Pyramid Network for Multi-person Pose Estimation2018
- Mask R-CNN2017