Kinematics Modeling Network for Video-based Human Pose Estimation
Authors:
Preprint
Abstract
Estimating human poses from videos is critical in human-computer interaction. Joints cooperate rather than move independently during human movement. There are both spatial and temporal correlations between joints. Despite the positive results of previous approaches, most focus on modeling the spatial correlation between joints while only straightforwardly integrating features along the temporal dimension, ignoring the temporal correlation between joints. In this work, we propose a plug-and-play kinematics modeling module (KMM) to explicitly model temporal correlations between joints across different frames by calculating their temporal similarity. In this way, KMM can capture motion cues of the current joint relative to all joints in different time. Besides, we formulate video-based human pose estimation as a Markov Decision Process and design a novel kinematics modeling network (KIMNet) to simulate the Markov Chain, allowing KIMNet to locate joints recursively. Our approach achieves state-of-the-art results on two challenging benchmarks. In particular, KIMNet shows robustness to the occlusion. The code will be released at https://github.com/YHDang/KIMNet.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- NRPose: Towards noise resistance for multi-person pose estimationPattern Recognition, 2023
- Aggregated pyramid gating network for human pose estimation without pre-trainingPattern Recognition, 2023
- Gated Region-Refine pose transformer for human pose estimationNeurocomputing, 2023
- Detecting and grouping keypoints for multi-person pose estimation using instance-aware attentionPattern Recognition, 2022
- Action Transformer: A self-attention model for short-time pose-based human action recognitionPattern Recognition, 2022
- Vector of Locally and Adaptively Aggregated Descriptors for Image Feature RepresentationPattern Recognition, 2021
- An adversarial human pose estimation network injected with graph structurePattern Recognition, 2021
- Structure-aware human pose estimation with graph convolutional networksPattern Recognition, 2020
- Exploring temporal consistency for human pose estimation in videosPattern Recognition, 2020
- Learning content and style: Joint action recognition and person identification from human skeletonsPattern Recognition, 2018