Trajectory-Aware Body Interaction Transformer for Multi-Person Pose Forecasting
Authors:
- 1 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 17121-17130
Abstract
Multi-person pose forecasting remains a challenging problem, especially in modeling fine-grained human body interaction in complex crowd scenarios. Existing methods typically represent the whole pose sequence as a temporal series, yet overlook interactive influences among people based on skeletal body parts. In this paper, we propose a novel Trajectory-Aware Body Interaction Transformer (TBIFormer) for multi-person pose forecasting via effectively modeling body part interactions. Specifically, we construct a Temporal Body Partition Module that transforms all the pose sequences into a Multi-Person Body-Part sequence to retain spatial and temporal information based on body semantics. Then, we devise a Social Body Interaction Self-Attention (SBI-MSA) module, utilizing the transformed sequence to learn body part dynamics for inter- and intra-individual interactions. Furthermore, different from prior Euclidean distance-based spatial encodings, we present a novel and efficient Trajectory-Aware Relative Position Encoding for SBI-MSA to offer discriminative spatial information and additional interactive clues. On both short-and long-term horizons, we empirically evaluate our frame-work on CMU-Mocap, MuPoTS-3D as well as synthesized datasets (6 ~ 10 persons), and demonstrate that our method greatly outperforms the state-of-the-art methods.Keywords
All Related Versions
Funding Information
- Zhejiang Provincial Natural Science Foundation (LGF21F20012)
- Zhejiang Provincial Science and Technology Program in China (2021C03137)
This publication has 27 references indexed in Scilit:
- Single-Shot Multi-person 3D Pose Estimation from Monocular RGB2018
- On Human Motion Prediction Using Recurrent Neural Networks2017
- Social LSTM: Human Trajectory Prediction in Crowded Spaces2016
- Structural-RNN: Deep Learning on Spatio-Temporal Graphs2016
- Recurrent Network Models for Human Dynamics2015
- Social Psychology2015
- UMPM benchmark: A multi-person dataset with synchronized video and motion capture data for evaluation of articulated human motion and interaction2011
- Long Short-Term MemoryNeural Computation, 1997
- Dynamic programming algorithm optimization for spoken word recognitionIEEE Transactions on Acoustics, Speech, and Signal Processing, 1978
- Discrete Cosine TransformIEEE Transactions on Computers, 1974