Trajectory-Aware Body Interaction Transformer for Multi-Person Pose Forecasting
Preprint
Abstract
Multi-person pose forecasting remains a challenging problem, especially in modeling fine-grained human body interaction in complex crowd scenarios. Existing methods typically represent the whole pose sequence as a temporal series, yet overlook interactive influences among people based on skeletal body parts. In this paper, we propose a novel Trajectory-Aware Body Interaction Transformer (TBIFormer) for multi-person pose forecasting via effectively modeling body part interactions. Specifically, we construct a Temporal Body Partition Module that transforms all the pose sequences into a Multi-Person Body-Part sequence to retain spatial and temporal information based on body semantics. Then, we devise a Social Body Interaction Self-Attention (SBI-MSA) module, utilizing the transformed sequence to learn body part dynamics for inter- and intra-individual interactions. Furthermore, different from prior Euclidean distance-based spatial encodings, we present a novel and efficient Trajectory-Aware Relative Position Encoding for SBI-MSA to offer discriminative spatial information and additional interactive clues. On both short- and long-term horizons, we empirically evaluate our framework on CMU-Mocap, MuPoTS-3D as well as synthesized datasets (6 ~ 10 persons), and demonstrate that our method greatly outperforms the state-of-the-art methods. Code will be made publicly available upon acceptance.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Skeleton-Parted Graph Scattering Networks for 3D Human Motion Prediction2022
- Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion PredictionIEEE Transactions on Image Processing, 2021
- Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction2020
- Socially and Contextually Aware Human Motion and Pose ForecastingIEEE Robotics and Automation Letters, 2020
- Unsupervised Online Video Object Segmentation With Motion Property UnderstandingIEEE Transactions on Image Processing, 2019
- Dynamic programming algorithm optimization for spoken word recognitionIEEE Transactions on Acoustics, Speech, and Signal Processing, 1978