Auto-Regressive Diffusion for Generating 3D Human-Object Interactions
Open Access
- 11 April 2025
- journal article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (3) , 3131-3139
Abstract
Text-driven Human-Object Interaction (Text-to-HOI) generation is an emerging field with applications in animation, video games, virtual reality, and robotics. A key challenge in HOI generation is maintaining interaction consistency in long sequences. Existing Text-to-Motion-based approaches, such as discrete motion tokenization, cannot be directly applied to HOI generation due to limited data in this domain and the complexity of the modality. To address the problem of interaction consistency in long sequences, we propose an autoregressive diffusion model (ARDHOI) that predicts the next continuous token. Specifically, we introduce a Contrastive Variational Autoencoder (cVAE) to learn a physically plausible space of continuous HOI tokens, thereby ensuring that generated human-object motions are realistic and natural. For generating sequences autoregressively, we develop a Mamba-based context encoder to capture and maintain consistent sequential actions. Additionally, we implement an MLP-based denoiser to generate the subsequent token conditioned on the encoded context. Our model has been evaluated on the OMOMO and BEHAVE datasets, where it outperforms existing state-of-the-art methods in terms of both performance and inference speed. This makes ARDHOI a robust and efficient solution for text-driven HOI tasks.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Video Mamba Suite: State Space Model as a Versatile Alternative for Video UnderstandingInternational Journal of Computer Vision, 2025
- THOR: Text to Human-Object Interaction Diffusion via Relation Intervention2025
- HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Models2025
- Controllable Human-Object Interaction Synthesis2024
- InterGen: Diffusion-Based Multi-human Motion Generation Under Complex InteractionsInternational Journal of Computer Vision, 2024
- InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion2023
- ActFormer: A GAN-based Transformer towards General Action-Conditioned 3D Human Motion Generation2023
- TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts2022
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- Long-Term Human Motion Prediction with Scene Context2020