UniffordInteractor: Reasoning the Unified Affordance for Human and Multi-Object Interaction
Open Access
- 9 November 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 45 (6) , 1-13
Abstract
Deriving plausible and dynamic Human Object Interaction (HOI) has gained tremendous progress in Embodied AI and Immersive AR/VR. While existing HOI methods have effectively explored human interaction with individual or fixed objects, they still show limitations in complex multi-object interaction scenes. Meanwhile, previous approaches focus more on the contact information held by human-object relation, overlooking the significance of the contact situation of various entities ( i.e. , object-object). To address these limitations, we propose UniffordInteractor , a Reasoning-then-Interacting framework that leverages unified affordance for realistic and long-horizon human and multi-object interaction modeling. At the reasoning stage, we devise a Unified Affordance Reasoner (UAR) that employs a fine-tuned 3D Multimodal Large Language Model for unified object-centric affordance prediction. At the interacting stage, we propose Hierarchical Interaction Generator (HIG) , which formulates the interaction generation as a hierarchical ( e.g. , root→body→hand→objects) autoregressive prediction, conditioned on the obtained unified affordance. Extensive experiments demonstrate that UniffordInteractor surpasses SOTA counterparts in human and multi-object interaction generation. Our project page is available at this site. Code and dataset will be released upon publication.Keywords
Funding Information
- NSFC (No.62306309 & U1836217)
This publication has 23 references indexed in Scilit:
- CG-HOI: Contact-Guided 3D Human-Object Interaction Generation2024
- HOIAnimator: Generating Text-Prompt Human-Object Animations Using Novel Perceptive Diffusion Models2024
- LASO: Language-Guided Affordance Segmentation on 3D Object2024
- COUCH: Towards Controllable Human-Chair Interactions2022
- Learning Affordance Grounding from Exocentric Images2022
- 3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding2021
- Expressive Body Capture: 3D Hands, Face, and Body From a Single Image2019
- PartNet: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level 3D Object Understanding2019
- Demo2Vec: Reasoning Object Affordances from Online Videos2018
- Learning to Act Properly: Predicting and Explaining Affordances from Images2018