Visual-Geometric Collaborative Guidance for Affordance Learning
Preprint
Abstract
Perceiving potential ``action possibilities'' (\ie, affordance) regions of images and learning interactive functionalities of objects from human demonstration is a challenging task due to the diversity of human-object interactions. Prevailing affordance learning algorithms often adopt the label assignment paradigm and presume that there is a unique relationship between functional region and affordance label, yielding poor performance when adapting to unseen environments with large appearance variations. In this paper, we propose to leverage interactive affinity for affordance learning, \ie extracting interactive affinity from human-object interaction and transferring it to non-interactive objects. Interactive affinity, which represents the contacts between different parts of the human body and local regions of the target object, can provide inherent cues of interconnectivity between humans and objects, thereby reducing the ambiguity of the perceived action possibilities. To this end, we propose a visual-geometric collaborative guided affordance learning network that incorporates visual and geometric cues to excavate interactive affinity from human-object interactions jointly. Besides, a contact-driven affordance learning (CAL) dataset is constructed by collecting and labeling over 55,047 images. Experimental results demonstrate that our method outperforms the representative models regarding objective metrics and visual quality. Project: \href{https://github.com/lhc1224/VCR-Net}{github.com/lhc1224/VCR-Net}.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- PEAR: phrase-based hand-object interaction anticipationScience China Information Sciences, 2025
- Grounded Affordance from Exocentric ViewInternational Journal of Computer Vision, 2023
- Learning Agent-Aware Affordances for Closed-Loop Interaction with Articulated Objects2023
- AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-Shot Interactions2022
- One-Shot Object Affordance Detection in the WildInternational Journal of Computer Vision, 2022
- Impression management strategies on enterprise social media platforms: An affordance perspectiveInternational Journal of Information Management, 2021
- A brief review of affordance in robotic manipulation researchAdvanced Robotics, 2017
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image AnnotationsInternational Journal of Computer Vision, 2017
- DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving2015
- Components of bottom-up gaze allocation in natural imagesVision Research, 2005