Learning Agent-Aware Affordances for Closed-Loop Interaction with Articulated Objects
Authors:
- 29 May 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5916-5922
Abstract
Interactions with articulated objects are a challenging but important task for mobile robots. To tackle this challenge, we propose a novel closed-loop control pipeline, which integrates manipulation priors from affordance estimation with sampling-based whole-body control. We introduce the concept of agent-aware affordances which fully reflect the agent's capabilities and embodiment and we show that they outperform their state-of-the-art counterparts which are only conditioned on the end-effector geometry. Additionally, closed-loop affordance inference is found to allow the agent to divide a task into multiple non-continuous motions and recover from failure and unexpected states. Finally, the pipeline is able to perform long-horizon mobile manipulation tasks, i.e. opening and closing an oven, in the real world with high success rates (opening: 71%, closing: 72%).Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- A Transition-Aware Method for the Simulation of Compliant Contact With Regularized FrictionIEEE Robotics and Automation Letters, 2020
- Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation2019
- Shape2Motion: Joint Analysis of Motion Parts and Attributes From 3D Shapes2019
- PartNet: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level 3D Object Understanding2019
- Factored Pose Estimation of Articulated Objects using Efficient Nonparametric Belief Propagation2019
- Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning2018
- Information theoretic MPC for model-based reinforcement learning2017
- Model Predictive Path Integral Control: From Theory to Parallel ComputationJournal of Guidance, Control, and Dynamics, 2017
- Affordance detection of tool parts from geometric features2015
- Operating articulated objects based on experience2010