Learning Visual Affordances with Target-Orientated Deep Q-Network to Grasp Objects by Harnessing Environmental Fixtures
Authors:
- 30 May 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2562-2568
Abstract
This paper introduces a challenging object grasping task and proposes a self-supervised learning approach. The goal of the task is to grasp an object which is not feasible with a single parallel gripper, but only with harnessing environment fixtures (e.g., walls, furniture, heavy objects). This Slide-to-Wall grasping task assumes no prior knowledge except the partial observation of a target object. Hence the robot should learn an effective policy given a scene observation that may include the target object, environmental fixtures, and any other disturbing objects. We formulate the problem as visual affordances learning for which Target-Oriented Deep Q-Network (TO-DQN) is proposed to efficiently learn visual affordance maps (i.e., Q-maps) to guide robot actions. Since the training necessitates robot's exploration and collision with the fixtures, TO-DQN is first trained safely with a simulated robot manipulator and then applied to a real robot. We empirically show that TO-DQN can learn to solve the task in different environment settings in simulation and outperforms a standard and a variant of Deep Q-Network (DQN) in terms of training efficiency and robustness. The testing performance in both simulation and real-robot experiments shows that the policy trained by TO-DQN achieves comparable performance to humans.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- Planning grasp strategies That Exploit Environmental Constraints2015
- Modeling compliant grasps exploiting environmental constraints2015
- Exploitation of environmental constraints in human and robotic graspingThe International Journal of Robotics Research, 2015
- Human-level control through deep reinforcement learningNature, 2015
- V-REP: A versatile and scalable robot simulation framework2013
- Templates for pre-grasp sliding interactionsRobotics and Autonomous Systems, 2012
- The mechanics of manipulation2005
- GraspIt!IEEE Robotics & Automation Magazine, 2004
- Scale-dependent graspIEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 2000
- Automatic Synthesis of Fine-Motion Strategies for RobotsThe International Journal of Robotics Research, 1984