One-Shot Imitation Learning With Graph Neural Networks for Pick-and-Place Manipulation Tasks
Open Access
- 2 August 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (9) , 5926-5933
Abstract
The proposed work presents a framework based on Graph Neural Networks (GNN) that abstracts the task to be executed and directly allows the robot to learn task-specific rules from synthetic demonstrations given through imitation learning. A graph representation of the state space is considered to encode the task-relevant entities as nodes for a Pick-and-Place task declined at different levels of difficulty. During training, the GNN-based policy learns the underlying rules of the manipulation task focusing on the structural relevance and the type of objects and goals, relying on an external primitive to move the robot to accomplish the task. The GNN-policy has been trained as a node-classification approach by looking at the different configurations of the objects and goals present in the scene, learning the association between them with respect to their type for the Pick-and-Place task. The experimental results show a high generalization capability of the proposed model in terms of the number, positions, height distributions, and even configurations of the objects/goals. Thanks to the generalization, only a single image of the desired goal configuration is required at inference time.Keywords
Funding Information
- Leonardo Company S.p.A. (LDO/CTI/P/0026995/21)
This publication has 16 references indexed in Scilit:
- Combining Task and Motion Planning: Challenges and GuidelinesFrontiers in Robotics and AI, 2021
- Integrated Task and Motion PlanningAnnual Review of Control, Robotics, and Autonomous Systems, 2021
- Recent Advances in Robot Learning from DemonstrationAnnual Review of Control, Robotics, and Autonomous Systems, 2020
- Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning2020
- Neural Task Graphs: Generalizing to Unseen Tasks From a Single Video Demonstration2019
- Survey on human–robot collaboration in industrial settings: Safety, intuitive interfaces and applicationsMechatronics, 2018
- Literature review of Industry 4.0 and related technologiesJournal of Intelligent Manufacturing, 2018
- One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning2018
- From Skills to Symbols: Learning Symbolic Representations for Abstract High-Level PlanningJournal of Artificial Intelligence Research, 2018
- Recent progress on programming methods for industrial robotsRobotics and Computer-Integrated Manufacturing, 2012