Exploring CausalWorld: Enhancing robotic manipulation via knowledge transfer and curriculum learning
Preprint
Abstract
This study explores a learning-based tri-finger robotic arm manipulating task, which requires complex movements and coordination among the fingers. By employing reinforcement learning, we train an agent to acquire the necessary skills for proficient manipulation. To enhance the efficiency and effectiveness of the learning process, two knowledge transfer strategies, fine-tuning and curriculum learning, were utilized within the soft actor-critic architecture. Fine-tuning allows the agent to leverage pre-trained knowledge and adapt it to new tasks. Several variations like model transfer, policy transfer, and across-task transfer were implemented and evaluated. To eliminate the need for pretraining, curriculum learning decomposes the advanced task into simpler, progressive stages, mirroring how humans learn. The number of learning stages, the context of the sub-tasks, and the transition timing were found to be the critical design parameters. The key factors of two learning strategies and corresponding effects were explored in context-aware and context-unaware scenarios, enabling us to identify the scenarios where the methods demonstrate optimal performance, derive conclusive insights, and contribute to a broader range of learning-based engineering applications.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Stable and Safe Reinforcement Learning via a Barrier-Lyapunov Actor-Critic Approach2023
- Long-Range Risk-Aware Path Planning for Autonomous Ships in Complex and Dynamic EnvironmentsJournal of Computing and Information Science in Engineering, 2023
- Path Planning for Autonomous Systems Design: A Focus Genetic Algorithm for Complex EnvironmentsJournal of Autonomous Vehicles and Systems, 2022
- Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-ReviewRobotics, 2021
- Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collectionThe International Journal of Robotics Research, 2017
- ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars2016