DeepCPG Policies for Robot Locomotion
Preprint
Abstract
Central Pattern Generators (CPGs) form the neural basis of the observed rhythmic behaviors for locomotion in legged animals. The CPG dynamics organized into networks allow the emergence of complex locomotor behaviors. In this work, we take this inspiration for developing walking behaviors in multi-legged robots. We present novel DeepCPG policies that embed CPGs as a layer in a larger neural network and facilitate end-to-end learning of locomotion behaviors in deep reinforcement learning (DRL) setup. We demonstrate the effectiveness of this approach on physics engine-based insectoid robots. We show that, compared to traditional approaches, DeepCPG policies allow sample-efficient end-to-end learning of effective locomotion strategies even in the case of high-dimensional sensor spaces (vision). We scale the DeepCPG policies using a modular robot configuration and multi-agent DRL. Our results suggest that gradual complexification with embedded priors of these policies in a modular fashion could achieve non-trivial sensor and motor integration on a robot platform. These results also indicate the efficacy of bootstrapping more complex intelligent systems from simpler ones based on biological principles. Finally, we present the experimental results for a proof-of-concept insectoid robot system for which DeepCPG learned policies initially using the simulation engine and these were afterwards transferred to real-world robots without any additional fine-tuning.Keywords
All Related Versions
This publication has 40 references indexed in Scilit:
- Decentralized control and local information for robust and adaptive decentralized Deep Reinforcement LearningNeural Networks, 2021
- CPG-Actor: Reinforcement Learning for Central Pattern Generators2021
- Tails stabilize landing of gliding geckos crashing head-first into tree trunksCommunications Biology, 2021
- A backward control based on-Hopf oscillator with decoupled parameters for smooth locomotion of bio-inspired legged robotRobotics and Autonomous Systems, 2018
- Walknet, a bio-inspired controller for hexapod walkingBiological Cybernetics, 2013
- HECTOR, a New Hexapod Robot Platform with Increased Mobility - Control Approach, Design and Communication2012
- Central pattern generators based on Matsuoka oscillators for the locomotion of biped robotsArtificial Life and Robotics, 2008
- Mechanical aspects of legged locomotion controlArthropod Structure & Development, 2004
- Central pattern generators and the control of rhythmic movementsCurrent Biology, 2001
- Chemical Oscillations, Waves, and Turbulence1984