Combined Reinforcement Learning and CPG Algorithm to Generate Terrain-Adaptive Gait of Hexapod Robots
Open Access
Abstract
Terrain adaptation research can significantly improve the motion performance of hexapod robots. In this paper, we propose a method that combines reinforcement learning with a central pattern generator (CPG) to enhance the terrain adaptation of hexapod robots in terms of gait planning. The hexapod robot’s complex task presents a high-dimensional observation and action space, which makes it challenging to directly apply reinforcement learning to robot control. Therefore, we utilize the CPG algorithm to generate the rhythmic gait while compressing the action space dimension of the agent. Additionally, the proposed method requires less internal sensor information, which exhibits strong applicability. Finally, we conduct experiments and deploy the proposed framework in the simulation environment. The results show that the terrain adaptation policy trained in our framework enables the hexapod robot to move more smoothly and efficiently on rugged terrain compared to the traditional CPG method.Keywords
Funding Information
- National Natural Science Foundation of China (61573148, 61603358, 2015B010919007, 2019A050520001)
- Science and Technology Planning Project of Guangdong Province, China (61573148, 61603358, 2015B010919007, 2019A050520001)
This publication has 30 references indexed in Scilit:
- Proprioceptive control of an over-actuated hexapod robot in unstructured terrain2016
- Robots that can adapt like animalsNature, 2015
- Terrain adaptation gait algorithm in a hexapod walking robot2014
- Rough terrain mapping and classification for foothold selection in a walking robotJournal of Field Robotics, 2011
- Pattern generators with sensory feedback for the control of quadruped locomotion2008
- The Kuramoto model: A simple paradigm for synchronization phenomenaReviews of Modern Physics, 2005
- Mechanisms of frequency and pattern control in the neural rhythm generatorsBiological Cybernetics, 1987
- Modeling of a Neural Pattern Generator with Coupled nonlinear OscillatorsIEEE Transactions on Biomedical Engineering, 1987
- Sustained oscillations generated by mutually inhibiting neurons with adaptationBiological Cybernetics, 1985
- LXXII.The heartbeat considered as a relaxation oscillation, and an electrical model of the heartJournal of Computers in Education, 1928