Lamarckian Platform: Pushing the Boundaries of Evolutionary Reinforcement Learning Toward Asynchronous Commercial Games
- 21 September 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Games
- Vol. 16 (1) , 51-63
Abstract
Despite the emerging progress of integrating evolutionary computation into reinforcement learning, the absence of a high-performance platform endowing composability and massive parallelism causes non-trivial difficulties for research and applications related to asynchronous commercial games. Here we introduce Lamarckian 1 1The code and demonstrational setup of Lamarckian are publicly available at https://github. com/lamarckian/lamarckian. – an open-source platform featuring support for evolutionary reinforcement learning scalable to distributed computing resources. To improve the training speed and data efficiency, Lamarckian adopts optimized communication methods and an asynchronous evolutionary reinforcement learning workflow. To meet the demand for an asynchronous interface by commercial games and various methods, Lamarckian tailors an asynchronous Markov Decision Process interface and designs an object-oriented software architecture with decoupled modules. In comparison with the state-of-the-art RLlib, we empirically demonstrate the unique advantages of Lamarckian on benchmark tests with up to 6000 CPU cores: i) both the sampling efficiency and training speed are doubled when running PPO on Google football game; ii) the training speed is 13 times faster when running PBT+PPO on Pong game. Moreover, we also present two use cases: i) how Lamarckian is applied to generating behavior-diverse game AI; ii) how Lamarckian is applied to game balancing tests for an asynchronous commercial game.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (61906081)
- Shenzhen Science and Technology Program (RCBS20200714114817264)
- Guangdong Provincial Key Laboratory (2020B121201001)
- Program for Guangdong Introducing Innovative and Entrepreneurial Teams (2017ZT07X386)
This publication has 26 references indexed in Scilit:
- Derivative-free reinforcement learning: a reviewFrontiers of Computer Science, 2021
- Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning2019
- Evolving the Hearthstone Meta2019
- Human-level control through deep reinforcement learningNature, 2015
- Empirical evaluation methods for multiobjective reinforcement learning algorithmsMachine Learning, 2010
- Generating diverse opponents with multiobjective evolution2008
- Comparison between Single-Objective and Multi-Objective Genetic Algorithms: Performance Comparison and Performance Measures2006
- Scalable Test Problems for Evolutionary Multiobjective Optimization2005
- A fast and elitist multiobjective genetic algorithm: NSGA-IIIEEE Transactions on Evolutionary Computation, 2002
- A genetic algorithm tutorialStatistics and Computing, 1994