Experience-Driven PCG via Reinforcement Learning: A Super Mario Bros Study
Authors:
- 17 August 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-9
Abstract
We introduce a procedural content generation (PCG) framework at the intersections of experience-driven PCG and PCG via reinforcement learning, named ED(PCG)RL, EDRL in short. EDRL is able to teach RL designers to generate endless playable levels in an online manner while respecting particular experiences for the player as designed in the form of reward functions. The framework is tested initially in the Super Mario Bros game. In particular, the RL designers of Super Mario Bros generate and concatenate level segments while considering the diversity among the segments. The correctness of the generation is ensured by a neural net-assisted evolutionary level repairer and the playability of the whole level is determined through AI-based testing. Our agents in this EDRL implementation learn to maximise a quantification of Koster's principle of fun by moderating the degree of diversity across level segments. Moreover, we test their ability to design fun levels that are diverse over time and playable. Our proposed framework is capable of generating endless, playable Super Mario Bros levels with varying degrees of fun, deviation from earlier segments, and playability. EDRL can be generalised to any game that is built as a segment-based sequential process and features a built-in compressed representation of its game content.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (61906083)
- Guangdong Provincial Key Laboratory (2020B121201001)
- Shenzhen Municipal Fundamental Research Program (JCYJ20190809121403553)
This publication has 32 references indexed in Scilit:
- ViZDoom: A Doom-based AI research platform for visual reinforcement learning2016
- Online level generation in Super Mario Bros via learning constructive primitives2016
- Player-adaptive Spelunky level generation2015
- Flow2014
- Intrinsic Motivation and Reinforcement Learning2012
- Novelty Search and the Problem with Objectives2011
- Experience-Driven Procedural Content GenerationIEEE Transactions on Affective Computing, 2011
- Developmental robotics, optimal artificial curiosity, creativity, music, and the fine artsConnection Science, 2006
- Episodic Memory: From Mind to BrainAnnual Review of Psychology, 2002
- On the Likelihood that One Unknown Probability Exceeds Another in View of the Evidence of Two SamplesBiometrika, 1933