Towards a common environment for learning scheduling algorithms
Authors:
- 17 November 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Abstract
We propose a way to model and integrate HPC scheduling simulators into a popular Reinforcement Learning toolkit. We show experimentally that such an approach not only aids researchers being able to iterate faster by means of software reuse, but also to achieve state-of-the-art performance with 10x less interactions with the environment. We validate the simulation model's correctness by using unit tests, assertions and experimental comparisons. We also share an open source implementation of the model that will benefit researchers in resource management tasks assisted by Machine Learning.Keywords
This publication has 13 references indexed in Scilit:
- DeepMimicACM Transactions on Graphics, 2018
- Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General AgentsJournal of Artificial Intelligence Research, 2018
- Towards understanding HPC users and systems: A NERSC case studyJournal of Parallel and Distributed Computing, 2018
- Job placement advisor based on turnaround predictions for HPC hybrid cloudsFuture Generation Computer Systems, 2017
- Resource Management with Deep Reinforcement Learning2016
- Helping HPC Users Specify Job Memory Requirements via Machine Learning2016
- The Arcade Learning Environment: An Evaluation Platform for General AgentsJournal of Artificial Intelligence Research, 2013
- Instability in parallel job scheduling simulation: the role of workload flurries2006
- TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level PlayNeural Computation, 1994
- Function Optimization using Connectionist Reinforcement Learning AlgorithmsConnection Science, 1991