A Survey of Domain-Specific Architectures for Reinforcement Learning
Open Access
- 26 January 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 13753-13767
Abstract
Reinforcement learning algorithms have been very successful at solving sequential decision-making problems in many different problem domains. However, their training is often time-consuming, with training times ranging from multiple hours to weeks. The development of domain-specific architectures for reinforcement learning promises faster computation times, decreased experiment turn-around time, and improved energy efficiency. This paper presents a review of hardware architectures for the acceleration of reinforcement learning algorithms. FPGA-based implementations are the focus of this work, but GPU-based approaches are considered as well. Both tabular and deep reinforcement learning algorithms are included in this survey. The techniques employed in different implementations are highlighted and compared. Finally, possible areas for future work are suggested, based on the preceding discussion of existing architectures.Keywords
Funding Information
- European Union’s Horizon 2020 Research and Innovation Program through the Very Efficient Deep Learning in IoT (VEDLIoT) Project (957197)
This publication has 53 references indexed in Scilit:
- Human-level control through deep reinforcement learningNature, 2015
- MuJoCo: A physics engine for model-based control2012
- Adaptive Power Management Based on Reinforcement Learning for Embedded SystemLecture Notes in Computer Science, 2008
- FPGA Architecture: Survey and ChallengesFoundations and Trends® in Electronic Design Automation, 2008
- Large-scale tabular-form hardware architecture for Q-Learning with delays2007
- Hardware Architecture of Reinforcement Learning Scheme for Dynamic Power Management in Embedded SystemsEURASIP Journal on Embedded Systems, 2007
- Extreme learning machine: a new learning scheme of feedforward neural networks2005
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- Fast Exact Multiplication by the HessianNeural Computation, 1994
- Q-learningMachine Learning, 1992