A FPGA Accelerator of Distributed A3C Algorithm with Optimal Resource Deployment
Open Access
- 1 January 2024
- journal article
- research article
- Published by Institution of Engineering and Technology (IET) in IET Computers & Digital Techniques
- Vol. 2024 (1)
Abstract
The asynchronous advantage actor‐critic (A3C) algorithm is widely regarded as one of the most effective and powerful algorithms among various deep reinforcement learning algorithms. However, the distributed and asynchronous nature of the A3C algorithm brings increased algorithm complexity and computational requirements, which not only leads to an increased training cost but also amplifies the difficulty of deploying the algorithm on resource‐limited field programmable gate array (FPGA) platforms. In addition, the resource wastage problem caused by the distributed training characteristics of A3C algorithms and the resource allocation problem affected by the imbalance between the computational amount of inference and training need to be carefully considered when designing accelerators. In this paper, we introduce a deployment strategy designed for distributed algorithms aimed at enhancing the resource utilization of hardware devices. Subsequently, a FPGA architecture is constructed specifically for accelerating the inference and training processes of the A3C algorithm. The experimental results show that our proposed deployment strategy reduces resource consumption by 62.5% and decreases the number of agents waiting for training by 32.2%, and the proposed A3C accelerator achieves 1.83× and 2.39× improvements in speedup compared to CPU (Intel i9‐13900K) and GPU (NVIDIA RTX 4090) with less power consumption respectively. Furthermore, our design shows superior resource efficiency compared to existing works.Keywords
Funding Information
- National Key Research and Development Program of China (2022YFD2000700)
This publication has 28 references indexed in Scilit:
- ReLAccS: A Multilevel Approach to Accelerator Design for Reinforcement Learning on FPGA-Based SystemsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2021
- A Hardware Accelerator for Language-Guided Reinforcement LearningIEEE Design & Test, 2021
- An Energy Efficient EdgeAI Autoencoder Accelerator for Reinforcement LearningIEEE Open Journal of Circuits and Systems, 2021
- An Efficient Hardware Implementation of Reinforcement Learning: The Q-Learning AlgorithmIEEE Access, 2019
- A CGRA based Neural Network Inference Engine for Deep Reinforcement Learning2018
- A Compact and Configurable Long Short-Term Memory Neural Network Hardware Architecture2018
- Overtaking Maneuvers in Simulated Highway Driving using Deep Reinforcement Learning2018
- Mastering the game of Go with deep neural networks and tree searchNature, 2016
- Human-level control through deep reinforcement learningNature, 2015
- Natural Actor-CriticNeurocomputing, 2008