A Lightweight DRDPG-Based RL DVFS for Video Rendering on CPU-GPU Integrated SoC
- 18 January 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems I: Regular Papers
- Vol. 71 (5) , 2119-2131
Abstract
Reinforcement learning (RL) based dynamic voltage and frequency scaling (DVFS) is an effective approach to balance performance and power consumption for video rendering applications. To approximate the “God’s eye view” regulation, the CPU-GPU should be fine-grained regulated and the SoC have to be fully observed by a RL-based DVFS governor. Fine-grained regulation with traditional value-based RL governor suffers action space explosion and it is impossible to have a fully observable SoC. To address these two issues, a governor based on deep recurrent deterministic gradient (DRDPG) governor is proposed. The governor is based on Deterministic Policy Gradient (DDPG) algorithm with embedded recurrent neural network (RNN). The DDPG algorithm guarantees fine-grained power regulation without action space explosion and the RNN-FC network topology mitigates the partial observability issue. Evaluated on the Nvidia Jetson NX platform, the proposed DRDPG governor achieves over 19% better regulation efficiency compared with Linux default governors and shows superior regulation efficiency to other RL-based state-of-the-arts. Implemented in a 55-nm CMOS process, the proposed governor draws merely 2.16-mA from a 1.2-V supply at 1-MHz clock occupying a silicon area of 0.075mm $^2$ .Keywords
Funding Information
- Alibaba DAMO Academy through Damo Academy Innovative Research Program
This publication has 25 references indexed in Scilit:
- An Integrated CPU-GPU Frequency Scaling Governor Based on Deep Recurrent Q-Network for Partially Observable Rendering Applications2023
- A Survey of Domain-Specific Architectures for Reinforcement LearningIEEE Access, 2022
- A survey paper on design and implementation of multipliers for digital system applicationsArtificial Intelligence Review, 2022
- An adaptive on-line CPU-GPU governor for games on mobile devices2017
- Processors and Systems2017
- Frame-based and thread-based power management for mobile games on HMP platforms2016
- Human-level control through deep reinforcement learningNature, 2015
- Learning based DVFS for simultaneous temperature, performance and energy management2012
- Partially Observable Markov Decision Processes2012
- Optimal piecewise linear approximation of digitized curves1997