Adaptive Bitrate Algorithms via Deep Reinforcement Learning With Digital Twins Assisted Trajectory
- 28 March 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Network Science and Engineering
- Vol. 11 (4) , 3522-3535
Abstract
Adaptive bitrate (ABR) algorithms based on deep reinforcement learning (RL) can continuously improve their adaptability to network conditions. Existing methods usually adopt a reward function to train the ABR policies. However, in some network conditions, the quality of experience (QoE) performances obtained from the deterministic reward functions are not consistent with the users' perception. In this paper, we propose a novel learning-based deep reward model from video trajectory preference data to obtain a dynamic reward function that better reflects users' preference of the video stream. To augment the training data with more precise aggregation in a quick way, we leverage the digital twin (DT) technology to map the users' trajectory into virtual space. The digital twin can mimic users' video trajectory preferences which is used to train a reward model that can precisely predict QoE. Integrated with this reward model, the RL algorithm can generate an ABR policy that optimizes the user-perceived QoE. Experiment results show that the accuracy of the proposed reward model outperforms state-of-the-art reward functions by 13.6% in preference prediction. Compared with other RL-based ABR policies, our RL-based trajectory-preferred ABR algorithm can increase the average user QoE by 16.4%.Keywords
Funding Information
- National Natural Science Foundation of China (U22A2021, U22A2054)
- Ministry of Education - Singapore (MOE-T2EP20221-0017)
This publication has 40 references indexed in Scilit:
- Adaptive Bitrate with User-level QoE Preference for Video Streaming2022
- Accelerating Reinforcement Learning using EEG-based implicit human feedbackNeurocomputing, 2021
- Adaptive Video Streaming via Deep Reinforcement Learning from User Trajectory Preferences2020
- Stick: A Harmonious Fusion of Buffer-based and Learning-based Approach for Adaptive Streaming2020
- Tiyuntsong: A Self-Play Reinforcement Learning Approach for ABR Video Streaming2019
- Feature-based prediction of streaming video QoE: Distortions, stalling and memorySignal Processing: Image Communication, 2018
- HotDASH: Hotspot Aware Adaptive Video Streaming Using Deep Reinforcement Learning2018
- A Survey on Bitrate Adaptation Schemes for Streaming Media Over HTTPIEEE Communications Surveys & Tutorials, 2018
- BOLA: Near-optimal bitrate adaptation for online videos2016
- HDR-VQM: An objective quality measure for high dynamic range videoSignal Processing: Image Communication, 2015