Learning Tailored Adaptive Bitrate Algorithms to Heterogeneous Network Conditions: A Domain-Specific Priors and Meta-Reinforcement Learning Approach
- 15 June 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Selected Areas in Communications
- Vol. 40 (8) , 2485-2503
Abstract
Internet adaptive video streaming is a typical form of video delivery that leverages adaptive bitrate (ABR) algorithms to provide video services with high quality of experience (QoE) for various users in diverse and unique network conditions. Such heterogeneous network environments, which can be viewed as exogenous input processes, often lead to the unstable performance of ABR algorithms. Unfortunately, learning-based ABR algorithm which generated by state-of-the-art reinforcement learning (RL) technologies achieves good average performance but fails to perform well in all kinds of network conditions. In this work, considering the video playback process as the Input-driven Markov Decision Process (IMDP), we propose $\text{A}^{2}$ BR (Adaptation of ABR), a novel meta-RL ABR approach. $\text{A}^{2}$ BR is mainly composed of an online stage and an offline stage. It leverages meta-RL to learn an initial meta-policy with various network conditions at the offline stage and makes decisions in personalized network conditions at the online stage. At the same time, we continually optimize the meta-policy to the tailor-made ABR policy for varying the current network environment within few shots. Moreover, in order to improve the learning efficiency, we fully utilize domain knowledge for implementing a virtual player to replay the previously experienced network. Using trace-driven experiments on various scenarios including different vehicles, users, network types, and heterogeneous user-preferences, we show that $\text{A}^{2}$ BR outperforming recent ABR approaches with rapidly adapting to the personalized QoE metrics and specific network conditions. Testbed experimental results also illustrate the superiority of $\text{A}^{2}$ BR in adapting to the unseen environments.Keywords
Funding Information
- National Key Research and Development Program of China (2018YFB1003703)
- NSFC (61936011)
- Beijing Key Laboratory of Networked Multimedia
- Kuaishou-Tsinghua Joint Project (20192000456)
This publication has 55 references indexed in Scilit:
- VMAF reproducibility: Validating a perceptual practical video quality metric2017
- A bitstream-based, scalable video-quality model for HTTP adaptive streaming: ITU-T P.1203.12017
- CS2P2016
- BOLA: Near-optimal bitrate adaptation for online videos2016
- A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTPACM SIGCOMM Computer Communication Review, 2015
- A buffer-based approach to rate adaptationACM SIGCOMM Computer Communication Review, 2014
- Improving Fairness, Efficiency, and Stability in HTTP-Based Adaptive Video Streaming With FestiveIEEE/ACM Transactions on Networking, 2014
- Commute path bandwidth traces from 3G networks2013
- Partially Observable Markov Decision Processes2012
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998