Collaborative Streaming and Super Resolution Adaptation for Mobile Immersive Videos
Authors:
- 17 May 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-10
Abstract
Tile-based streaming and super resolution are two representative technologies adopted to improve bandwidth efficiency of immersive video steaming. The former allows selective download of contents in the user viewport by splitting the video into multiple independently decodable tiles. The latter leverages client-side computation to reconstruct the received video into higher quality using advanced neural network models. In this work, we propose CASE, a collaborated adaptive streaming and enhancement framework for mobile immersive videos, which integrates super resolution with tile-based streaming to optimize user experience with dynamic bandwidth and limited computing capability. To coordinate the video transmission and reconstruction in CASE, we identify and address several key design issues including unified video quality assessment, computation complexity model for super resolution, and buffer analysis considering the interplay between transmission and reconstruction. We further formulate the quality-of-experience (QoE) maximization problem for mobile immersive video streaming and propose a rate adaptation algorithm to make the best decisions for download and for reconstruction based on the Lyapunov optimization theory. Extensive evaluation results validate the superiority of our proposed approach, which presents stable performance with considerable QoE improvement, while enabling trade-off between playback smoothness and video quality.Keywords
Funding Information
- National Natural Science Foundation of China
This publication has 22 references indexed in Scilit:
- HEVC-compliant Tile-based Streaming of Panoramic Video for Virtual Reality Applications2016
- HTTP/2-Based Adaptive Streaming of HEVC Video Over 4G/LTE NetworksIEEE Communications Letters, 2016
- A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTPACM SIGCOMM Computer Communication Review, 2015
- Applying the roofline model2014
- Scope of validity of PSNR in image/video quality assessmentElectronics Letters, 2008
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Advances and challenges in super‐resolutionInternational Journal of Imaging Systems and Technology, 2004
- A locally adaptive zooming algorithm for digital imagesImage and Vision Computing, 2002
- Image Interpolation and Resampling2000
- The aim and method of the color image scaleColor Research & Application, 1981