Pixel-Level Tiling with End-to-End Neural Compression for Tile-Based Video Streaming

Authors:
Publisher website
Google Scholar
Abstract
Multimedia data has become the dominant traffic through the mobile network. In recent years, the forms of videos become more immersive and interactive, raising significant challenges for video streaming due to the limited network resources. Since a viewer’s viewport is limited, video frame partition and tile-based streaming stand out as a feasible solution. Existing works all follow the Tiling-on-Frame (ToF) strategy, i.e., partition each frame into tiles before encoding. However, there is a challenging trade-off. Coarse tiling leads to redundant content transmission, while fine tiling reduces compression ratio due to loss of spatial information. ToF inevitably struggles to balance the tiling granularity and compression efficiency. To tackle this challenge, we introduce a pixel-level framework called Tiling-on-Bitstream (ToB). It eliminates the redundant contents and achieves high compression efficiency. The key idea is that, instead of ToF, we first encode the entire video into bitstreams and then select the corresponding bitstream as requested for streaming; and at the receiver side, the bitstream is decoded to retrieve the requested video content. Such ToB framework enables a fine-grained partition at pixel level, avoiding redundancy in boundary. To efficiently implement a prototype of ToB, we design Neural-EGDL, a neural network-based codec algorithm. To demonstrate the power of our design, we also apply it to 360° videos.
Funding Information
  • Science and Technology Planning Project of Guangdong Province (2024B0101030002)
  • Basic Research Project of Hetao Shenzhen-HK Scicence and Technology Cooperation Zone (HZQB-KCZYZ-2021067)
  • National Natural Science Foundation of China (62293482)
  • Shenzhen Science and Technology Program (JCYJ20230807114204010)
  • Guangdong Talents Program (2024TQ08X346)

This publication has 42 references indexed in Scilit: