Progressive Frame Patching for FoV-Based Point Cloud Video Streaming
- 19 May 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 6154-6167
Abstract
Many XR applications require the delivery of volumetric video to users. Point Cloud has become a popular volumetric video format. A dense point cloud consumes much higher bandwidth than a 2D/360 $^{\circ }$ video frame. User Field of View (FoV) is more dynamic with 6-DoF movement than 3-DoF movement. To save bandwidth, FoV-adaptive streaming predicts a user's FoV and only downloads point cloud data falling in the predicted FoV. However, it is vulnerable to FoV prediction errors, which can be significant when a long buffer is utilized for smoothed streaming. In this work, we propose a multi-round progressive refinement framework for point cloud video streaming. Instead of sequentially downloading point cloud frames, our solution simultaneously downloads/patches multiple frames falling into a sliding time-window, leveraging the inherent scalability of octree-based point-cloud coding. The optimal rate allocation among all tiles of active frames are solved numerically using the heterogeneous tile rate-quality functions calibrated by the predicted user FoV. Multi-frame downloading/patching simultaneously takes advantage of the streaming smoothness resulting from long buffer and the FoV prediction accuracy at short buffer length. We evaluate our streaming solution using simulations driven by real point cloud videos, real bandwidth traces, and 6-DoF FoV traces of real users. Our solution is robust against the bandwidth/FoV prediction errors, and can deliver high and smooth view quality in the face of bandwidth variations and dynamic user and point cloud movements.Keywords
All Related Versions
Funding Information
- USA
- NSF (CNS-2312839)
This publication has 37 references indexed in Scilit:
- A Toolkit to Benchmark Point Cloud Quality Metrics with Multi-Track Evaluation Criteria2024
- Learning to Predict on Octree for Scalable Point Cloud Geometry Coding2022
- Spatially Scalable Video-Based Point Cloud Compression2022
- QoE-Driven and Tile-Based Adaptive Streaming for Point Clouds2021
- Fuzzy Logic-Based Adaptive Point Cloud Video StreamingIEEE Open Journal of the Computer Society, 2020
- An overview of ongoing point cloud compression standardization activities: video-based (V-PCC) and geometry-based (G-PCC)APSIPA Transactions on Signal and Information Processing, 2020
- Realtime Mobile Bandwidth Prediction Using LSTM Neural Network2019
- A Two-Tier System for On-Demand Streaming of 360 Degree Video Over Dynamic NetworksIEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2019
- Rate-Utility Optimized Streaming of Volumetric Media for Augmented RealityIEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2019
- Real-time compression of point cloud streams2012