Viewport Prediction for Volumetric Video Streaming by Exploring Video Saliency and User Trajectory Information
- 11 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 35 (12) , 12816-12829
Abstract
Volumetric video, also referred to as hologram video, is an emerging medium that represents 3D content in extended reality. As a next-generation video technology, it is poised to become a key application in 5G and future wireless communication networks. Because each user generally views only a specific portion of the volumetric video, known as the viewport, accurate prediction of the viewport is crucial for ensuring an optimal streaming performance. Despite its significance, research in this area is still in the early stages. To this end, this paper introduces a novel approach called Saliency and Trajectory-based Viewport Prediction (STVP), which enhances the accuracy of viewport prediction in volumetric video streaming by effectively leveraging both video saliency and viewport trajectory information. In particular, we first introduce a novel sampling method, Uniform Random Sampling (URS), which efficiently preserves video features while minimizing computational complexity. Next, we propose a saliency detection technique that integrates both spatial and temporal information to identify visually static and dynamic geometric and luminance-salient regions. Finally, we fuse saliency and trajectory information to achieve more accurate viewport prediction. Extensive experimental results validate the superiority of our method over existing state-of-the-art schemes. To the best of our knowledge, this is the first comprehensive study of viewport prediction in volumetric video streaming. We also make the source code of this work publicly available.Keywords
Funding Information
- National Natural Science Foundation of China (52077049)
- Anhui Provincial Natural Science Foundation (2008085UD04)
This publication has 51 references indexed in Scilit:
- PointFlowNet: Learning Representations for Rigid Motion Estimation From Point Clouds2019
- FlowNet3D: Learning Scene Flow in 3D Point Clouds2019
- Graph Attention Convolution for Point Cloud Semantic Segmentation2019
- PointWeb: Enhancing Local Neighborhood Features for Point Cloud Processing2019
- Flex-Convolution2019
- VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection2018
- SO-Net: Self-Organizing Network for Point Cloud Analysis2018
- Revisiting Video Saliency: A Large-Scale Benchmark and a New Model2018
- Shooting a moving target: Motion-prediction-based transmission for 360-degree videos2016
- A model of saliency-based visual attention for rapid scene analysisIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998