Latency Compensation Through Image Warping For Remote Rendering-Based Volumetric Video Streaming
Authors:
- 16 October 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2026-2030
Abstract
Rendering multiple high-quality volumetric videos is still a challenge for today’s mobile devices. Remote rendering offloads complex rendering operations to a powerful server and provides the final result to the end device as a 2D video stream. A drawback of remote rendering is the significant increase of interaction latency that can degrade the user experience. We present a planar homography-based approach that compensates minor changes of the user’s head pose due to the interaction latency by warping the transmitted image on the client-side, just before it is sent to the display. In detail, we use the homography between the initial head pose when the image is rendered at the server and the latest available head pose of the user at the client. We perform controlled experiments using artificial camera traces to evaluate our approach. The results show that the proposed approach reduces the rendering errors significantly in terms of the mean-squared error between the rendered and reference images, especially combined with initial head motion prediction.Keywords
Funding Information
- Ministry of Education
This publication has 16 references indexed in Scilit:
- A Survey of Interactive Remote Rendering SystemsACM Computing Surveys, 2015
- The effect of registration error on tracking distant augmented objects2008
- Visual flow influences gait transition speed and preferred walking speedExperimental Brain Research, 2007
- Multiple View Geometry in Computer Vision2004
- A comparison of unscented and extended Kalman filtering for estimating quaternion motion2004
- A quaternion-based unscented Kalman filter for orientation tracking2003
- Tolerance of temporal delay in virtual environments2002
- Post-rendering 3D warping1997
- A frequency-domain analysis of head-motion prediction1995
- Priority rendering with a virtual reality address recalculation pipeline1994