Rendering Optimizations for Virtual Reality Using Eye-Tracking
Authors:
- 1 November 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 398-405
Abstract
Optimizing rendering in virtual reality is an open problem in computer science. The nature of modern VR display technology (high refresh rate and increasing pixel density), coupled with the relatively slow growth modern compute capability, is leading to a bottleneck in VR performance. As we further research methodologies for improving rendering performance and accuracy for VR, it is important to understand the historical approaches and where they succeeded or failed in their approaches. Some implementations will double computing because of the need of stereoscopy, and thus have higher overhead for rendering. This can be improved with Multi-View Rendering where the GPU hardware can assist in duplicating rasterization for multiple views with differing projections. More recently, perception-based rendering has gained traction, which can be further accelerated using Variable Shading Rate or Multi-Rate Shading technology found on more recent GPUs. There has also been some success in using deep neural networks to assist with transmitting foveated content over a network. The advances in the field leave many open research questions, including sparse pixel rendering, driving user attention, and techniques and methodologies for combining variable shading rate images. This review focuses research associated with rendering optimizations for virtual reality using eye tracking, since it is becoming a feature present in consumer-level head-mounted displays. From our review, affordable off-the-shelf virtual reality and eye tracking are both leading to freeing up rendering resources towards improved performance and visual fidelity, as well as providing new and exciting opportunities for human-computer interaction.Keywords
Funding Information
- Natural Sciences and Engineering Research Council of Canada
This publication has 29 references indexed in Scilit:
- Perception-driven Accelerated RenderingComputer Graphics Forum, 2017
- Towards foveated rendering for gaze-tracked virtual realityACM Transactions on Graphics, 2016
- Extending the graphics pipeline with adaptive, multi-rate shadingACM Transactions on Graphics, 2014
- Foveated 3D graphicsACM Transactions on Graphics, 2012
- Yarbus, Eye Movements, and VisionI-Perception, 2010
- A GPU based saliency map for high-fidelity selective rendering2006
- Virtual Reality TechnologyPRESENCE: Virtual and Augmented Reality, 2003
- Eyes at the interface1982
- Stereopsis and stereoblindnessExperimental Brain Research, 1970
- METHODS OF INVESTIGATING EYE MOVEMENTSBritish Journal of Ophthalmology, 1948