Volumetric Rendering with Baked Quadrature Fields
Authors:
Preprint
Abstract
We propose a novel Neural Radiance Field (NeRF) representation for non-opaque scenes that allows fast inference by utilizing textured polygons. Despite the high-quality novel view rendering that NeRF provides, a critical limitation is that it relies on volume rendering that can be computationally expensive and does not utilize the advancements in modern graphics hardware. Existing methods for this problem fall short when it comes to modelling volumetric effects as they rely purely on surface rendering. We thus propose to model the scene with polygons, which can then be used to obtain the quadrature points required to model volumetric effects, and also their opacity and colour from the texture. To obtain such polygonal mesh, we train a specialized field whose zero-crossings would correspond to the quadrature points when volume rendering, and perform marching cubes on this field. We then rasterize the polygons and utilize the fragment shaders to obtain the final colour image. Our method allows rendering on various devices and easy integration with existing graphics frameworks while keeping the benefits of volume rendering alive.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Baking Neural Radiance Fields for Real-Time View SynthesisIEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
- NerfAcc: Efficient Sampling Accelerates NeRFs2023
- MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures2023
- nerf2nerf: Pairwise Registration of Neural Radiance Fields2023
- Urban Radiance Fields2022
- Plenoxels: Radiance Fields without Neural Networks2022
- Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields2022
- Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields2022
- PlenOctrees for Real-time Rendering of Neural Radiance Fields2021
- State of the Art on Neural RenderingComputer Graphics Forum, 2020