Real-time Rendering with a Neural Irradiance Volume
Preprint
Abstract
Rendering diffuse global illumination in real-time is often approximated by pre-computing and storing irradiance in a 3D grid of probes. As long as most of the scene remains static, probes approximate irradiance for all surfaces immersed in the irradiance volume, including novel dynamic objects. This approach, however, suffers from aliasing artifacts and high memory consumption. We propose Neural Irradiance Volume (NIV), a neural-based technique that allows accurate real-time rendering of diffuse global illumination via a compact pre-computed model, overcoming the limitations of traditional probe-based methods, such as the expensive memory footprint, aliasing artifacts, and scene-specific heuristics. The key insight is that neural compression creates an adaptive and amortized representation of irradiance, circumventing the cubic scaling of grid-based methods. Our superior memory-scaling improves quality by at least 10x at the same memory budget, and enables a straightforward representation of higher-dimensional irradiance fields, allowing rendering of time-varying or dynamic effects without requiring additional computation at runtime. Unlike other neural rendering techniques, our method works within strict real-time constraints, providing fast inference (around 1 ms per frame on consumer GPUs at full HD resolution), reduced memory usage (1-5 MB for medium-sized scenes), and only requires a G-buffer as input, without expensive ray tracing or denoising.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- LightFormer: Light-Oriented Global Neural Rendering in Dynamic SceneACM Transactions on Graphics, 2024
- Real‐time Neural Rendering of Dynamic Light FieldsComputer Graphics Forum, 2024
- Active Exploration for Neural Global Illumination of Variable ScenesACM Transactions on Graphics, 2022
- Neural radiosityACM Transactions on Graphics, 2021
- Glossy probe reprojection for interactive global illuminationACM Transactions on Graphics, 2020
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lightingACM Transactions on Graphics, 2020
- The design and evolution of the UberBake light baking systemACM Transactions on Graphics, 2020
- Global illumination with radiance regression functionsACM Transactions on Graphics, 2013
- Joint bilateral upsamplingACM Transactions on Graphics, 2007