Active Exploration for Neural Global Illumination of Variable Scenes
Open Access
- 13 May 2022
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 41 (5) , 1-18
Abstract
Neural rendering algorithms introduce a fundamentally new approach for photorealistic rendering, typically by learning a neural representation of illumination on large numbers of ground truth images. When training for a given variable scene, such as changing objects, materials, lights, and viewpoint, the space \( \mathcal {D} \) of possible training data instances quickly becomes unmanageable as the dimensions of variable parameters increase. We introduce a novel Active Exploration method using Markov Chain Monte Carlo, which explores \( \mathcal {D} \) , generating samples (i.e., ground truth renderings) that best help training and interleaves training and on-the-fly sample data generation. We introduce a self-tuning sample reuse strategy to minimize the expensive step of rendering training samples. We apply our approach on a neural generator that learns to render novel scene instances given an explicit parameterization of the scene configuration. Our results show that Active Exploration trains our network much more efficiently than uniformly sampling and, together with our resolution enhancement approach, achieves better quality than uniform sampling at convergence. Our method allows interactive rendering of hard light transport paths (e.g., complex caustics), which require very high samples counts to be captured, and provides dynamic scene navigation and manipulation, after training for 5 to 18 hours depending on required quality and variations.
Keywords
All Related Versions
Funding Information
- Beijing Municipal Natural Science Foundation for Distinguished Young Scholars (JQ21013)
- National Natural Science Foundation of China (62061136007, and 61872440)
- Royal Society Newton Advanced Fellowship (NAF\verb|\|R2\verb|\|192151)
- Samsung GRO Program (CHS-1528025, and W911NF-21-2-0104)
This publication has 38 references indexed in Scilit:
- Deep Shading: Convolutional Neural Networks for Screen Space ShadingComputer Graphics Forum, 2017
- Real-time global illumination using precomputed light field probes2017
- Learning Weight Uncertainty with Stochastic Gradient MCMC for Shape Classification2016
- Global illumination with radiance regression functionsACM Transactions on Graphics, 2013
- Approximating dynamic global illumination in image space2009
- Structural Similarity-Based Object Tracking in Video Sequences2006
- Local, deformable precomputed radiance transferACM Transactions on Graphics, 2005
- The irradiance volumeIEEE Computer Graphics and Applications, 1998
- Metropolis light transport1997
- Bayesian Learning for Neural Networks1996