Neural Temporal Adaptive Sampling and Denoising
- 1 May 2020
- journal article
- conference paper
- Published by Wiley in Computer Graphics Forum
- Vol. 39 (2) , 147-155
Abstract
Despite recent advances in Monte Carlo path tracing at interactive rates, denoised image sequences generated with few samples per‐pixel often yield temporally unstable results and loss of high‐frequency details. We present a novel adaptive rendering method that increases temporal stability and image fidelity of low sample count path tracing by distributing samples via spatio‐temporal joint optimization of sampling and denoising. Adding temporal optimization to the sample predictor enables it to learn spatio‐temporal sampling strategies such as placing more samples in disoccluded regions, tracking specular highlights, etc; adding temporal feedback to the denoiser boosts the effective input sample count and increases temporal stability. The temporal approach also allows us to remove the initial uniform sampling step typically present in adaptive sampling algorithms. The sample predictor and denoiser are deep neural networks that we co‐train end‐to‐end over multiple consecutive frames. Our approach is scalable, allowing trade‐off between quality and performance, and runs at near real‐time rates while achieving significantly better image quality and temporal stability than previous methods.Keywords
This publication has 12 references indexed in Scilit:
- Sample-based Monte Carlo denoising using a kernel-splatting networkACM Transactions on Graphics, 2019
- Blockwise Multi-Order Feature Regression for Real-Time Path-Tracing ReconstructionACM Transactions on Graphics, 2019
- Gradient Estimation for Real-time Adaptive Temporal FilteringProceedings of the ACM on Computer Graphics and Interactive Techniques, 2018
- Denoising with kernel prediction and asymmetric loss functionsACM Transactions on Graphics, 2018
- Deep Adaptive Sampling for Low Sample Count RenderingComputer Graphics Forum, 2018
- Frame-Recurrent Video Super-Resolution2018
- Kernel-predicting convolutional networks for denoising Monte Carlo renderingsACM Transactions on Graphics, 2017
- Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoderACM Transactions on Graphics, 2017
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Recent Advances in Adaptive Sampling and Reconstruction for Monte Carlo RenderingComputer Graphics Forum, 2015