Real-Time Monte Carlo Denoising With Adaptive Fusion Network
Open Access
- 23 February 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 29154-29165
Abstract
Real-time Monte Carlo denoising aims to denoise a 1spp-rendered image with a limited time budget. Many latest techniques for real-time Monte Carlo denoising utilize temporal accumulation (TA) as a pre-processing to improve the temporal stability of successive frames and increase the effective spp. However, existing techniques using TA used to suffer from significant performance degradation when TA does not work well. In addition, they have the disadvantage of deteriorating performance in dynamic scenes because pixel information of the current frame cannot be sufficiently utilized due to the pixel averaging effect between temporally adjacent frames. To solve this problem, this paper proposes a framework that utilizes both 1spp images and temporally accumulated 1spp (TA-1spp) images. First, the multi-scale kernel prediction module estimates kernel maps for filtering 1spp images and TA-1spp images, respectively. Then, the filtered images are properly fused so that the two advantages of 1spp and TA-1spp images can create synergy. Also, the remaining noise is removed through the refinement module and fine details are reconstructed to improve the model flexibility, beyond using only the kernel prediction module. As a result, we achieve better quantitative and qualitative performance at 39% faster than state-of-the-art (SOTA) real-time Monte Carlo denoisers.Keywords
Funding Information
- Ministry of Science and ICT, South Korea (2022-0-00955, 2022R1A2C2010095, 2022R1A4A1033549, RS-2022-00155915)
This publication has 26 references indexed in Scilit:
- Progressive Denoising of Monte Carlo Rendered ImagesComputer Graphics Forum, 2022
- Real‐time Monte Carlo Denoising with Weight Sharing Kernel Prediction NetworkComputer Graphics Forum, 2021
- RepVGG: Making VGG-style ConvNets Great Again2021
- A survey on deep learning-based Monte Carlo denoisingComputational Visual Media, 2021
- Neural Denoising with Layer EmbeddingsComputer Graphics Forum, 2020
- Neural Temporal Adaptive Sampling and DenoisingComputer Graphics Forum, 2020
- Nonlinearly Weighted First-order Regression for Denoising Monte Carlo RenderingsComputer Graphics Forum, 2016
- Amortized supersamplingACM Transactions on Graphics, 2009
- An improved illumination model for shaded display2005
- Multiscale structural similarity for image quality assessment2004