Hallucinated Neural Radiance Fields in the Wild
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 12933-12942
Abstract
Neural Radiance Fields (NeRF) has recently gained popularity for its impressive novel view synthesis ability. This paper studies the problem of hallucinated NeRF: i.e., recovering a realistic NeRF at a different time of day from a group of tourism images. Existing solutions adopt NeRF with a controllable appearance embedding to render novel views under various conditions, but they cannot render view-consistent images with an unseen appearance. To solve this problem, we present an end-to-end framework for constructing a hallucinated NeRF, dubbed as Ha-NeRF. Specifically, we propose an appearance hallucination module to handle time-varying appearances and transfer them to novel views. Considering the complex occlusions of tourism images, we introduce an anti-occlusion module to decompose the static subjects for visibility accurately. Experimental results on synthetic data and real tourism photo collections demonstrate that our method can hallucinate the desired appearances and render occlusion-free images from different views. The project and supplementary materials are available at https://rover-xingyu.github.io/Ha-NeRF/.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields2021
- Stable View Synthesis2021
- State of the Art on Neural RenderingComputer Graphics Forum, 2020
- A Bayesian Approach for Selective Image-Based Rendering Using Superpixels2015
- Reasoning about Photo Collections using Models of Outdoor Illumination2014
- Unstructured Light FieldsComputer Graphics Forum, 2012
- Light field rendering1996
- Plenoptic modeling1995