LAENeRF: Local Appearance Editing for Neural Radiance Fields
Authors:
- 16 June 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4969-4978
Abstract
Due to the omnipresence of Neural Radiance Fields (NeRFs), the interest towards editable implicit 3D repre-sentations has surged over the last years. However, editing implicit or hybrid representations as used for NeRFs is difficult due to the entanglement of appearance and geom-etry encoded in the model parameters. Despite these chal-lenges, recent research has shown first promising steps to-wards photorealistic and non-photorealistic appearance ed-its. The main open issues of related work include limited in-teractivity, a lack of support for local edits and large memory requirements, rendering them less useful in practice. We address these limitations with LAENeRF, a unified framework for photorealistic and non-photorealistic appearance editing of NeRFs. To tackle local editing, we leverage a voxel grid as starting point for region selection. We learn a mapping from expected ray terminations to final output color, which can optionally be supervised by a style loss, resulting in a framework which can perform photorealistic and non-photorealistic appearance editing of selected re-gions. Relying on a single point per ray for our mapping, we limit memory requirements and enable fast optimization. To guarantee interactivity, we compose the output color using a set of learned, modifiable base colors, composed with additive layer mixing. selection. Compared to concurrent work, LAENeRF enables recoloring and stylization while keeping processing time low. Furthermore, we demonstrate that our approach surpasses baseline methods both quanti-tatively and qualitatively.Keywords
All Related Versions
This publication has 43 references indexed in Scilit:
- Unified Implicit Neural Stylization2022
- AdaNeRF: Adaptive Sampling for Real-Time Rendering of Neural Radiance Fields2022
- ARF: Artistic Radiance Fields2022
- Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields2022
- D-NeRF: Neural Radiance Fields for Dynamic Scenes2021
- NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections2021
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- Controlling Perceptual Factors in Neural Style Transfer2017
- Image Style Transfer Using Convolutional Neural Networks2016
- Perceptual Losses for Real-Time Style Transfer and Super-Resolution2016