MultiPlaneNeRF: Neural Radiance Field with Non-Trainable Representation
Preprint
Abstract
NeRF is a popular model that efficiently represents 3D objects from 2D images. However, vanilla NeRF has a few important limitations. NeRF must be trained on each object separately. The training time is long since we encode the object's shape and color in neural network weights. Moreover, NeRF does not generalize well to unseen data. In this paper, we present MultiPlaneNeRF -- a first model that simultaneously solves all the above problems. Our model works directly on 2D images. We project 3D points on 2D images to produce non-trainable representations. The projection step is not parametrized, and a very shallow decoder can efficiently process the representation. Using existing images as part of NeRF can significantly reduce the number of parameters since we train only a small implicit decoder. Furthermore, we can train MultiPlaneNeRF on a large data set and force our implicit decoder to generalize across many objects. Consequently, we can only replace the 2D images (without additional training) to produce a NeRF representation of the new object. In the experimental section, we demonstrate that MultiPlaneNeRF achieves comparable results to state-of-the-art models for synthesizing new views and has generalization properties.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Points2NeRF: Generating Neural Radiance Fields from 3D point cloudPattern Recognition Letters, 2024
- Wasserstein distributional harvesting for highly dense 3D point cloudsPattern Recognition, 2022
- Instant neural graphics primitives with a multiresolution hash encodingACM Transactions on Graphics, 2022
- VFMVAC: View-filtering-based multi-view aggregating convolution for 3D shape recognition and retrievalPattern Recognition, 2022
- Continuous conditional random field convolution for point cloud segmentationPattern Recognition, 2022
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- Learning a model of facial shape and expression from 4D scansACM Transactions on Graphics, 2017
- Ray tracing volume densitiesACM SIGGRAPH Computer Graphics, 1984