Semantic Is Enough: Only Semantic Information For NeRF Reconstruction
Authors:
- 13 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 906-912
Abstract
Recent research that combines implicit 3D repre-sentation with semantic information, like Semantic-NeRF, has proven that NeRF model could perform excellently in rendering 3D structures with semantic labels. This research aims to extend the Semantic Neural Radiance Fields (Semantic-NeRF) model by focusing solely on semantic output and removing the RGB output component. We reformulate the model and its training procedure to leverage only the cross-entropy loss between the model's semantic output and the ground truth semantic images, removing the colour data traditionally used in the original Semantic-NeRF approach. We then conduct a series of identical experiments using the original and the modified Semantic-NeRF model. Our primary objective is to obverse the impact of this modification on the model's performance by Semantic-NeRF, focusing on tasks such as scene understanding, object detection, and segmentation. The results offer valuable insights into the new way of rendering the scenes and provide an avenue for further research and development in semantic-focused 3D scene understanding,Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- NeRFCommunications of the ACM, 2021
- PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things2019
- Habitat: A Platform for Embodied AI Research2019
- MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects2018
- Fusion++: Volumetric Object-Level SLAM2018
- 3D-RCNN: Instance-Level 3D Object Reconstruction via Render-and-Compare2018
- Meaningful maps with object-oriented semantic mapping2017
- SemanticFusion: Dense 3D semantic mapping with convolutional neural networks2017
- Dense 3D semantic mapping of indoor scenes from RGB-D images2014
- Indoor Segmentation and Support Inference from RGBD Images2012