Multispectral-NeRF: A Multispectral Modeling Approach Based on Neural Radiance Fields

Authors:
Open Access
Publisher website
Google Scholar
Abstract
3D reconstruction technology generates three-dimensional representations of real-world objects, scenes, or environments using sensor data such as 2D images, with extensive applications in robotics, autonomous vehicles, and virtual reality systems. Traditional 3D reconstruction techniques based on 2D images typically rely on RGB spectral information. With advances in sensor technology, additional spectral bands beyond RGB have been increasingly incorporated into 3D reconstruction workflows. Existing methods that integrate these expanded spectral data often suffer from expensive scheme prices, low accuracy, and poor geometric features. Three-dimensional reconstruction based on NeRF can effectively address the various issues in current multispectral 3D reconstruction methods, producing high-precision and high-quality reconstruction results. However, currently, NeRF and some improved models such as NeRFacto are trained on three-band data and cannot take into account the multi-band information. To address this problem, we propose Multispectral-NeRF—an enhanced neural architecture derived from NeRF that can effectively integrate multispectral information. Our technical contributions comprise threefold modifications: Expanding hidden layer dimensionality to accommodate 6-band spectral inputs; redesigning residual functions to optimize spectral discrepancy calculations between reconstructed and reference images; and adapting data compression modules to address the increased bit-depth requirements of multispectral imagery. Experimental results confirm that Multispectral-NeRF successfully processes multi-band spectral features while accurately preserving the original scenes’ spectral characteristics.
Funding Information
  • National Natural Science Foundation of China (42330108)

This publication has 47 references indexed in Scilit: