NeuS-PIR: Learning Relightable Neural Surface using Pre-Integrated Rendering
Preprint
Abstract
This paper presents a method, namely NeuS-PIR, for recovering relightable neural surfaces using pre-integrated rendering from multi-view images or video. Unlike methods based on NeRF and discrete meshes, our method utilizes implicit neural surface representation to reconstruct high-quality geometry, which facilitates the factorization of the radiance field into two components: a spatially varying material field and an all-frequency lighting representation. This factorization, jointly optimized using an adapted differentiable pre-integrated rendering framework with material encoding regularization, in turn addresses the ambiguity of geometry reconstruction and leads to better disentanglement and refinement of each scene property. Additionally, we introduced a method to distil indirect illumination fields from the learned representations, further recovering the complex illumination effect like inter-reflection. Consequently, our method enables advanced applications such as relighting, which can be seamlessly integrated with modern graphics engines. Qualitative and quantitative experiments have shown that NeuS-PIR outperforms existing methods across various tasks on both synthetic and real datasets. Source code is available at https://github.com/Sheldonmao/NeuSPIRKeywords
All Related Versions
This publication has 16 references indexed in Scilit:
- GIR: 3D Gaussian Inverse Rendering for Relightable Scene FactorizationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2025
- Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing2024
- Neural Reflectance Decomposition Under Dynamic Point LightIEEE Transactions on Circuits and Systems for Video Technology, 2023
- Multiview Textured Mesh Recovery by Differentiable RenderingIEEE Transactions on Circuits and Systems for Video Technology, 2022
- Uncertainty Guided Multi-View Stereo Network for Depth EstimationIEEE Transactions on Circuits and Systems for Video Technology, 2022
- Illumination Unification for Person Re-IdentificationIEEE Transactions on Circuits and Systems for Video Technology, 2022
- The Farther the Better: Balanced Stereo Matching via Depth-Based Sampling and Adaptive Feature RefinementIEEE Transactions on Circuits and Systems for Video Technology, 2021
- Adaptive Disparity Candidates Prediction Network for Efficient Real-Time Stereo MatchingIEEE Transactions on Circuits and Systems for Video Technology, 2021
- A High-Throughput Depth Estimation Processor for Accurate Semiglobal Stereo Matching Using Pipelined Inter-Pixel AggregationIEEE Transactions on Circuits and Systems for Video Technology, 2021
- Canonical Illumination Decomposition and Its ApplicationsIEEE Transactions on Circuits and Systems for Video Technology, 2019