Surgical neural radiance fields from one image
Authors:
- 19 June 2025
- journal article
- research article
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 20 (11) , 2329-2333
Abstract
Purpose Neural radiance fields (NeRF) offer exceptional capabilities for 3D reconstruction and view synthesis, yet their reliance on extensive multi-view data limits their application in surgical intraoperative settings where only limited data are available. This work addresses this challenge by leveraging a single intraoperative image and preoperative data to train NeRF efficiently for surgical scenarios. Methods We leverage preoperative MRI data to define the set of camera viewpoints and images needed for robust and unobstructed training. Intraoperatively, the appearance of the surgical image is transferred to the pre-constructed training set through neural style transfer, specifically combining WTC2 and STROTSS to prevent over-stylization. This process enables the creation of a dataset for instant and fast single-image NeRF training. Results The method is evaluated with four clinical neurosurgical cases. Quantitative comparisons to NeRF models trained on real surgical microscope images demonstrate strong synthesis agreement, with similarity metrics indicating high reconstruction fidelity and stylistic alignment. When compared with ground truth, our method demonstrates high structural similarity, confirming good reconstruction quality and texture preservation. Conclusion Our approach demonstrates the feasibility of single-image NeRF training in surgical settings, overcoming the limitations of traditional multi-view methods. By eliminating the dependency on a large multi-view dataset, our method offers a faster, more adaptable solution for generating accurate 3D reconstructions in real-time surgical scenarios.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Intraoperative Registration by Cross-Modal Inverse Neural Rendering2024
- SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image2022
- Neural Rendering for Stereo 3D Reconstruction of Deformable Tissues in Robotic Surgery2022
- Instant neural graphics primitives with a multiresolution hash encodingACM Transactions on Graphics, 2022
- NeRFCommunications of the ACM, 2021
- Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis2021
- pixelNeRF: Neural Radiance Fields from One or Few Images2021
- Photorealistic Style Transfer via Wavelet Transforms2019
- Style Transfer by Relaxed Optimal Transport and Self-Similarity2019
- ObjectNet3D: A Large Scale Database for 3D Object Recognition2016