EndoLRMGS: Combining Large Reconstruction Modelling and Gaussian Splatting for Complete Endoscopic Scene Reconstruction
- 1 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 45 (9) , 1
Abstract
Reconstructing dynamic surgical scenes from endoscopic videos remains a fundamental challenge in robot-assisted surgery. Existing methods primarily focus on deformable tissues, overlooking the presence of articulated instruments. To bridge this gap, we present EndoLRMGS, the first unified framework capable of reconstructing both deformable tissue and articulated instruments in a modular approach from monocular video and depth priors. We introduce Frequency-Modulated Gaussian Splatting (FMGS), for deformable tissue reconstruction, which modulates the spatial frequency of Gaussian primitives according to the Nyquist-Shannon sampling theorem, improving the visual fidelity while maintaining robust geometric accuracy. For instrument reconstruction, we leverage the Large Reconstruction Model (LRM) to generate high-quality, watertight 3D models from single images, and introduce a novel Orthographic and Perspective joint Projection Optimization (OPjPO) module to recover metric scale and spatial alignment. Extensive experiments on public datasets demonstrate the effectiveness of EndoLRMGS, achieving PSNR values from 28.4981 to 38.4179, with Chamfer distance ranging from 1.43 to 4.71 mm in tissue reconstruction. For instrument reconstruction, EndoLRMGS achieves PSNR values ranging from 19.7341 to 22.4393 on left views and 17.7442 to 20.8436 on right views. In terms of spatial alignment accuracy, it attains IoU values between 71.56% and 85.82%, with Chamfer distance ranging from 12.59 to 17.38 mm. These results highlight EndoLRMGS as a powerful and versatile solution for accurate, complete, and photorealistic 3D reconstruction of surgical scenes. Code is available at: EndoLRMGS.Keywords
Funding Information
- UCL Research Excellence Scholarship
- Engineering and Physical Sciences Research Council (EP/S021930/1)
- Engineering and Physical Sciences Research Council (EP/Z534754/1)
- OASIS hub (UKRI145)
- NIHR UCLH BRC (NIHR203328)
- Department for Science, Innovation and Technology
- Royal Academy of Engineering
This publication has 35 references indexed in Scilit:
- A Review of 3D Reconstruction Techniques for Deformable Tissues in Robotic Surgery2025
- GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation2024
- Endo-4DGS: Endoscopic Monocular Scene Reconstruction with 4D Gaussian Splatting2024
- Deform3DGS: Flexible Deformation for Fast Surgical Scene Reconstruction with Gaussian Splatting2024
- LGM: Large Multi-view Gaussian Model for High-Resolution 3D Content Creation2024
- EndoSurf: Neural Surface Reconstruction of Deformable Tissues with Stereo Endoscope Videos2023
- Neural LerPlane Representations for Fast 4D Reconstruction of Deformable Tissues2023
- Deep Laparoscopic Stereo Matching with Transformers2022
- Neural Rendering for Stereo 3D Reconstruction of Deformable Tissues in Robotic Surgery2022
- Soft-Tissue Motion Tracking and Structure Estimation for Robotic Assisted MIS Procedures2005