Multi-Layer Gaussian Splatting for Immersive Anatomy Visualization
Preprint
Abstract
In medical image visualization, path tracing of volumetric medical data like CT scans produces lifelike three-dimensional visualizations. Immersive VR displays can further enhance the understanding of complex anatomies. Going beyond the diagnostic quality of traditional 2D slices, they enable interactive 3D evaluation of anatomies, supporting medical education and planning. Rendering high-quality visualizations in real-time, however, is computationally intensive and impractical for compute-constrained devices like mobile headsets. We propose a novel approach utilizing GS to create an efficient but static intermediate representation of CT scans. We introduce a layered GS representation, incrementally including different anatomical structures while minimizing overlap and extending the GS training to remove inactive Gaussians. We further compress the created model with clustering across layers. Our approach achieves interactive frame rates while preserving anatomical structures, with quality adjustable to the target hardware. Compared to standard GS, our representation retains some of the explorative qualities initially enabled by immersive path tracing. Selective activation and clipping of layers are possible at rendering time, adding a degree of interactivity to otherwise static GS models. This could enable scenarios where high computational demands would otherwise prohibit using path-traced medical volumes.Keywords
All Related Versions
This publication has 52 references indexed in Scilit:
- Compact 3D Scene Representation via Self-Organizing Gaussian Grids2024
- SurgicalGaussian: Deformable 3D Gaussians for High-Fidelity Surgical Scene Reconstruction2024
- LGS: A Light-Weight 4D Gaussian Splatting for Efficient Surgical Scene Reconstruction2024
- HAC: Hash-Grid Assisted Context for 3D Gaussian Splatting Compression2024
- Efficient Unbiased Volume Path Tracing on the GPU2021
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- Cinematic Rendering in Anatomy: A Crossover Study Comparing a Novel 3D Reconstruction Technique to Conventional Computed TomographyAnatomical Sciences Education, 2020
- The application of cinematic rendering to CT evaluation of upper tract urothelial tumors: principles and practiceAbdominal Radiology, 2019
- Volume Visualization: A Technical Overview with a Focus on Medical ApplicationsJournal of Digital Imaging, 2010
- Representative Views and Paths for Volume Models2008