Gaussian Prompter: Linking 2D Prompts for 3D Gaussian Segmentation
- 5 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 47 (9) , 8165-8179
Abstract
Interactive 3D segmentation in radiance fields is crucial for advanced 3D scene understanding and manipulation. However, existing methods often struggle to achieve both volumetric completeness and segmentation accuracy, primarily because they fail to consider the critical links between 2D prompt-based segmentations across multiple views. Motivated by this gap, we introduce Gaussian Prompter, a novel approach specifically designed for 3D Gaussian Splatting. The core idea behind Gaussian Prompter is to seamlessly integrate a Gaussian-centric segmentation paradigm by effectively linking various 2D prompts from multi-view segmentations to ensure consistent 3D segmentation. To realize this, we employ two tailored approaches: GaussBlend and PinPrompt. GaussBlend aggregates multi-view 2D segmentation masks into a cohesive 3D segmentation, ensuring both accuracy and completeness. PinPrompt leverages high-confidence prompts from adjacent views to enhance segmentation precision further. Additionally, to address the lack of complex datasets in 3D segmentation, we introduce the SegMip-360 dataset, which includes over 350 precisely annotated masks across seven scenes. Extensive experiments demonstrate that the Gaussian Prompter significantly outperforms state-of-the-art methods in both segmentation accuracy and completeness. Our code and video demonstrations can be found at our repository and project page.Keywords
Funding Information
- Natural Science Foundation of Guangdong Province (2024B1515040010)
- NSFC Key project (U23A20391)
- China National Key R&D Program (2023YFE0202700, 2024YFB4709200)
- Key-Area Research and Development Program of Guangzhou City (2023B01J0022)
- Fundamental Research Funds for the Central Universities (2024ZYGXZR021)
- Guangdong Provincial Basic and Applied Basic Research Fund (2025A1515011884)
- Guangdong Natural Science Funds for Distinguished Young Scholars (2023B1515020097)
- National Research Foundation Singapore
- AI Singapore Programme (AISG3-GV-2023-011)
- Lee Kong Chian Fellowships
This publication has 63 references indexed in Scilit:
- Beat-It: Beat-Synchronized Multi-condition 3D Dance Generation2024
- Gaussian Grouping: Segment and Edit Anything in 3D Scenes2024
- Segment anything in medical imagesNature Communications, 2024
- Segment anything model for medical images?Medical Image Analysis, 2023
- Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantification2022
- Deep Hierarchical Semantic Segmentation2022
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- BiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation2018
- RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation2017
- Fully convolutional networks for semantic segmentation2015