GaussianCut: Interactive segmentation via graph cut for 3D Gaussian Splatting
Authors:
- Umangi JainUJUmangi Jain
- Ashkan MirzaeiAMAshkan Mirzaei
- Igor GilitschenskiIGIgor Gilitschenski
Preprint
Abstract
We introduce GaussianCut, a new method for interactive multiview segmentation of scenes represented as 3D Gaussians. Our approach allows for selecting the objects to be segmented by interacting with a single view. It accepts intuitive user input, such as point clicks, coarse scribbles, or text. Using 3D Gaussian Splatting (3DGS) as the underlying scene representation simplifies the extraction of objects of interest which are considered to be a subset of the scene's Gaussians. Our key idea is to represent the scene as a graph and use the graph-cut algorithm to minimize an energy function to effectively partition the Gaussians into foreground and background. To achieve this, we construct a graph based on scene Gaussians and devise a segmentation-aligned energy function on the graph to combine user inputs with scene properties. To obtain an initial coarse segmentation, we leverage 2D image/video segmentation models and further refine these coarse estimates using our graph construction. Our empirical evaluations show the adaptability of GaussianCut across a diverse set of scenes. GaussianCut achieves competitive performance with state-of-the-art approaches for 3D segmentation without requiring any additional segmentation-aware training.Keywords
All Related Versions
This publication has 33 references indexed in Scilit:
- Contrastive Gaussian Clustering for Weakly Supervised 3D Scene Segmentation2024
- SAM-Guided Graph Cut for 3D Instance Segmentation2024
- Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection2024
- Gaussian Grouping: Segment and Edit Anything in 3D Scenes2024
- Open-World Semantic Segmentation via Contrasting and Clustering Vision-Language Embedding2022
- Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representations2022
- Dense graph convolutional neural networks on 3D meshes for 3D object segmentation and classificationImage and Vision Computing, 2021
- V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation2016
- A survey of graph theoretical approaches to image segmentationPattern Recognition, 2013
- Fast approximate energy minimization via graph cutsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2001