LABELMAKER: Automatic Semantic Label Generation from RGB-D Trajectories
Authors:
- 18 March 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 334-343
Abstract
Semantic annotations are indispensable to train or evaluate perception models, yet very costly to acquire. This work introduces a fully automated 2D/3D labeling framework that, without any human intervention, can generate labels for RGB-D scans at equal (or better) level of accuracy than comparable manually annotated datasets such as ScanNet. Our approach is based on an ensemble of state-of-the-art segmentation models and 3D lifting through neural rendering. We demonstrate the effectiveness of our LabelMaker pipeline by generating significantly better labels for the ScanNet datasets and automatically labelling the previously unlabeled ARKitScenes dataset. Code and models are available at labelmaker.org labelmaker.org.Keywords
All Related Versions
Funding Information
- National Science Foundation
- Scan
This publication has 27 references indexed in Scilit:
- 4D-StOP: Panoptic Segmentation of 4D LiDAR Using Spatio-Temporal Object Proposal Generation and Aggregation2023
- Language-Grounded Indoor 3D Semantic Segmentation in the Wild2022
- Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation using Bounding Boxes2022
- Mix3D: Out-of-Context Data Augmentation for 3D Scenes2021
- COCO-Stuff: Thing and Stuff Classes in Context2018
- Matterport3D: Learning from RGB-D Data in Indoor Environments2017
- ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes2017
- Scene Parsing through ADE20K Dataset2017
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Indoor Segmentation and Support Inference from RGBD Images2012