WARM-3D: A Weakly-Supervised Sim2Real Domain Adaptation Framework for Roadside Monocular 3D Object Detection
Preprint
Abstract
Existing roadside perception systems are limited by the absence of publicly available, large-scale, high-quality 3D datasets. Exploring the use of cost-effective, extensive synthetic datasets offers a viable solution to tackle this challenge and enhance the performance of roadside monocular 3D detection. In this study, we introduce the TUMTraf Synthetic Dataset, offering a diverse and substantial collection of high-quality 3D data to augment scarce real-world datasets. Besides, we present WARM-3D, a concise yet effective framework to aid the Sim2Real domain transfer for roadside monocular 3D detection. Our method leverages cheap synthetic datasets and 2D labels from an off-the-shelf 2D detector for weak supervision. We show that WARM-3D significantly enhances performance, achieving a +12.40% increase in mAP 3D over the baseline with only pseudo-2D supervision. With 2D GT as weak labels, WARM-3D even reaches performance close to the Oracle baseline. Moreover, WARM-3D improves the ability of 3D detectors to unseen sample recognition across various real-world environments, highlighting its potential for practical applications.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information2024
- TUMTraf V2X Cooperative Perception Dataset2024
- A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future OutlookIEEE Transactions on Intelligent Vehicles, 2024
- TUMTraf Intersection Dataset: All You Need for Urban 3D Camera-LiDAR Roadside Perception2023
- InfraDet3D: Multi-Modal 3D Object Detection based on Roadside Infrastructure Camera and LiDAR Sensors2023
- BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detection2023
- Unsupervised Domain Adaptation for Monocular 3D Object Detection via Self-training2022
- DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection2022
- MonoDTR: Monocular 3D Object Detection with Depth-Aware Transformer2022
- Weakly Supervised Adversarial Domain Adaptation for Semantic Segmentation in Urban ScenesIEEE Transactions on Image Processing, 2019