PLOT: Pseudo-Labeling via Video Object Tracking for Scalable Monocular 3D Object Detection
Preprint
Abstract
Monocular 3D object detection (M3OD) has long faced challenges due to data scarcity caused by high annotation costs and inherent 2D-to-3D ambiguity. Although various weakly supervised methods and pseudo-labeling methods have been proposed to address these issues, they are mostly limited by domain-specific learning or rely solely on shape information from a single observation. In this paper, we propose a novel pseudo-labeling framework that uses only video data and is more robust to occlusion, without requiring a multi-view setup, additional sensors, camera poses, or domain-specific training. Specifically, we explore a technique for aggregating the pseudo-LiDARs of both static and dynamic objects across temporally adjacent frames using object point tracking, enabling 3D attribute extraction in scenarios where 3D data acquisition is infeasible. Extensive experiments demonstrate that our method ensures reliable accuracy and strong scalability, making it a practical and effective solution for M3OD.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Toward Unified 3D Object Detection via Algorithm and Data UnificationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2025
- Weakly Supervised Monocular 3D Object Detection by Spatial-Temporal View ConsistencyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
- VGGSfM: Visual Geometry Grounded Deep Structure from Motion2024
- UniDepth: Universal Monocular Metric Depth Estimation2024
- Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data2024
- MonoMAE: Enhancing Monocular 3D Detection through Depth-Aware Masked Autoencoders2024
- DIVOTrack: A Novel Dataset and Baseline Method for Cross-View Multi-Object Tracking in DIVerse Open ScenesInternational Journal of Computer Vision, 2023
- Open-Vocabulary Point-Cloud Object Detection without 3D Annotation2023
- Perspective Fields for Single Image Camera Calibration2023
- nuScenes: A Multimodal Dataset for Autonomous Driving2020