Far3Det: Towards Far-Field 3D Detection
Authors:
- Shubham GuptaSGShubham Gupta
- Jeet KanjaniJKJeet Kanjani
- Mengtian LiMLMengtian Li
Preprint
Abstract
We focus on the task of far-field 3D detection (Far3Det) of objects beyond a certain distance from an observer, e.g., $>$50m. Far3Det is particularly important for autonomous vehicles (AVs) operating at highway speeds, which require detections of far-field obstacles to ensure sufficient braking distances. However, contemporary AV benchmarks such as nuScenes underemphasize this problem because they evaluate performance only up to a certain distance (50m). One reason is that obtaining far-field 3D annotations is difficult, particularly for lidar sensors that produce very few point returns for far-away objects. Indeed, we find that almost 50% of far-field objects (beyond 50m) contain zero lidar points. Secondly, current metrics for 3D detection employ a "one-size-fits-all" philosophy, using the same tolerance thresholds for near and far objects, inconsistent with tolerances for both human vision and stereo disparities. Both factors lead to an incomplete analysis of the Far3Det task. For example, while conventional wisdom tells us that high-resolution RGB sensors should be vital for 3D detection of far-away objects, lidar-based methods still rank higher compared to RGB counterparts on the current benchmark leaderboards. As a first step towards a Far3Det benchmark, we develop a method to find well-annotated scenes from the nuScenes dataset and derive a well-annotated far-field validation set. We also propose a Far3Det evaluation protocol and explore various 3D detection methods for Far3Det. Our result convincingly justifies the long-held conventional wisdom that high-resolution RGB improves 3D detection in the far-field. We further propose a simple yet effective method that fuses detections from RGB and lidar detectors based on non-maximum suppression, which remarkably outperforms state-of-the-art 3D detectors in the far-field.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- LET-3D-AP: Longitudinal Error Tolerant 3D Average Precision for Camera-Only 3D Detection2024
- Center-based 3D Object Detection and Tracking2021
- SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds2020
- Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection2019
- Joint 3D Proposal Generation and Object Detection from View Aggregation2018
- Efficiently Scaling up Crowdsourced Video AnnotationInternational Journal of Computer Vision, 2012
- Traffic: Why We Drive the Way We Do (and What It Says about Us)Traffic: Why We Drive the Way We Do (and What It Says about Us)Alfred A. Knopf978-0-307-26478-7Leadership and Management in Engineering, 2010
- Observers cannot accurately estimate the speed of an approaching object in flightVision Research, 2009
- Human observers are biased in judging the angular approach of a projectileVision Research, 2004
- Tools and techniques for video performance evaluation2002