Scalable Scene Flow from Point Clouds in the Real World
Preprint
Abstract
Autonomous vehicles operate in highly dynamic environments necessitating an accurate assessment of which aspects of a scene are moving and where they are moving to. A popular approach to 3D motion estimation -- termed scene flow -- is to employ 3D point cloud data from consecutive LiDAR scans, although such approaches have been limited by the small size of real-world, annotated LiDAR data. In this work, we introduce a new large scale benchmark for scene flow based on the Waymo Open Dataset. The dataset is $\sim$1,000$\times$ larger than previous real-world datasets in terms of the number of annotated frames and is derived from the corresponding tracked 3D objects. We demonstrate how previous works were bounded based on the amount of real LiDAR data available, suggesting that larger datasets are required to achieve state-of-the-art predictive performance. Furthermore, we show how previous heuristics for operating on point clouds such as artificial down-sampling heavily degrade performance, motivating a new class of models that are tractable on the full point cloud. To address this issue, we introduce the model architecture FastFlow3D that provides real time inference on the full point cloud. Finally, we demonstrate that this problem is amenable to techniques from semi-supervised learning by highlighting open problems for generalizing methods for predicting motion on unlabeled objects. We hope that this dataset may provide new opportunities for developing real world scene flow systems and motivate a new class of machine learning problems.
Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Any Motion Detector: Learning Class-agnostic Scene Dynamics from a Sequence of LiDAR Point Clouds2020
- PointPillars: Fast Encoders for Object Detection From Point Clouds2019
- Fast neighbor search by using revised k-d treeInformation Sciences, 2019
- Rigid scene flow for 3D LiDAR scans2016
- Weakly-and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image Segmentation2015
- Local Path Planning for Off-Road Autonomous Driving With Avoidance of Static ObstaclesIEEE Transactions on Intelligent Transportation Systems, 2012
- Joint Optimization for Object Class Segmentation and Dense Stereo ReconstructionInternational Journal of Computer Vision, 2011
- Real-time KD-tree construction on graphics hardwareACM Transactions on Graphics, 2008
- Stanley: The robot that won the DARPA Grand ChallengeJournal of Field Robotics, 2006
- A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence AlgorithmsInternational Journal of Computer Vision, 2002