Scalable Scene Flow From Point Clouds in the Real World
Open Access
- 31 December 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 1589-1596
Abstract
Autonomous vehicles operate in highly dynamic environments necessitating an accurate assessment of which aspects of a scene are moving and where they are moving to. A popular approach to 3D motion estimation, termed scene flow, is to employ 3D point cloud data from consecutive LiDAR scans, although such approaches have been limited by the small size of real-world, annotated LiDAR data. In this work, we introduce a new large-scale dataset for scene flow estimation derived from corresponding tracked 3D objects, which is $\sim 1,000\times$ larger than previous real-world datasets in terms of the number of annotated frames. We demonstrate how previous works were bounded by the amount of real LiDAR data available, suggesting that larger datasets are required to achieve state-of-the-art predictive performance. Furthermore, we show how previous heuristics such as down-sampling heavily degrade performance, motivating a new class of models that are tractable on the full point cloud. To address this issue, we introduce the FastFlow3D architecture which provides real time inference on the full point cloud. Additionally, we design human-interpretable metrics that better capture real world aspects by accounting for ego-motion and providing breakdowns per object type. We hope that this dataset may provide new opportunities for developing real world scene flow systems.Keywords
All Related Versions
Funding Information
- Google Brain
- Waymo LLC.
This publication has 42 references indexed in Scilit:
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Exploiting the Power of Stereo Confidences2013
- On Performance Analysis of Optical Flow Algorithms2012
- A Naturalistic Open Source Movie for Optical Flow Evaluation2012
- Joint Optimization for Object Class Segmentation and Dense Stereo ReconstructionInternational Journal of Computer Vision, 2011
- Ground Truth Evaluation of Stereo Algorithms for Real World Applications2011
- A Database and Evaluation Methodology for Optical FlowInternational Journal of Computer Vision, 2010
- Real-time KD-tree construction on graphics hardwareACM Transactions on Graphics, 2008
- Robotic Grasping of Novel Objects using VisionThe International Journal of Robotics Research, 2008
- Stanley: The robot that won the DARPA Grand ChallengeJournal of Field Robotics, 2006