nuScenes: A multimodal dataset for autonomous driving
Authors:
- Holger CaesarHCHolger Caesar
- Varun BankitiVBVarun Bankiti
- Alex H. LangALAlex H. Lang
- Sourabh VoraSVSourabh Vora
- Venice Erin LiongVLVenice Erin Liong
- Qiang XuQXQiang Xu
- Anush KrishnanAKAnush Krishnan
- Yu PanYPYu Pan
- Giancarlo BaldanGBGiancarlo Baldan
- Oscar BeijbomOBOscar Beijbom
Preprint
Abstract
Robust detection and tracking of objects is crucial for the deployment of autonomous vehicle technology. Image-based benchmark datasets have driven the development in computer vision tasks such as object detection, tracking and segmentation of agents in the environment. Most autonomous vehicles, however, carry a combination of cameras and range sensors such as lidar and radar. As machine learning based methods for detection and tracking become more prevalent, there is a need to train and evaluate such methods on datasets containing range sensor data along with images. In this work we present nuTonomy scenes (nuScenes), the first dataset to carry the full autonomous vehicle sensor suite: 6 cameras, 5 radars and 1 lidar, all with full 360 degree field of view. nuScenes comprises 1000 scenes, each 20s long and fully annotated with 3D bounding boxes for 23 classes and 8 attributes. It has 7x as many annotations and 100x as many images as the pioneering KITTI dataset. We also define a new metric for 3D detection which consolidates the multiple aspects of the detection task: classification, localization, size, orientation, velocity and attribute estimation. We provide careful dataset analysis as well as baseline performance for lidar and image based detection methods. Data, development kit, and more information are available at www.nuscenes.org.Keywords
All Related Versions
This publication has 48 references indexed in Scilit:
- NightOwls: A Pedestrians at Night Dataset2019
- Deep Continuous Fusion for Multi-sensor 3D Object Detection2018
- The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes2016
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Monocular 3D Object Detection for Autonomous Driving2016
- Are we ready for autonomous driving? The KITTI vision benchmark suite2012
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- A mobile vision system for robust multi-person tracking2008
- Segmentation and Recognition Using Structure from Motion Point Clouds2008
- Histograms of Oriented Gradients for Human Detection2005