The Oxford Spires Dataset: Benchmarking large-scale LiDAR-visual localisation, reconstruction and radiance field methods
Top Cited Papers
Open Access
- 20 September 2025
- journal article
- Published by SAGE Publications in The International Journal of Robotics Research
- Vol. 45 (6) , 839-857
Abstract
This paper introduces a large-scale multi-modal dataset captured in and around well-known landmarks in Oxford using a custom-built multi-sensor perception unit as well as a millimetre-accurate map from a Terrestrial LiDAR Scanner (TLS). The perception unit includes three synchronised global shutter colour cameras, an automotive 3D LiDAR scanner, and an inertial sensor – all precisely calibrated. We also establish benchmarks for tasks involving localisation, reconstruction, and novel-view synthesis, which enable the evaluation of Simultaneous Localisation and Mapping (SLAM) methods, Structure-from-Motion (SfM) and Multi-view Stereo (MVS) methods as well as radiance field methods such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting. To evaluate 3D reconstruction, the TLS 3D models are used as ground truth. Localisation ground truth is computed by registering the mobile LiDAR scans to the TLS 3D models. Radiance field methods are evaluated not only with poses sampled from the input trajectory, but also from viewpoints that are from trajectories which are distant from the training poses. Our evaluation demonstrates a key limitation of state-of-the-art radiance field methods: we show that they tend to overfit to the training poses/images and do not generalise well to out-of-sequence poses. They also underperform in 3D reconstruction compared to MVS systems using the same visual inputs. Our dataset and benchmarks are intended to facilitate better integration of radiance field methods and SLAM systems. The raw and processed data, along with software for parsing and evaluation, can be accessed at https://dynamic.robots.ox.ac.uk/datasets/oxford-spires/ .Keywords
All Related Versions
Funding Information
- Valencian Community Government and the European Union (CIBEST/2023/44)
- Valencian Community Government and the European Union (PROMETEO/2021/075)
- Korea government (RS-2024-00461409)
This publication has 61 references indexed in Scilit:
- LaMAR: Benchmarking Localization and Mapping for Augmented Reality2022
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018
- Matterport3D: Learning from RGB-D Data in Indoor Environments2017
- Fast Fourier Color Constancy2017
- A Multi-view Stereo Benchmark with High-Resolution Images and Multi-camera Videos2017
- ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes2017
- Structure-from-Motion Revisited2016
- Large-Scale Data for Multiple-View StereopsisInternational Journal of Computer Vision, 2016
- OctoMap: an efficient probabilistic 3D mapping framework based on octreesAutonomous Robots, 2013
- A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms2006