Benchmarking Open-Access Building Footprints: A Multi-Dimensional Assessment with High-Fidelity References

Authors:
Open Access
Publisher website
Google Scholar
Abstract
The rapid proliferation of open-access vector building footprint datasets has created substantial opportunities for large-scale urban analysis, yet their reliability remains difficult to assess effectively and consistently. This challenge arises primarily from the scarcity of large-scale, high-fidelity reference benchmarks across diverse urban environments. To address these limitations, we construct a benchmark-grade reference dataset comprising over 200,000 manually annotated building footprints across 10 purposively selected cities across diverse global regions through an exhaustive curation campaign and strict manual-automated quality validation. Using this dataset, we develop a three-tier evaluation framework that assesses statistical consistency, object recovery and spatial accuracy, and morphology fidelity among matched building pairs. Evaluating 11 mainstream datasets reveals pervasive underestimation in both building area (average bias: −14.9%) and count (−34.0%). Average city-level intersection-over-union values range from 29.1% to 58.3%, indicating low-to-moderate spatial overlap. For matched building pairs, the datasets demonstrate relatively high geometric fidelity, with morphological similarity scores of 0.495 to 0.803. Pronounced regional performance variations indicate that no single dataset is universally optimal. Specifically, Microsoft Global ML Building Footprints offers the most balanced multi-city performance, Google Research Open Buildings is recommended for boundary shape fidelity, and regional datasets remain competitive for local analyses.
Funding Information
  • Open Project Funds for the Joint Laboratory of Spatial Intelligent Perception and Large Model Application (SIPLMA-2025-YB-10)
  • Key Laboratory of Smart Earth (KF2023ZD02-01)

This publication has 55 references indexed in Scilit: