STRMs: Spatial Temporal Reasoning Models for Vision-Based Localization Rivaling GPS Precision
Preprint
Abstract
This paper explores vision-based localization through a biologically-inspired approach that mirrors how humans and animals link views or perspectives when navigating their world. We introduce two sequential generative models, VAE-RNN and VAE-Transformer, which transform first-person perspective (FPP) observations into global map perspective (GMP) representations and precise geographical coordinates. Unlike retrieval-based methods, our approach frames localization as a generative task, learning direct mappings between perspectives without relying on dense satellite image databases. We evaluate these models across two real-world environments: a university campus navigated by a Jackal robot and an urban downtown area navigated by a Tesla sedan. The VAE-Transformer achieves impressive precision, with median deviations of 2.29m (1.37% of environment size) and 4.45m (0.35% of environment size) respectively, outperforming both VAE-RNN and prior cross-view geo-localization approaches. Our comprehensive Localization Performance Characteristics (LPC) analysis demonstrates superior performance with the VAE-Transformer achieving an AUC of 0.777 compared to 0.295 for VIGOR 200 and 0.225 for TransGeo, establishing a new state-of-the-art in vision-based localization. In some scenarios, our vision-based system rivals commercial smartphone GPS accuracy (AUC of 0.797) while requiring 5x less GPU memory and delivering 3x faster inference than existing methods in cross-view geo-localization. These results demonstrate that models inspired by biological spatial navigation can effectively memorize complex, dynamic environments and provide precise localization with minimal computational resources.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- SCE-SLAM: a real-time semantic RGBD SLAM system in dynamic scenes based on spatial coordinate errorMeasurement Science and Technology, 2023
- Linking global top-down views to first-person views in the brainProceedings of the National Academy of Sciences, 2022
- ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAMIEEE Transactions on Robotics, 2021
- Cross-view image synthesis using geometry-guided conditional GANsComputer Vision and Image Understanding, 2019
- ORB-SLAM: A Versatile and Accurate Monocular SLAM SystemIEEE Transactions on Robotics, 2015
- Boundary Vector Cells in the Subiculum of the Hippocampal FormationThe Journal of Neuroscience, 2009
- Representation of Geometric Borders in the Entorhinal CortexScience, 2008
- Hippocampal cellular and network activity in freely moving echolocating batsNature Neuroscience, 2007