MMEarth-Bench: Global Model Adaptation via Multimodal Test-Time Training
Authors:
Preprint
Abstract
Recent research in geospatial machine learning has demonstrated that models pretrained with self-supervised learning on Earth observation data can perform well on downstream tasks with limited training data. However, most of the existing geospatial benchmark datasets have few data modalities and poor global representation, limiting the ability to evaluate multimodal pretrained models at global scales. To fill this gap, we introduce MMEarth-Bench, a collection of five new multimodal environmental tasks with 12 modalities, globally distributed data, and both in- and out-of-distribution test splits. We benchmark a diverse set of pretrained models and find that while (multimodal) pretraining tends to improve model robustness in limited data settings, geographic generalization abilities remain poor. In order to facilitate model adaptation to new downstream tasks and geographic domains, we propose a model-agnostic method for test-time training with multimodal reconstruction (TTT-MMR) that uses all the modalities available at test time as auxiliary tasks, regardless of whether a pretrained model accepts them as input. Our method improves model performance on both the random and geographic test splits, and geographic batching leads to a good trade-off between regularization and specialization during TTT. Our dataset, code, and visualization tool are linked from the project page at lgordon99.github.io/mmearth-bench.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- Spatial occurrence records and distributions of tropical Asian butterfliesScientific Data, 2025
- Terramesh: A Planetary Mosaic of Multimodal Earth Observation Data2025
- AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalities2025
- MMEarth: Exploring Multi-modal Pretext Tasks for Geospatial Representation Learning2024
- Dynamic World, Near real-time global 10 m land use land cover mappingScientific Data, 2022
- MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentation2022
- Multiple soil map comparison highlights challenges for predicting topsoil organic carbon concentration at national scaleScientific Reports, 2022
- When Does Self-supervision Improve Few-Shot Learning?2020
- Functional Map of the World2018
- Google Earth Engine: Planetary-scale geospatial analysis for everyoneRemote Sensing of Environment, 2017