MAPEX: Modality-Aware Pruning of Experts for Remote Sensing Foundation Models
Preprint
Abstract
Remote sensing data is commonly used for tasks such as flood mapping, wildfire detection, or land-use studies. For each task, scientists carefully choose appropriate modalities or leverage data from purpose-built instruments. Recent work on remote sensing foundation models pre-trains computer vision models on large amounts of remote sensing data. These large-scale models tend to focus on specific modalities, often optical RGB or multispectral data. For many important applications, this introduces a mismatch between the application modalities and the pre-training data. Moreover, the large size of foundation models makes them expensive and difficult to fine-tune on typically small datasets for each task. We address this mismatch with MAPEX, a remote sensing foundation model based on mixture-of-modality experts. MAPEX is pre-trained on multi-modal remote sensing data with a novel modality-conditioned token routing mechanism that elicits modality-specific experts. To apply the model on a specific task, we propose a modality aware pruning technique, which only retains experts specialized for the task modalities. This yields efficient modality-specific models while simplifying fine-tuning and deployment for the modalities of interest. We experimentally validate MAPEX on diverse remote sensing datasets and show strong performance compared to fully supervised training and state-of-the-art remote sensing foundation models. Code is available at https://github.com/HSG-AIML/MAPEX.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Foundation Models for Remote Sensing and Earth Observation: A surveyIEEE Geoscience and Remote Sensing Magazine, 2025
- Vision Foundation Models in Remote Sensing: A surveyIEEE Geoscience and Remote Sensing Magazine, 2025
- MMEarth: Exploring Multi-modal Pretext Tasks for Geospatial Representation Learning2024
- RingMo-SAM: A Foundation Model for Segment Anything in Multimodal Remote-Sensing ImagesIEEE Transactions on Geoscience and Remote Sensing, 2023
- BigEarthNet-MM: A Large-Scale, Multimodal, Multilabel Benchmark Archive for Remote Sensing Image Classification and Retrieval [Software and Data Sets]IEEE Geoscience and Remote Sensing Magazine, 2021
- Multimodal Machine Learning: A Survey and TaxonomyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- Deep Learning in Remote Sensing: A Comprehensive Review and List of ResourcesIEEE Geoscience and Remote Sensing Magazine, 2017
- Remote sensing platforms and sensors: A surveyISPRS Journal of Photogrammetry and Remote Sensing, 2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Sentinel-2: ESA's Optical High-Resolution Mission for GMES Operational ServicesRemote Sensing of Environment, 2012