Multi-Teacher Knowledge Distillation for Efficient Satellite-Based Change Detection in Remote Sensing: Advancing Edge Computing for Real-Time Environmental Hazard Monitoring in LEO Networks
Preprint
- 3 February 2026
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
The rapid growth of low Earth orbit satellite constellations has revolutionized Earth observation, yet processing high-resolution remote sensing data on resource-constrained hardware remains challenging for time-sensitive natural disaster management applications. We introduce the Multi-Teacher Knowledge Distillation Change Detection (MTKDCD) framework, which leverages a Kubernetes orchestrated distributed edge computing system model to enable distributed AI intelligence across LEO satellite networks, achieving low-latency, resilient on-board change detection for real-time responses to hazards such as earthquakes, floods, and wildfires. The teacher model features a Contextual Adaptive Dual Attention Fusion module with linear self-attention for capturing global and local details in imagery. A multi-teacher ensemble, trained sequentially with bootstrap sampling and Jensen-Shannon divergence-based diversity loss on resource-constrained worker nodes and distilled in parallel over inter-satellite links, transfers robust knowledge to a lightweight student model suited for distributed on-board deployment with local reference feature caching. A compound loss combining Binary Cross-Entropy, Generalized Dice Loss, and novel Boundary Density Loss enhances edge delineation and robustness to annotation imperfections in remote sensing data. The distilled student achieves competitive accuracy with 1.51M parameters and 3.60B FLOPs, reducing parameters by 95.41% and complexity by 90.95% versus the ensemble teacher. Experiments on three benchmarks show teacher F1-scores of 92.00%, 93.15%, and 97.13%, with ablations confirming effectiveness for real-time hazard monitoring.Keywords
This publication has 46 references indexed in Scilit:
- Optimizing deep learning models for on-orbit deployment through neural architecture searchScientific Reports, 2025
- Applications of knowledge distillation in remote sensing: A surveyInformation Fusion, 2024
- Few-shot satellite image classification for bringing deep learning on board OPS-SATExpert Systems with Applications, 2024
- Opportunities and challenges of on-board AI-based image recognition for small satellite Earth observation missionsAdvances In Space Research, 2024
- A dual-branch siamese spatial-spectral transformer attention network for Hyperspectral Image Change DetectionExpert Systems with Applications, 2023
- TINYCD: a (not so) deep learning model for change detectionNeural Computing and Applications, 2022
- Teacher-student collaborative knowledge distillation for image classificationApplied Intelligence, 2022
- A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing imagesISPRS Journal of Photogrammetry and Remote Sensing, 2020
- Improving the Efficiency of Encoder-Decoder Architecture for Pixel-Level Crack DetectionIEEE Access, 2019
- Divergence measures based on the Shannon entropyIEEE Transactions on Information Theory, 1991