Joint Learning for Feature Reconstruction and Prediction in Agricultural Semantic Segmentation From Incomplete Satellite Image Time Series
- 29 December 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 64 (01962892) , 1-23
Abstract
Satellite Image Time Series (SITS) is crucial for semantic segmentation (SS) in agriculture applications. However, cloud contamination introduces time gaps in SITS, disrupting temporal dependencies and causing feature shifts, which decreases the performance of models pre-trained on complete SITS. Existing methods typically address this challenge by reconstructing the entire SITS before prediction or using data augmentation to simulate missing data. Yet, full reconstruction may introduce noise and redundancy, whereas the data augmentation model can only handle limited missing patterns, leading to poor generalization. We propose a joint learning framework with feature reconstruction and prediction to address incomplete SITS more effectively. Specifically, we simulate temporal missing scenarios using temporal masks. The two tasks are guided by both ground-truth labels and the teacher model trained on complete SITS. The prediction task constrains the model from selectively reconstructing temporal features, ensuring they align with the teacher’s complete representations and are beneficial for prediction. It reduces unnecessary reconstruction and limits noise propagation. By integrating reconstructed features into the prediction task, the model avoids learning shortcuts and maintains its ability to handle varied missing patterns and complete SITS. The experiments show strong generalization capabilities in multiple aspects: (1) Different SS tasks, including cropland extraction and crop classification; (2) Different study areas, including Hunan Province,Western France, and Catalonia; (3) Different sensors, primarily Sentinel-2 and PlanetScope; (4) Different numbers of time gaps, while retaining the model’s performance on complete SITS; (5) Different model architectures, including seven typical backbones. The source codes are available at: https://github.com/wangyuze-csu/Joint_FRP.Keywords
Funding Information
- National Science Fund for Distinguished Young Scholars (2022JJ10072)
- National Natural Science Foundation of China (42471419)
- National Natural Science Foundation of China (42171376)
This publication has 61 references indexed in Scilit:
- Auxiliary data-driven imputation of incomplete remote-sensing time series for unknown crop types mappingComputers and Electronics in Agriculture, 2025
- Deep Learning-Based Image and Video Inpainting: A SurveyInternational Journal of Computer Vision, 2024
- Data Augmentation techniques in time series domain: a survey and taxonomyNeural Computing and Applications, 2023
- SAITS: Self-attention-based imputation for time seriesExpert Systems with Applications, 2023
- Evaluation of a deep-learning model for multispectral remote sensing of land use and crop classificationThe Crop Journal, 2022
- SAR data applications in earth observation: An overviewExpert Systems with Applications, 2022
- A Review on SAR Image and its DespecklingArchives of Computational Methods in Engineering, 2021
- A Study on Catastrophic Forgetting in Deep LSTM Networks2019
- Changes in Global Cloud Cover Based on Remote Sensing Data from 2003 to 2012Chinese Geographical Science, 2019
- Climate and management interaction cause diverse crop phenology trendsAgricultural and Forest Meteorology, 2017