Lilun Deng

Multi-temporal remote sensing image cloud removal aims to reconstruct land surface information in regions obscured by clouds and their shadows, thereby mitigating a major constraint on the application of remote sensing imagery. However, existing multi-temporal deep learning methods for cloud removal often fail to model complex spatio-temporal dynamics, leading to suboptimal performance. To address this challenge, we propose a novel framework for multi-temporal cloud removal. In this architecture, the most critical component is the Spatio-Temporal Alternating Iterative Transformer (STAIT), which primarily consists of temporal and spatial attention mechanisms. STAIT is engineered to refine spatio-temporal feature representation by establishing an effective interplay between spatial details and temporal dynamics. Our framework is enhanced by an efficient image token generator with group convolution-based multi-level feature extraction to manage complexity, and a pixel reconstruction decoder with a shared progressive upsampling network to improve reconstruction by learning time-invariant features. Experimental results demonstrate that by explicitly modeling spatio-temporal feature dependencies, our approach achieves superior performance in restoring high-fidelity, cloud-free imagery.

Multi-modal image fusion synthesizes information from multiple sources into a single image, facilitating downstream tasks such as semantic segmentation. Current approaches primarily focus on acquiring informative fusion images at the visual display stratum through intricate mappings. Although some approaches attempt to jointly optimize image fusion and downstream tasks, these efforts often lack direct guidance or interaction, serving only to assist with a predefined fusion loss. To address this, we propose an ``Unfolding Attribution Analysis Fusion network'' (UAAFusion), using attribution analysis to tailor fused images more effectively for semantic segmentation, enhancing the interaction between the fusion and segmentation. Specifically, we utilize attribution analysis techniques to explore the contributions of semantic regions in the source images to task discrimination. At the same time, our fusion algorithm incorporates more beneficial features from the source images, thereby allowing the segmentation to guide the fusion process. Our method constructs a model-driven unfolding network that uses optimization objectives derived from attribution analysis, with an attribution fusion loss calculated from the current state of the segmentation network. We also develop a new pathway function for attribution analysis, specifically tailored to the fusion tasks in our unfolding network. An attribution attention mechanism is integrated at each network stage, allowing the fusion network to prioritize areas and pixels crucial for high-level recognition tasks. Additionally, to mitigate the information loss in traditional unfolding networks, a memory augmentation module is incorporated into our network to improve the information flow across various network layers. Extensive experiments demonstrate our method's superiority in image fusion and applicability to semantic segmentation.