Hongyu Chen

Cross-domain few-shot learning (FSL) has shown promising performance in hyperspectral image classification (HSIC) under limited labeled data. However, existing approaches often suffer from insufficient meta-knowledge transfer due to reliance on a single source domain, and fail to fully bridge the domain gap owing to the use of single-level alignment strategies. In this paper, we propose a heterogeneous data-driven, global-to-local cross-domain FSL framework for HSIC, leveraging richly labeled natural RGB images and hyperspectral data as source domains to support classification in the target domain with only a few labeled samples. The proposed method consists of two stages. In the first stage, we employ natural RGB images to learn a powerful spatial feature extractor with FSL and self-supervised learning, which will be fine-tuned for HSI in the second stage. To alleviate the domain gap between the source and target HSI domains at the second cross-domain FSL stage, we propose a global-to-local domain adaptation strategy that performs alignment both at the domain level and class level, effectively reducing the learning bias toward the source HSI while promoting discriminative feature learning. Additionally, to address large intra-class variance in the target HSI domain, we introduce a self-supervised contrastive loss based on positive pairs only, enhancing the within-class representation compactness. Extensive experiments on three benchmark datasets demonstrate that our method outperforms the state-of-the-art.

The presence of metallic implants introduces bright and dark streaks that appear in computed tomography (CT) images, degrading image quality and interfering with medical diagnosis. To reduce these artifacts, deep learning approaches have been applied for metal-corrupted restoration, which usually requires a large amount of simulated degraded-clean pairs for training. To achieve metal artifact reduction (MAR) without reference images, implicit neural representation (INR) has emerged and shown capabilities for image restoration in an unsupervised manner. However, existing INR methods for MAR usually treat the spatial coordinates independently and ignore their correlation, resulting in detail loss and artifacts remaining. In this paper, we propose an INR-based unsupervised MAR framework and design a High-order Line Attention Network to capture local contextual and geometric representations from X-rays, which maps the spatial coordinates into discrete linear attenuation coefficients of imaged objects for artifact-free CT image reconstruction. The second-order feature interaction can effectively improve the spectral bias problems and fit low and high-frequency details of real signals well. The proposed line-attention module with linear complexity can establish global relationships among spatial point tokens from sampled rays. To provide more local contextual information, a multiple local adjacent ray sampling strategy is adopted to compose several sub-fan beams with more context as a training batch. With the help of these components, the unsupervised MAR framework can approximate the implicit continuous function to estimate measurements and generate artifact-free CT images. Simulated and real experiments indicated that the proposed approach achieved superior MAR performance compared with other state-of-the-art methods.

Recently, transformer-based models have achieved remarkable performance in the hyperspectral image (HSI) classification. However, due to the limited training data, existing methods often show limited capability of capturing fine-grained local features. Although attempts have been made to solve this problem, the large amount of parameters imposes the risk of overfitting. In this paper, we propose a locally enhanced transformer network for HSI classification with fewer network parameters, which mainly consists of a multi-branch spatial-spectral tokenization (MSST) module and a dual-branch transformer encoder (DTE) module. The MSST generates effective spatialspectral tokens through diverse convolutions with a residual connection. The DTE consists of a global transformer branch and a locally enhanced transformer branch, which are used to capture the global and local spatial dependencies of HSI, respectively. Unlike the conventional self-attention module used in the global branch, we propose an improved multi-head selfattention (IMSA) module in the local branch by incorporating the local prior information of HSI with graph convolution, to enhance the local information extraction. To fuse the global and local features from the two branches, we introduce an adaptive strategy by using learnable weights for both branches. We devise our MSST and DTE with a shallow architecture, significantly reducing the number of parameters. Experimental results on benchmark datasets demonstrate that the proposed method outperforms the state-of-the-art.

Landsat optical sensor is crucial for long-term observations of the Earth’s surface with a 30-m spatial resolution. However, the 16-day revisit cycle and severe atmospheric interference have impeded the monitoring of rapid surface changes. Spatiotemporal fusion (STF) is a classic method of predicting Landsat surface reflectance with multitemporal and multisource data, but it is limited by unpredictable temporal changes and cloudy Landsat–moderate resolution imaging spectroradiometer (MODIS) image pairs. Another emerging solution is synthetic aperture radar (SAR)-to-optical image translation (S2OIT), which always produces spectral distortions. To tackle these defects, we propose a new data-driven solution, SAR–optical data-based spatial–spectral fusion (SOSSF), which combines the high-spatial and cloud-free advantages of Sentinel-1 data and the high-spectral and high-temporal advantages of MODIS images to synthesize high-spatial and high-temporal Landsat-8 images. To achieve this solution, we first establish a worldwide benchmark dataset, namely, Sentinel-1, MODIS, and Landsat-8 triplets (SMILE), with various land cover types and all meteorological seasons, satisfying the big data requirements of deep learning (DL). Second, we design an attention-based dual-path fusion network (ADFNet) to respectively extract and fully fuse spatial and spectral information from SAR–optical data. Extensive experiments suggest that the proposed SOSSF solution outperforms the state-of-the-art (SOTA) STF and S2OIT solutions, robustly performing in the continuously changing and frequently cloudy regions. The proposed ADFNet model achieves the best visual effect and the highest accuracy in different scenes, seasons, and bands. Furthermore, the proposed SOSSF solution is proven to be a practical way to simulate time-series and large-scale Landsat-8 surface reflectance, considerably enriching raw Landsat-8 products.

Hyperspectral image (HSI) qualities are limited by a mixture of Gaussian noise, impulse noise, stripes, and deadlines during the sensor imaging process, resulting in weak application performance. To enhance HSI qualities, methods based on convolutional neural networks have been successively applied to restore clean data from the observed data. However, the architecture of these methods lacks spectral and spatial constraints, and the convolution operators have limited receptive fields and inflexible model inferences. Thus, in this study, we propose an efficient end-to-end transformer, named HSI denoising transformer (Hider), for mixed HSI noise removal. First, a U-shaped 3-D transformer architecture is built for multiscale feature aggregation. Second, a multihead global spectral attention module within the spectral transformer block is designed to excavate information in different spectral patterns. Finally, an additional locally enhanced cross-spatial attention module within the spatial–spectral transformer block is constructed to build the long-range spatial relationship to avoid the high computational complexity of global spatial self-attention. Through the imposition of global correlations along spectrum and spatial self-similarity constraints on the transformer, our proposed Hider aims to capture long-range spatial contextual information and cluster objects with the same spectral pattern for HSI denoising. To verify the effectiveness and efficiency of Hider, we conducted extensive simulated and real experiments. The denoising results on both simulated and real-world datasets show that Hider achieves superior evaluation metrics and visual assessments compared with other state-of-the-art methods.

Although deep learning has made semantic segmentation of very-high-resolution (VHR) remote sensing (RS) images practical and efficient, its large-scale application is still limited. Given the diversity of imaging sensors, acquisition conditions, and regional styles, a deep learning network well-trained on one source domain dataset often suffers from drastic performance drops when applied to other target domain datasets. Thus, we propose a novel end-to-end mutual information domain adaptation network (MIDANet) that can shift between semantic segmentation domains by integrating multitask learning in the convolutional neural networks within an entropy adversarial learning (EAL) framework. Through the joint learning of semantic segmentation and elevation estimation, the features extracted by MIDANet can concentrate more on the elevation clues while dropping the domain-variant information (i.e., texture, spectral information). First, one encoder is applied to excavate general semantic features. Two decoders that share the same architecture are used to perform pixel-level classification and digital surface model (DSM) regression. Second, feature interaction modules (FIMs) and a mutual information attention unit (MIAU) are designed to mine the latent relationships between the two tasks and enhance their feature representations. Finally, a final MIDANet is obtained for semantic segmentation that does not require any semantic segmentation labels in the target domain after the adversarial learning of the classification entropy at the output level. Extensive comparative experiments and ablation studies were conducted on the International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam and Vaihingen test datasets. The results show that MIDANet outperforms other state-of-the-art domain adaptation (DA) methods in both evaluation metrics and visual assessment.

Due to the physical limitations of the imaging devices, hyperspectral images (HSIs) are commonly distorted by a mixture of Gaussian noise, impulse noise, stripes, and dead lines, leading to the decline in the performance of unmixing, classification, and other subsequent applications. In this paper, we propose a novel end-to-end low-rank spatial-spectral network (LR-Net) for the removal of the hybrid noise in HSIs. By integrating the low-rank physical property into a deep convolutional neural network (DCNN), the proposed LR-Net simultaneously enjoys the strong feature representation ability from DCNN and the implicit physical constraint of clean HSIs. Firstly, spatial-spectral atrous blocks (SSABs) are built to exploit spatial-spectral features of HSIs. Secondly, these spatial-spectral features are forwarded to a multi-atrous block (MAB) to aggregate the context in different receptive fields. Thirdly, the contextual features and spatial-spectral features from different levels are concatenated before being fed into a plug-and-play low-rank module (LRM) for feature reconstruction. With the help of the LRM, the workflow of low-rank matrix reconstruction can be streamlined in a differentiable manner. Finally, the low-rank features are utilized to capture the latent semantic relationships of the HSIs to recover clean HSIs. Extensive experiments on both simulated and real-world datasets were conducted. The experimental results show that the LR-Net outperforms other state-of-the-art denoising methods in terms of evaluation metrics and visual assessments. Particularly, through the collaborative integration of DCNNs and the low-rank property, the LR-Net shows strong stability and capacity for generalization.

In this paper, we present the scientific outcomes of the 2019 Data Fusion Contest organized by the Image Analysis and Data Fusion Technical Committee of the IEEE Geoscience and Remote Sensing Society. The 2019 Contest addressed the problem of 3D reconstruction and 3D semantic understanding on a large scale. Several competitions were organized to assess specific issues, such as elevation estimation and semantic mapping from a single view, two views, or multiple views. In this Part A, we report the results of the best-performing approaches for semantic 3D reconstruction according to these various set-ups, while 3D point cloud semantic mapping is discussed in \Part B [1].