Hongbo Liang

Synthetic aperture radar (SAR) image to optical (S2O) image translation not only assists information interpretability, but also fills the gaps in optical applications due to weather and light limitations. However, several studies have pointed out that specialized methods heavily struggle to deliver images with widely varying optical imaging styles, thus, resulting in poor image translation with disharmonious and repetitive artifacts. Another critical issue attributes to the scarcity of geographical prior knowledge. The generator always attempts to produce images within a narrow scope of the data space, which severely restricts the semantic correspondence between SAR content and optical styles. In this paper, we introduce a novel tokenization, namely geographical imaging tokenizer (GIT), which captures imaging style of ground materials in the optical image. Based on the GIT, we propose a geographical feature tokenization transformer framework (GFTT) that discovers the consensus between SAR and optical images. In addition, we leverage a self-supervisory task to encourage the transformer to learn meaningful semantic correspondence from local and global style patterns. Lastly, we utilize the noise-contrastive estimation (NCE) loss to maximize mutual information between the input and translated image. Through qualitative and quantitative experimental evaluations, we verify the reliability of the proposed GIT that aligns with authentic expressions of the optical observation scenario, and indicates the superiority of GFTT in contrast to the state-of-the-art (SOTA) algorithms.

In this letter, we propose the CLN-Fusion, a novel hybrid fusion approach that leverages the merits of CNNs and vision Transformers (ViTs) to couple local-nonlocal feature representations between SAR and MS images. Specifically, we construct a paired token projection (PTP) to match the observation scenario content consistency of the two. Meanwhile, in terms of merging the complementary features between structures in SAR images and textures in MS images, we establish the pyramid CNN and ViT branches that assemble two pure feature volumes with convolutional inductive biases and nonlocal statistical correlation respectively into a mixed one. Furthermore, our CLN-Fusion maintains semantic alignment by maximizing mutual information throughout the PTP. Extensive experiments validate the superiority of the CLN-Fusion in terms of quantitative metrics, achieving SAR/MS image fusion under three scenarios from Sentinel-1 and Landsat8 data. with PSNRs of 33.1565, 30.9815, and 29.9821, showcasing the utmost fusion performance in contrast to other state-of-the-art (SOTA) methods. The codes of this work will be available at https://github.com/Blueseatear/CLN-Fusion.

In recent years, bidirectional encoder representation from transformers (BERT) models have achieved superior performance in hyperspectral images (HSIs). It can capture the long-range correlations between HSI elements, but the local space and spectral band information of HSI is insufficient. We propose a spatially augmented guided sequence BERT network for HSI classification study, referred to as SAS-BERT, which makes more effective use of HSI’s spatial and spectral information by improving the BERT model. First, a spatial augmentation learning module is added in the preprocessing stage to obtain more significant spatial features before the input network and better guide the spatial sequence. Then a spectral correlation module was used to represent the spectral band features of the HSI and to establish a correlation with the spatial location of the images to obtain better classification performance. Experimental results on three datasets show that the method proposed achieves better classification performance than other state-of-the-art methods.

Transformer networks have shown impressive performance for hyperspectral interpretation. Nevertheless, the high-dimensional redundant spectral distribution of hyperspectral images (HSIs) hinders their validity of interaction between features from distant locations. In this letter, we propose the HSI-Mixer, a novel extremely simple convolution neural network (CNN), which is similar in spirit to Transformer to reconsider the remarkable inductive biases of convolutions. In specific, we construct a hybrid measurement-based linear projection (HMLP) to merge spectral signatures and spatial positions of an HSI cuboid. Meanwhile, according to the merging relations between spectral–spatial attributes, we establish both spectral and spatial Mixer blocks to separate features from a mixed volume to a pure one, across either spectral bands or spatial locations, respectively. Furthermore, our HSI-Mixer maintains the same-depth-and-resolution throughout the network. Experimental results on three benchmark datasets demonstrate that our proposal achieves promising performance, in contrast to other state-of-the-art (SOTA) methods. The codes of this work will be available at https://github.com/Blueseatear/IEEE_GRSL_2022_HSI-Mixer .

Hyperspectral image (HSI) consists of hundreds of narrow and continuous spectral bands, in which abundant spectral information enhances the ability to identify land cover materials.^{1 – 5} With the rapid development of imaging sensor technology, the acquisition of high spatial resolution HSI is no longer a challenge, which provides an ideal research base for environmental protection,⁶ land monitoring,⁷ and urban development.⁸ Hyperspectral image classification (HSIC), which provides a unique land cover category for each pixel according to spectral signatures and spatial context, is an important task in these remote sensing applications.^{9 – 11} Typical HSIC methods consist of two separate steps: feature extraction and classifier training. However, facing the challenges of high-dimensional nonlinear distribution and redundant bands of information, traditional methods generally suffer from low generalization ability and limited degree of feature representation.

Recent research shows that generative adversarial network (GAN) based deep learning derived frameworks can improve the accuracy of hyperspectral image (HSI) classification on limited labeled samples. However, several studies point out that existing GAN-based methods are heavily affected by the complexity and inefficient description issues of HSIs. The discriminator in GAN always attempts to interpret high-dimensional nonlinear spectral knowledge of HSIs, thus resulting in the Hughes phenomenon. Another critical issue is sample generation. The generator is only used as a regularizer for the discriminator, which seriously restricts the performance for classification. In this article, we propose SSAT-GAN, a semisupervised spectral–spatial attention feature extraction approach based on the GAN that feeds raw data into a deep learning framework, in an end-to-end fashion. First, the unlabeled data is added into the discriminator to alleviate the problems of training samples and supplies a reconstructed real HSI data distribution through adversarial training. Second, to enhance the description of HSIs, we build spectral–spatial attention modules (SSAT) and extend them to the discriminator and the generator to extract discriminative characteristics from abundant spatial contexts and spectral signatures. The SSAT modules learn a three-dimensional filter bank with spectral–spatial attention weights to obtain meaningful feature maps to improve the discrimination of the feature representation. In terms of the mode collapse of GANs, the mean minimization loss is employed for unsupervised learning. Experimental results from three real datasets indicate that SSAT-GAN has certain advantages over the state-of-the-art methods.

This letter proposes a two-phase strategy-based grouped collaborative representation classifier (CRC) for hyperspectral image classification. Specifically, a spectral correlation-based CRC (SCCR) is proposed in the first phase, which considers a regularization term that uses the spectral correlations between the test sample and training samples. The representation coefficient vector generated by SCCR is then transformed into class-specific group weights. In the second phase, we integrate group weights and spectral correlations into the CRC and propose a grouped CRC (GCRC). Experimental results obtained from three real hyperspectral data sets demonstrate that the proposed SCCR and GCRC can provide better classification performance over other state-of-the-art representation-based classifiers.

Due to the limited spatial resolution of a hyperspectral sensor, different materials can jointly occupy a single pixel, which occurs as a mixed pixel in a hyperspectral image. The process of decomposing a mixed pixel into a collection of constituent spectra, or endmembers, and a set of corresponding fractions, or abundances, is called hyperspectral unmixing.¹

Recently, generative adversarial network (GAN)-based methods for hyperspectral image (HSI) classification have attracted research attention due to their ability to alleviate the challenges brought by having limited labeled samples. However, several studies have demonstrated that existing GAN-based HSI classification methods are limited in redundant spectral knowledge and cannot extract discriminative characteristics, thus affecting classification performance. In addition, GAN-based methods always suffer from the model collapse, which seriously hinders their development. In this study, we proposed a semi-supervised adaptive weighting feature fusion generative adversarial network (AWF²-GAN) to alleviate these problems. We introduced unlabeled data to address the issue of having a small number of samples. First, to build valid spectral–spatial feature engineering, the discriminator learns both the dense global spectrum and neighboring separable spatial context via well-designed extractors. Second, a lightweight adaptive feature weighting component is proposed for feature fusion; it considers four predictive fusion options, that is, adding or concatenating feature maps with similar or adaptive weights. Finally, for the mode collapse, the proposed AWF²-GAN combines supervised central loss and unsupervised mean minimization loss for optimization. Quantitative results on two HSI datasets show that our AWF²-GAN achieves superior performance over state-of-the-art GAN-based methods.