Jing Yu

Recent training-free layout-to-image diffusion models have demonstrated remarkable performance in generating high-quality images with controllable layouts. These models follow a one-stage framework: Encouraging the model to focus the attention map of each concept on its corresponding region by defining attention map-based losses. However, these models still struggle to accurately follow layouts with significant overlap, often leading to issues like attribute leakage and missing entities. In this paper, we propose ToLo, a two-stage, training-free layout-to-image generation framework for high-overlap layouts. Our framework consists of two stages: the aggregation stage and the separation stage, each with its own loss function based on the attention map. To provide a more effective evaluation, we partition the HRS dataset based on the Intersection over Union (IoU) of the input layouts, creating a new dataset for layout-to-image generation with varying levels of overlap. Through extensive experiments on this dataset, we demonstrate that ToLo significantly enhances the performance of existing methods when dealing with high-overlap layouts. Our code and dataset are available here: https://github.com/misaka12435/ToLo.

In recent years, deep learning has been widely applied to hyperspectral image (HSI) classification with great success. However, since labeling hyperspectral images is time-consuming and labor-intensive, a limited number of labeled hyperspectral images are available, making it difficult to train feature extractors and classifiers. To address this challenge, this paper introduces a self-supervised spectral-spatial graph prototypical network for few-shot HSI classification (S ⁴ GPN). In addition, we combine self-supervised learning with few-shot learning to provide additional semantic information to improve classification accuracy. Our method consists of three stages, including the prototype network (PN) stage, the self-supervised learning (SSL) stage, and the fusion stage (Fusion stage), with each stage progressively improving the classification performance. In the PN stage, we perform supervised learning using a prototype network structure and leverage the supervised information to guide self-supervised learning. In the SSL stage, we use the SimSiam structure in a novel way for model training after the augmentation of spectral and spatial data separately. Finally, the features learned in the first two stages are fused to improve the quality of the feature representation. These three stages use the same structured feature extractors. To extract more diverse and discriminative feature representations for the HSI classification task, our method uses the graph convolution network (GCN) and the dense network (Densenet) to extract spectral information and spatial information, respectively. Experiments on four data sets and comparisons with the state-of-the-art methods demonstrate that our proposed S ⁴ GPN outperforms other methods for HSI classification.

Hyperspectral classification is an important technique for remote sensing image analysis. For the current classification methods, limited training data affect the classification results. Recently, Conditional Variational Autoencoder Generative Adversarial Network (CVAEGAN) has been used to generate virtual samples to augment the training data, which could improve the classification performance. To further improve the classification performance, based on the CVAEGAN, we propose a Self-Attention-Based Conditional Variational Autoencoder Generative Adversarial Network (SACVAEGAN). Compared with CVAEGAN, we first use random latent vectors to obtain more enhanced virtual samples, which can improve the generalization performance. Then, we introduce the self-attention mechanism into our model to force the training process to pay more attention to global information, which can achieve better classification accuracy. Moreover, we explore model stability by incorporating the WGAN-GP loss function into our model to reduce the mode collapse probability. Experiments on three data sets and a comparison of the state-of-art methods show that SACVAEGAN has great advantages in accuracy compared with state-of-the-art HSI classification methods.

The convolutional neural network (CNN) is a feed-forward neural network with deep structure and convolution operation. In the hyperspectral image (HSI) classification, CNN has demonstrated excellent performance in extracting spectral and spatial information. However, the inherent complexity and high dimension of HSIs still limit the performance of most neural network models. The powerful feature extraction ability of CNN is normally achieved by dozens or more layers, which brings a series of problems such as gradient vanishing, overfitting, and slow training speed. In order to address these problems, this article presents a CNN architecture-based stochastic depth residual network (SDRN), which is specially designed for HSI data. This model takes the original 3-D cube as the input and 3-D convolution is used to extract abundant spectral and spatial features through corresponding residual blocks. In order to reduce the training time, we adopt a stochastic depth strategy. For each small batch, a sublayer is randomly discarded by an identity function. During the testing stage, the residual network with complete depth is used. Experiments on three datasets and a comparison of the state-of-art methods show that SDRN has great advantages in accuracy and training time compared with state-of-the-art HSI classification methods.

Convolutional neural networks (CNN) have achieved excellent performance for the hyperspectral image (HSI) classification problem due to better extracting spectral and spatial information. However, CNN can only perform convolution calculations on Euclidean data sets. To solve this problem, recently, graph convolutional neural network (GCN) is proposed, which can be applied to the semi-supervised HSI classification problem. However, the GCN is a direct push learning method, which requires all nodes to participate in training process to get the node embedding. This may bring great computational for the hyperspectral data with large number of pixels. Therefore, in this paper, we propose a inductive learning method to solve the problem. It constructs the graph by sampling and aggregating (graphSAGE) feature from a node's local neighbourhood. This could greatly reduce the space complexity. Moreover, to enhance the classification performance, we also construct graph using spectral and spatial information (spectral-spatial GraphSAGE). Experiments on several hyperspectral image datasets show that the proposed method can achieve better classification performance compared with state-of-the-art HSI classification methods.

Spectral or spatial dictionary has been widely used in fusing low-spatial-resolution hyperspectral (LH) images and high-spatial-resolution multispectral (HM) images. However, only using spectral dictionary is insufficient for preserving spatial information, and vice versa. To address this problem, a new LH and HM image fusion method termed OTD using optimized twin dictionaries is proposed in this paper. The fusion problem of OTD is formulated analytically in the framework of sparse representation, as an optimization of twin spectral-spatial dictionaries and their corresponding sparse coefficients. More specifically, the spectral dictionary representing the generalized spectrums and its spectral sparse coefficients are optimized by utilizing the observed LH and HM images in the spectral domain; and the spatial dictionary representing the spatial information and its spatial sparse coefficients are optimized by modeling the rest of high-frequency information in the spatial domain. In addition, without non-negative constraints, the alternating direction methods of multipliers (ADMM) are employed to implement the above optimization process. Comparison results with the related state-of-the-art fusion methods on various datasets demonstrate that our proposed OTD method achieves a better fusion performance in both spatial and spectral domains.