Baisen Liu

Unmixing networks for hyperspectral images (HSIs) often need to be redesigned for each sensor and initialized with endmember-estimation algorithms, which limits crossscene generalization. To address this limitation, this work finetunes a cross-sensor pretrained hyperspectral foundation model for spectral unmixing, thereby eliminating the dependence on endmember-estimation algorithms. Specifically, the pretrained model is retained as an intact backbone, and only a lightweight task head for joint endmember and abundance estimation is appended. Without modifying the backbone or invoking endmember-estimation algorithms, the proposed framework can be directly trained on the public datasets. Experimental results show performance on par with current start-of-the-art methods while exhibiting zero endmember-estimation capability. Ablation studies further confirm that large-scale pretraining is critical to the observed gains. This is the first attempt to achieve cross-sensor, endmember-free hyperspectral unmixing (HU) via a foundation model, providing a universal framework for future unmixing applications.

This work has delivered a novelty foundational model for hyperspectral remote sensing images. Current approaches for hyperspectral data interpretation often require specialized models that are specifically tailored to individual datasets or tasks. In some specific tasks, the availability of hyperspectral data is often limited, posing significant challenges to training due to data insufficiency. Furthermore, the diverse structure of hyperspectral data often complicates the transfer of knowledge from other available datasets, severely limiting cross-scenario capabilities. To bridge this gap, we introduce a highly adaptable foundational model with strong transferability, capable of processing all forms of hyperspectral data across diverse spectral bands and ranges. Compared to previous methods, our approach: 1) standardizes all types spectral vectors into a common token format, enabling a single model for multi-source hyperspectral data; 2) aligns spectral features across different ranges by embedding wavelength information into position encoding; 3) has been pre-trained on over 300 million spectral instances worldwide, ensuring broad generalization; 4) transfers learned knowledge effortlessly to downstream tasks and new datasets without architectural modifications or training from scratch. Experimental results demonstrate that, compared to other mainstream methods, our approach achieves state-of-the-art classification performance across various datasets with different spectral characteristics in both supervised and unsupervised learning settings, while also delivering impressive results in change detection tasks. The source code and the pretrained weights are available at https://github.com/kkweil/HyperSL.

Deep learning-based processing of hyperspectral remote sensing images (HSIs) has emerged as a research hotspot. In recent years, the delivery of numerous novel HSI acquisition platforms has resulted in an exponential growth in the number of available HSI datasets from various sources. However, due to variations among hyperspectral sensors, the acquired data frequently consists of various spectral dimensions. This results in a challenge since standard deep learning approaches often require a different model for each HSI source, which impedes the construction of fundamental models for HSIs. To address this issue, we propose a unified representation mechanism for multi-source HSIs that can transform spectra from numerous dimensions to a shared representation space, yielding in a scalable pre-training model. Compared to existing methods, our strategy has the following advantages: 1)compatibility with HSIs of arbitrary spectral dimensions, ranges, resolutions, 2)fully leveraging existing multi-source HSIs, 3) spontaneously capturing spectral features via self-supervised learning, 4) pretrained on large-scale multi-source HSI datasets and a considerable enhancement in classification accuracy. In three reconstuction testsets, the PSNR are 27.07dB, 22.27dB, 29.35dB, and the SSIM are 0.93, 0.82, 0.88, respectively. Compared with the randomly initialized model, there are 8.28% and 1.4% improvements on the Indian Pines dataset and Pavia University dataset respectively.

Cross-scene hyperspectral image (HSI) classification using transfer learning can effectively address the challenge of insufficient labeled samples in classification. The majority of existing transfer learning methods assume consistency in feature labels between the source and target HSI. In practical situations, the target-domain hyperspectral data are likely to contain some categories that do not exist in the source-domain hyperspectral data. In order to solve this problem, this article proposes an unsupervised open-set cross-domain manifold mapping aligned method for target-domain HSI classification. This method designs manifold embedding maps to align the spectral–spatial features of hyperspectral in different scenes. The spectral–spatial feature data are mapped to the subspace using both the source-domain and target-domain mapping matrices. Gradually, target-domain samples are chosen for pseudolabeling within the shared subspace, while samples from unknown classes are rejected. This method achieves subspace learning and pseudolabeled sample updating through iteration, reducing the intraclass distance of all the classes and pushing rejected target data away from known classes. Experiments on the three cross-domain hyperspectral data pairs demonstrate that the proposed method outperforms related state-of-the-art methods.

Nowadays, an increasing number of hyperspectral images (HSIs) are becoming available. However, the utilization of unlabeled HSIs is extremely low due to high annotation costs. Thus, it is crucial to figure out how to use these unlabeled HSIs and enhance the classification performance. Fortunately, self-supervised training enables us to acquire latent features from unlabeled HSIs, thereby enhancing network performance via transfer learning. Whereas, most current networks for HSIs are inflexible, it is challenging for them to perform learning and accommodate multimodal HSIs. Therefore, we devise a scalable self-supervised network called instructional mask autoencoder, which can extract general patterns of HSIs by these unannotated data. It primarily consists of a spatial–spectral embedding block and a transformer-based masked autoencoder, which are utilized for projecting input samples into the same latent space and learning higher level semantic information, respectively. Moreover, we utilize a random token called $ins\_{t}oken$ to instruct the model learn components of global information, which are highly correlated with the target pixel in HSI samples. In the fine-tuning stage, we design a learnable aggregation mechanism to put all tokens into full play. The obtained results illustrate that our method exhibits robust generalization performance and accelerates convergence across diverse datasets. In cases of limited samples, we conducted experiments on three structurally distinct HSIs, all of which achieved competitive performance. Compared to state-of-the-art methods, our approach demonstrated respective improvements of 1.97%, 0.44%, and 3.35% on these three datasets.

Talent flow is a vital component of the region’s competitiveness and development. Accurately grasping talent trends can aid in the development of the region. Unlike text and image data, the characteristics of questionnaire data are more dependent on prior and potential correlations between each factor than on its order and position relationship. To efficiently extract features from questionnaire data and accurately predict talent flow, this paper proposes a novel talent flow prediction method based on the graph attention network. In our method, the data is transformed into graph data, the data feature is saved in the nodes, the adjacency matrix stores the link between the nodes, and an attention mechanism allocates the aggregate weight when the neighbor nodes are aggregated. The experiment demonstrates that the method proposed in this paper outperforms other algorithms for similar tasks both in accuracy and efficiency. It achieves an overall classification accuracy of 92.33% and more than 90% of the classification accuracy in each category prediction classification on our dataset.

Due to the complex properties of hyperspectral images (HSI), such as spatial‐spectral structure, high dimension, and great spectral variability, HSI clustering is a challenging operation. In this paper, we propose a novel deep convolutional asymmetric autoencoder‐based spatial‐spectral clustering network (DCAAES²C‐Net) which employs a convolutional autoencoder (CAE) and an asymmetric autoencoder to investigate spatial‐spectral information. First, we use a CAE to extract spatial‐spectral features. Then, we introduce an asymmetric autoencoder between the encoder and decoder of CAE to suppress some non‐material‐related spatial information in latten feature like shading and texture. By using a collaborative strategy to train the proposed networks, we obtain the representation features in a low dimension. Furthermore, we improve the k‐means algorithm by using the concept of over‐clustering to handle fuzzy representation which is difficult to distinguish the cluster, and utilize it to obtain the final HSI clustering result. The results of the experiments demonstrated that the proposed methodology outperforms other methods on the frequently used hyperspectral image dataset.