Haibo Liu

High-speed full-field displacement measurement is constrained by the cost of high-speed cameras and the trade-off between frame rate and spatial resolution. Video snapshot compressive imaging (SCI) reconstructs multiple frames from a single exposure, offering an effective route to low-cost high-speed displacement measurement. Among SCI reconstruction methods, plug-and-play (PnP)-based methods offer favorable theoretical interpretability and flexibility, and therefore hold great promise for high-speed imaging applications such as photomechanical measurement. However, they typically adopt full-precision deep networks as the denoising prior, and this cost becomes significant in high-resolution, resource-constrained measurement systems. To address this issue, we propose a binary deep prior guided snapshot compressive imaging for full-field displacement measurement, termed BDPnP. Specifically, to the best of our knowledge, we first propose a lightweight deep video denoiser as an implicit prior based on binary convolutions, which significantly reduces the computational complexity and parameter count. Building upon this, a cross-measurement temporal collaboration strategy and a hybrid prior alternating strategy are proposed to alleviate temporal-context fragmentation at measurement boundaries and performance degradation under non-ideal hardware imaging, respectively. These two strategies further compensate for the information loss from binarization. Finally, we constructed an SCI hardware system and integrated BDPnP with a digital image correlation pipeline, using rigid-body translation as a benchmark case with exactly known ground truth. At 1024 × 1024 resolution with a compression ratio of 8, i.e., an eightfold gain in the effective acquisition frame rate, BDPnP achieves a mean displacement error of 0.045 mm, only 0.34% of the total displacement. This is comparable to the lowest mean error achieved by the fine-tuned deep learning method in our experiments (0.044 mm) while requiring only 1/47.7 of its FLOPs and 1/7.7 of its parameters. Our code is available at https://github.com/hnu-VML/qmx/tree/main/BDPnP .

Syntheticaperture radar (SAR) and optical (OPT) image fusion aims to leverage their complementary information to obtain a more comprehensive representation of ground objects. However, significant discrepancies exist between the two modalities in terms of imaging mechanisms and feature distributions. Consequently, existing multi-modal image fusion methods struggle to achieve robust cross-modal feature alignment and deep semantic consistency between the fused results and the source modalities. To address the above challenges, this paper proposes a correlation and semantic prior-guided multi-scale cross-modal interaction network (CSP-MCIN) for effective SAR-OPT image fusion. Specifically, CSP-MCIN first employs two modality-specific encoders based on ResNet-18 to extract low-level details and high-level semantic features from SAR and OPT images, respectively. Subsequently, a multi-scale interactive decoder integrating cross-modal Transformers and gated fusion units is constructed to align and aggregate semantic and detail information from both encoders. Finally, to strengthen source-modality representations, a novel loss function combining a pixel-domain correlation loss and a CLIP-guided semantic consistency loss is designed and optimized under a PCGrad-based multi-objective optimization scheme. Experimental results on public SAR-OPT image datasets demonstrate that the proposed CSP-MCIN achieves superior fusion performance and computational efficiency compared with state-of-the-art approaches.

Multi-view clustering (MVC) based on anchor learning has been proven to be effective in improving clustering accuracy and efficiency. Existing MVC methods are mainly based on single-granularity anchor learning, that is, the number of anchors corresponding to different views is constant and consistent, which will lead to information redundancy or insufficient mining. In addition, aggregating anchors of varying scales from all views to obtain multi-view shared clustering results remains a problem to be further explored. To address the above problems, a novel MVC method named View-adaptive Multi-granularity Anchor Learning (VMAL) is proposed in this paper, where view-adaptive anchor pruning and view-shared sample clustering are jointly optimized. On the one hand, VMAL can dynamically adjust the optimal number of anchors for each view during optimization by exploiting the reconstruction error of samples. On the other hand, an intuitive and effective mapping-aggregation message passing strategy is cleverly designed, which first maps the anchor representations of different views to the cluster space and then transfers the obtained cluster information of anchors to the sample space through an aggregation matrix. As a byproduct, VMAL can directly obtain the discrete cluster distribution of samples without additional partitioning. Finally, an iterative optimization algorithm is developed to solve the proposed VMAL method. Experimental results on multiple datasets have demonstrated the superiority of VMAL in terms of clustering results when compared with other state-of-the-art methods.

Extracting rotation-invariant local descriptors is vital for many downstream tasks, such as image matching and 3D reconstruction. However, learning-based methods designed specifically for arbitrary rotations have rarely been studied. Therefore, we propose a novel local descriptors learning method consisting of circular polar transform (CPT) and lightweight cylindrical convolution network (L2C-Net) in this paper. On the one hand, in order to better cope with rotation, CPT is first used to obtain representations of input patches in polar coordinate space, which converts a rotation problem into a translation problem suitable for convolutional networks. Compared with polar transform, CPT can achieve relatively uniform sampling from the center to the circumference along the radius direction of polar coordinates, thereby effectively reducing information loss. On the other hand, L2C-Net based on cylindrical convolution is designed to deal with the property of warp-around connections of the obtained polar representations. As a result, local descriptors that are invariant to arbitrary rotation angles can be obtained. Extensive experimental results have demonstrated that our proposed method can surpass the current state-of-the-art rotation-invariant descriptors under various rotations and also shows competitive performance on four different tasks.

Cross-view image geo-localization aims to estimate the geographic position of a query image from the ground platform (such as mobile phone, vehicle camera) by matching it with geo-tagged reference images from the aerial platform (such as drone, satellite). Although existing studies have achieved promising results, they usually rely only on depth features and fail to effectively handle the serious changes in geometric shape and appearance caused by view differences. In this paper, a novel Semantic Concept Perception Network (SCPNet) with interactive prompting is proposed, whose core is to extract and integrate semantic concept information reflecting spatial position relationship between objects. Specifically, for a given of pair input images, a CNN stem with positional embedding is first adopted to extract depth features. Meanwhile, a semantic concept mining module is designed to distinguish different objects and capture the associations between them, thereby achieving the purpose of extracting semantic concept information. Furthermore, to obtain global descriptions of different views, a feature bidirectional injection fusion module based on attention mechanism is proposed to exploit the long-range dependencies of semantic concept and depth features. Finally, a triplet loss with a flexible hard sample mining strategy is used to guide the optimization of the network. Experimental results have shown that our proposed method can achieve better performance compared with state-of-the-art methods on mainstream cross-view datasets.

Bipartite graph (BiG) has been proven to be efficient in handling massive multiview data for clustering. However, how to regulate the structural information of view-specific anchors and view-shared BiG is still open and needs to be further studied. Hence, a novel dual-structural BiG learning (DsBiGL) method is proposed in the article. It transforms BiG learning into a joint optimization problem of IntrA-view and InteR-view subspace learning (IASL and IRSL) with the structural constraints, such as k-nearest neighbor (KNN) and low-rank. On one hand, IASL uses the KNN and view-specific low-rank constraints to enhance the discriminativeness of view-specific anchors. On the other hand, IRSL uses an adaptive weighting strategy to obtain view-shared BiG directly from multiview samples, where the KNN and view-shared low-rank constraints are adopted to encode local connectivity and cluster information between samples. Note that IASL and IRSL are integrated into a unified optimization model, which ensures the interactive enhancement of view-specific anchor representation and view-shared BiG learning. Finally, an algorithm based on iterative optimization is designed to solve the proposed DsBiGL model. Experimental results on various multiview datasets have demonstrated the superiority of DsBiGL in terms of clustering results when compared with other comparative methods.