Jiangxin Yang

Passive non-line-of-sight (NLOS) imaging has garnered significant interest as a promising technique for cost-effective “corner-turning sensing”. Existing methods, however, face fundamental limitations: data-driven models generally suffer from limited generalization and interpretability, while physics-based approaches typically produce low-fidelity reconstructions. To address these challenges, this paper proposes a hybrid physics-data-driven imaging (HPDI) framework. HPDI employs a dual-path architecture that integrates a physics-informed coarse-to-fine pathway (CTFP) with a data-driven implicit reconstruction pathway (IRP). The former executes a staged progression from physics-informed coarse reconstruction to final refinement, while the latter encodes scene-specific statistical priors in an end-to-end manner. An adaptive fusion network synergistically integrates the physical insights of CTFP and the statistical abstractions of IRP, yielding enhanced reconstruction performance. To enable comprehensive evaluation, we construct multiple datasets covering diverse acquisition conditions (with/without occluder) and representative scenarios (sparse/complex). Experimental results demonstrate that HPDI consistently outperforms state-of-the-art methods in reconstruction fidelity, while exhibiting enhanced generalization under distribution shifts and higher data efficiency. This work represents a significant step toward harnessing the complementary strengths of physics-informed modeling and data-driven learning, thereby advancing the development of high-performance NLOS imaging. The code and dataset are publicly available at https://github.com/RL-NLOS/HPDI_for_Passive_NLOS.

The galvanometer-driven dual-camera systems can maintain a wide field of view while simultaneously capturing high-resolution details of targets in milliseconds, which is highly valuable for various high-speed tracking applications. Most existing systems utilize a precalibrated mapping function between image coordinates and pan-tilt mirror angles for dynamic target tracking. However, this ignores the dual-camera optical center misalignment, leading to severe system malfunctions when the target distance changes. To address this limitation, we propose a novel galvanometer-driven system that makes use of target information in both wide-view and zoom-in images for pan-tilt mirror angle modification, achieving high-accuracy target tracking at varying distances. Our proposed system introduces a practical dual-camera information interaction framework to accomplish iterative manipulation control of the galvanometer mirrors and adaptive adjustment of the coordinate-angle mapping function, improving the stability and precision of dynamic target tracking at varying distances. With a multithreaded task synchronization design, our developed framework enables the proposed system to remain target-to-track central-located within the zoom-in images without incurring extra time costs. Both qualitative and quantitative experimental results validate the superiority of our proposed system, extending the applicability of the galvanometer-driven dual-camera setup for target-tracking tasks over a wide range of distances.

Drone-based target detection is an indispensable technology in numerous applications, such as surveillance, search, and rescue. The widely used single-modal RGB detector is easily affected by numerous factors, such as inadequate illumination and adverse weather. Introducing the thermal modal on top of the RGB modal can significantly enhance detection performance and robustness in complex environments. Nevertheless, drone-based RGBT target detection faces several challenges, such as the difficulty of detecting tiny targets, misaligned modality space, and redundant modality information. To address these issues, we propose a novel Adaptive Modality Selection Drone-based RGBT Detector (AMSDet), which eliminates redundant information in different modalities and enables accurate detection of tiny targets. Specifically, we first design a policy module to filter modalities before the fusion module and select the relevant modality information for input into the subsequent network. Second, we introduce a fusion module based on the attention mechanism to integrate the complementary information of the two modalities and improve the detection performance of tiny targets. Furthermore, we employ an appropriate training strategy and loss functions to jointly train the policy module and other layers in AMSDet. The superiority of our proposed method is validated through extensive experiments on the RGBTDronePerson and VTUAV-det datasets, achieving excellent detection performance for drone-based tiny targets.

Multispectral imaging plays a critical role in a range of intelligent transportation applications, including advanced driver assistance systems (ADAS), traffic monitoring, and night vision. However, accurate visible and thermal (RGB-T) image registration poses a significant challenge due to the considerable modality differences. In this paper, we present a novel joint Self-Correlation and Cross-Correspondence Estimation Framework (SC³EF), leveraging both local representative features and global contextual cues to effectively generate RGB-T correspondences. For this purpose, we design a convolution-transformer-based pipeline to extract local representative features and encode global correlations of intra-modality for inter-modality correspondence estimation between unaligned visible and thermal images. After merging the local and global correspondence estimation results, we further employ a hierarchical optical flow estimation decoder to progressively refine the estimated dense correspondence maps. Extensive experiments demonstrate the effectiveness of our proposed method, outperforming the current state-of-the-art (SOTA) methods on representative RGB-T datasets. Furthermore, it also shows competitive generalization capabilities across challenging scenarios, including large parallax, severe occlusions, adverse weather, and other cross-modal datasets (e.g., RGB-N and RGB-D).

The fusion of far infrared (FIR) and visible images aims to generate a high-quality composite image that contains salient structures and abundant texture details for human visual perception. However, the existing fusion methods typically fall short of utilizing complementary source image characteristics to boost the features extracted from degraded visible or FIR images, thus they cannot generate satisfactory fusion results in adverse lighting or weather conditions. In this paper, we propose a novel Cross-Modal multispectral image Enhancement and Fusion framework (CMEFusion), which adaptively enhances both FIR and visible inputs by leveraging complementary crossmodal features to further facilitate multispectral feature aggregation. Specifically, we first present a new cross-modal image enhancement sub-network (CMIENet), which is built on a CNNTransformer hybrid architecture to perform the complementary exchange of local-salient and global-contextual features extracted from FIR and visible modalities, respectively. Then, we design a gradient-content differential fusion sub-network (GCDFNet) to progressively integrate decoupled gradient and content information via modified central difference convolution. Finally, we present a comprehensive joint enhancement-fusion multi-term loss function to drive the model to narrow the optimization gap betweentheabove-mentionedtwosub-networksbasedontheselfsupervised aspects of exposure, color, structure, and intensity. In this manner, the proposed CMEFusion model facilitates betterperforming visible and FIR image fusion in an end-to-end way, achieving enhanced visual quality with more natural and realistic appearances. Extensive experiments validate that CMEFusion surpasses state-of-the-art image fusion algorithms, as evidenced by superior performance in both visual quality and quantitative evaluations.

This paper presents an efficient factorial-design-based method to calculate uncertainty propagation in the triangulation process in stereo-vision photogrammetry. A three-level full factorial numerical simulation experiment is executed by regarding 3D measuring points as responses and image point pairs as factors. Hence, the covariance matrix of the 3D measuring point coordinate can be computed from the responses of the factorial experiment with corresponding weights. The measurement uncertainties in x, y, and z axis directions as well as the principal axis direction are computed from the covariance matrix of the 3D measuring point coordinate. The results calculated by the proposed method are compared with that of the Monte Carlo method, which shows that the proposed method can achieve equivalent precision with less computation cost. The spatial-variant uncertainty distribution of stereo-vision photogrammetry is analysed using the proposed method. Measurement uncertainties under different parameters of stereo-vision photogrammetric system are discussed.

Multispectral filter array (MSFA) sensors provide a cost-effective and one-shot acquisition solution to obtain well-aligned multi-band images, which are helpful for various optical and remote sensing applications. However, the sparse spatial sampling rate and strong spectral cross-correlation make MSFA image demosaicing a challenging problem. Therefore, it is essential to develop effective MSFA demosaicing solutions to reconstruct full-resolution and high-fidelity multispectral images from the raw mosaic image. In this paper, we present a Pseudo-panchromatic Image (PPI) Edge infused Spatial-Spectral Adaptive Residual Network (PPIE-SSARN) for multispectral filter array image demosaicing. The proposed two-branch model deploys a residual sub-branch to adaptively compensate for the spatial and spectral differences of reconstructed multispectral images and a PPI edge infusion sub-branch to enrich the edge-related information. Moreover, we design an effective mosaic initial feature extraction module with a spatial- and spectral-adaptive weight-sharing strategy whose kernel weights can change adaptively with spatial locations and spectral bands to avoid artifacts and aliasing problems. Experimental results demonstrate the superiority of our proposed method, outperforming the state-of-the-art MSFA demosaicing approaches and achieving satisfying demosaicing results in terms of spatial accuracy and spectral fidelity. Our models and code will be publicly available.

Fusion of images acquired using different sensors generates a single output with enhanced information for high-level visual perception applications. The transformer architecture has demonstrated its powerful ability to obtain important global contextual dependencies for multi-modal image fusion tasks. However, transformer-based image fusion methods face many critical issues, such as incurring huge computational burdens, limited ability to learn local features, and the difficulty of handling images of arbitrary sizes. To address the above limits, we proposed a novel Laplacian Pyramid Hybrid (LapH) network to combine the advantages of CNN and transformer architectures for multi-modal image fusion tasks. With the divide-and-conquer philosophy, we first build a light-weight CNN-based branch, performing effective extraction and fusion of texture/edge features via central difference convolutions, to process the high-resolution components with abundant details encoded in the lower pyramid levels of the Laplacian pyramid. Then, we design a transformer-based branch to process the low-resolution base components, learning long-range dependencies of global-contextual features without incurring extensive computational loads. Here, we design a multi-scale recurrent modulation mechanism to integrate the edge/texture features from the CNN branch as guidance to progressively refine the feature extraction and fusion on low-frequency components. Finally, we propose a new multi-scale spatial consistency loss term based on the neighbor contrast in source images, generating fused images with more natural and realistic appearances. Extensive experiments on two different multi-modal image fusion tasks verify the superiority of our method. The source codes are made publicly available at https://github.com/rgttadv/LapH.

Under the new geometric product specification (GPS), a two-dimensional chain cannot completely guarantee quality of the product. To optimize the allocation of three-dimensional tolerances in the conceptual design stage, the geometric variations of the tolerance zone to the deviation domain will be mapped in this paper. The deviation-processing cost, deviation-quality loss cost, and deviation-sensitivity cost function relationships combined with the tolerance zone described by the small displacement torsor theory are discussed. Then, synchronous constraint of the function structure and tolerance is realized. Finally, an improved bat algorithm is used to solve the established three-dimensional tolerance mathematical model. A case study in the optimization of three-part tolerance design is used to illustrate the proposed model and algorithms. The performance and advantage of the proposed method are discussed in the end.