Jinpu Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Jinpu Zhang has a total of 24 co-authors. Here are the top co-authors by number of shared publications:
- Yuehuan Wang
Huazhong University of Science and Technology
8 shared publications - Ziwen Li
Huazhong University of Science and Technology
5 shared publications - Ruonan Wei
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China
3 shared publications - Dewen Hu
National university of defense technology, Changsha, China
2 shared publications - Zhen Tan
National University of Defense Technology, China
2 shared publications - Chenhao Li
National Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China
1 shared publication
Latest Publications
- Open Access
- 16 November 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
Overview of Deep Learning-based Visual Localization Methods
Visual localization seeks to estimate the six degrees of freedom (6-DoF) camera pose from a single input image, and has found extensive applications in augmented reality, mobile robotics, and autonomous driving. This paper provides an overview of the current landscape of visual localization. We present a comprehensive review of mainstream deep learning-based visual localization approaches. Organized around the central theme of data association, this review systematically examines three primary technical paradigms: (1) Absolute Pose Regression (APR), which employs a CNN to directly regress the camera pose from an image; (2) Scene Coordinate Regression (SCR), which predicts per-pixel 3D scene coordinates and subsequently estimates the pose using geometric solvers; and (3) Neural Rendering-based Localization, which refines poses through differentiable rendering and optimization, leveraging implicit neural representations or explicit geometric primitives. The review provides a thorough analysis of representative methods within each category. Finally, we discuss current challenges, including the limited geometric reasoning capability of APR, the scalability issues of SCR in large-scale environments, and the computational bottlenecks associated with neural rendering. This review aims to guide the advancement of robust, efficient, and scalable visual localization systems.
- 16 November 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
Open AccessOverview of Deep Learning-based Visual Localization Methods
Visual localization seeks to estimate the six degrees of freedom (6-DoF) camera pose from a single input image, and has found extensive applications in augmented reality, mobile robotics, and autonomous driving. This paper provides an overview of the current landscape of visual localization. We present a comprehensive review of mainstream deep learning-based visual localization approaches. Organized around the central theme of data association, this review systematically examines three primary technical paradigms: (1) Absolute Pose Regression (APR), which employs a CNN to directly regress the camera pose from an image; (2) Scene Coordinate Regression (SCR), which predicts per-pixel 3D scene coordinates and subsequently estimates the pose using geometric solvers; and (3) Neural Rendering-based Localization, which refines poses through differentiable rendering and optimization, leveraging implicit neural representations or explicit geometric primitives. The review provides a thorough analysis of representative methods within each category. Finally, we discuss current challenges, including the limited geometric reasoning capability of APR, the scalability issues of SCR in large-scale environments, and the computational bottlenecks associated with neural rendering. This review aims to guide the advancement of robust, efficient, and scalable visual localization systems.
- 24 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 75 (00189456) , 1-12
A Plug-and-Play Learning-Based IMU Bias Factor for Robust Visual–Inertial Odometry
Accurate and reliable estimation of biases of low-cost Inertial Measurement Units (IMU) is a key factor to maintain the resilience of Visual-Inertial Odometry (VIO), particularly when visual tracking fails in challenging areas. To address the issue of inaccurate estimation of low-cost IMU bias resulting from visual tracking failures, we propose an Inertial Prior Network (IPNet), a plug-and-play module that captures platform-specific motion characteristics directly from raw IMU measurements to infer bias priors. This approach eliminates the dependency on recursive bias estimation combining visual features, thus effectively preventing error propagation in challenging areas. Additionally, to compensate for the scarcity of ground-truth bias in most visual-inertial datasets, we introduce an iterative method to compute the mean IMU bias for each sequence to facilitate network training. Extensive experimental results on the EuRoC and TumVi public datasets, as well as an in-house dataset, demonstrate that the IPNet significantly enhances localization precision and robustness. Specifically,on the public benchmarks, the average improvements in ATE-RMSE and RPE-RMSE reached 46% and 48%, respectively. Moreover, the model’s cross-scene generalization is confirmed by successfully applying the indoor-trained prior network to outdoor autonomous driving scenarios. The source code is available at: https://github.com/yiyscut/VIO-IPNet.git.
- 24 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 75 (00189456) , 1-12
A Plug-and-Play Learning-Based IMU Bias Factor for Robust Visual–Inertial Odometry
Accurate and reliable estimation of biases of low-cost Inertial Measurement Units (IMU) is a key factor to maintain the resilience of Visual-Inertial Odometry (VIO), particularly when visual tracking fails in challenging areas. To address the issue of inaccurate estimation of low-cost IMU bias resulting from visual tracking failures, we propose an Inertial Prior Network (IPNet), a plug-and-play module that captures platform-specific motion characteristics directly from raw IMU measurements to infer bias priors. This approach eliminates the dependency on recursive bias estimation combining visual features, thus effectively preventing error propagation in challenging areas. Additionally, to compensate for the scarcity of ground-truth bias in most visual-inertial datasets, we introduce an iterative method to compute the mean IMU bias for each sequence to facilitate network training. Extensive experimental results on the EuRoC and TumVi public datasets, as well as an in-house dataset, demonstrate that the IPNet significantly enhances localization precision and robustness. Specifically,on the public benchmarks, the average improvements in ATE-RMSE and RPE-RMSE reached 46% and 48%, respectively. Moreover, the model’s cross-scene generalization is confirmed by successfully applying the indoor-trained prior network to outdoor autonomous driving scenarios. The source code is available at: https://github.com/yiyscut/VIO-IPNet.git.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28, 16250-16265
High-Quality Exposure Correction With Diffusion-Based Image Generation Priors
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28, 16250-16265
High-Quality Exposure Correction With Diffusion-Based Image Generation Priors
- 24 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 1461-1474
Augment One With Others: Generalizing to Unforeseen Variations for Visual Tracking
Unforeseen appearance variation is a challenging factor for visual tracking. This paper provides a novel solution from semantic data augmentation, which facilitates offline training of trackers for better generalization. We utilize existing samples to obtain knowledge to augment another in terms of diversity and hardness. First, we propose that the similarity matching space in Siamese-like models has class-agnostic transferability. Based on this, we design the Latent Augmentation (LaAug) to transfer relevant variations and suppress irrelevant ones between training similarity embeddings of different classes. Thus the model can generalize across a more diverse semantic distribution. Then, we propose the Semantic Interaction Mix (SIMix), which interacts moments between different feature samples to contaminate structure and texture attributes and retain other semantic attributes. SIMix simulates the occlusion and complements the training distribution with hard cases. The mixed features with adversarial perturbations can empirically enable the model against external environmental disturbances. Experiments on six challenging benchmarks demonstrate that three representative tracking models, i.e., SiamBAN, TransT and OSTrack, can be consistently improved by incorporating the proposed methods without extra parameters and inference cost The code will be released
- 24 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 1461-1474
Augment One With Others: Generalizing to Unforeseen Variations for Visual Tracking
Unforeseen appearance variation is a challenging factor for visual tracking. This paper provides a novel solution from semantic data augmentation, which facilitates offline training of trackers for better generalization. We utilize existing samples to obtain knowledge to augment another in terms of diversity and hardness. First, we propose that the similarity matching space in Siamese-like models has class-agnostic transferability. Based on this, we design the Latent Augmentation (LaAug) to transfer relevant variations and suppress irrelevant ones between training similarity embeddings of different classes. Thus the model can generalize across a more diverse semantic distribution. Then, we propose the Semantic Interaction Mix (SIMix), which interacts moments between different feature samples to contaminate structure and texture attributes and retain other semantic attributes. SIMix simulates the occlusion and complements the training distribution with hard cases. The mixed features with adversarial perturbations can empirically enable the model against external environmental disturbances. Experiments on six challenging benchmarks demonstrate that three representative tracking models, i.e., SiamBAN, TransT and OSTrack, can be consistently improved by incorporating the proposed methods without extra parameters and inference cost The code will be released
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 2574-2583
MFRGN: Multi-scale Feature Representation Generalization Network for Ground-to-Aerial Geo-localization
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 2574-2583
Open AccessMFRGN: Multi-scale Feature Representation Generalization Network for Ground-to-Aerial Geo-localization
- 28 December 2023
- Published by Springer Nature
- p. 185-197
A Dynamic Tracking Framework Based on Scene Perception
- 28 December 2023
- Published by Springer Nature
- p. 185-197
A Dynamic Tracking Framework Based on Scene Perception
- 7 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (6) , 5033-5048
Difficulty-Aware Dynamic Network for Lightweight Exposure Correction
Recently, deep learning-based methods have been successfully applied to the field of exposure correction. However, most of the existing methods treat different locations of an image in the same way, ignoring the inhomogeneous recovery difficulty and spatially-varying visual patterns in the image, which is sub-optimal and not perfectly efficient. In this paper, we propose a difficulty-aware dynamic network (DDNet) for lightweight exposure correction. Specifically, we propose a difficulty-aware strategy that determines the difficulty of feature patches according to a difficulty mask. Then, only the difficult patches are further refined instead of the whole features, which greatly reduces the overall computational complexity. Moreover, in order to achieve spatially-varying processing with a minimal computational burden, we design a spatial-aware dynamic convolution (SDConv), which is generated by predicting a set of basic kernels and a spatial-aware weight map. Benefiting from these designs, our method can strike a good trade-off between performance and complexity. Extensive experiments on several datasets demonstrate that our approach outperforms the state-of-the-art methods both qualitatively and quantitatively while requiring cheaper computational costs.
- 7 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (6) , 5033-5048
Difficulty-Aware Dynamic Network for Lightweight Exposure Correction
Recently, deep learning-based methods have been successfully applied to the field of exposure correction. However, most of the existing methods treat different locations of an image in the same way, ignoring the inhomogeneous recovery difficulty and spatially-varying visual patterns in the image, which is sub-optimal and not perfectly efficient. In this paper, we propose a difficulty-aware dynamic network (DDNet) for lightweight exposure correction. Specifically, we propose a difficulty-aware strategy that determines the difficulty of feature patches according to a difficulty mask. Then, only the difficult patches are further refined instead of the whole features, which greatly reduces the overall computational complexity. Moreover, in order to achieve spatially-varying processing with a minimal computational burden, we design a spatial-aware dynamic convolution (SDConv), which is generated by predicting a set of basic kernels and a spatial-aware weight map. Benefiting from these designs, our method can strike a good trade-off between performance and complexity. Extensive experiments on several datasets demonstrate that our approach outperforms the state-of-the-art methods both qualitatively and quantitatively while requiring cheaper computational costs.
- Open Access
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 7324-7334
Progressive Domain-style Translation for Nighttime Tracking
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 7324-7334
Open AccessProgressive Domain-style Translation for Nighttime Tracking
- 14 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Geoscience and Remote Sensing Letters
- Vol. 20 (1545598X) , 1-5
SMTN: Multidimensional Fusion and Time-Domain Coding for Object Tracking in Satellite Videos
Object tracking in satellite videos contains various small targets, such as cars and ships. However, since small targets always lack salient texture features, and have low contrast to the background, it is difficult to detect targets, and distinguish different target instances, which results in tracking failure. In this letter, we propose a Siamese multidimensional fusion and time domain coding network (SMTN) with an efficient attention-based multidimensional information fusion (MDF) module and a time domain information fusion (TDF) module. The MDF module fuses multi-scale template map and search map information to make the network focus on the target, thus improving the target discriminability. In the TDF module, the aggregated temporal information of previous frames is used to adjust the current frame response map through a local-sensing Metaformer module, which suppresses the response of similar interferences. Different from other temporal fusion methods in object tracking in satellite videos, the TDF enables video satellite trackers to be optimized end-to-end, and improves the tracking success rate. Experiments demonstrate that the proposed method outperforms state-of-the-art trackers.
- 14 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Geoscience and Remote Sensing Letters
- Vol. 20 (1545598X) , 1-5
SMTN: Multidimensional Fusion and Time-Domain Coding for Object Tracking in Satellite Videos
Object tracking in satellite videos contains various small targets, such as cars and ships. However, since small targets always lack salient texture features, and have low contrast to the background, it is difficult to detect targets, and distinguish different target instances, which results in tracking failure. In this letter, we propose a Siamese multidimensional fusion and time domain coding network (SMTN) with an efficient attention-based multidimensional information fusion (MDF) module and a time domain information fusion (TDF) module. The MDF module fuses multi-scale template map and search map information to make the network focus on the target, thus improving the target discriminability. In the TDF module, the aggregated temporal information of previous frames is used to adjust the current frame response map through a local-sensing Metaformer module, which suppresses the response of similar interferences. Different from other temporal fusion methods in object tracking in satellite videos, the TDF enables video satellite trackers to be optimized end-to-end, and improves the tracking success rate. Experiments demonstrate that the proposed method outperforms state-of-the-art trackers.
- 1 November 2021
- Published by Elsevier in Journal of Visual Communication and Image Representation
- Vol. 81
YOLSO: You Only Look Small Object
- 1 November 2021
- Published by Elsevier in Journal of Visual Communication and Image Representation
- Vol. 81
YOLSO: You Only Look Small Object