Di Zhu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Di Zhu has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
- Yongbin Zheng
National University of Defense Technology
3 shared publications - Peng Sun
College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China
2 shared publications - Wanying Xu
National University of Defense Technology
2 shared publications - Dongxu Yang
National University of Defense Technology
1 shared publication - Dongxu Yang1 shared publication
Shengjian Bai
College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China
1 shared publication
Latest Publications
- Open Access
- 6 May 2023
- Published by MDPI AG in Remote Sensing
- Vol. 15 (9) , 2440
LPGAN: A LBP-Based Proportional Input Generative Adversarial Network for Image Fusion
Image fusion is the process of combining multiple input images from single or multiple imaging modalities into a fused image, which is expected to be more informative for human or machine perception as compared to any of the input images. In this paper, we propose a novel method based on deep learning for fusing infrared images and visible images, named the local binary pattern (LBP)-based proportional input generative adversarial network (LPGAN). In the image fusion task, the preservation of structural similarity and image gradient information is contradictory, and it is difficult for both to achieve good performance at the same time. To solve this problem, we innovatively introduce LBP into GANs, enabling the network to have stronger texture feature extraction and utilization capabilities, as well as anti-interference capabilities. In the feature extraction stage, we introduce a pseudo-Siamese network for the generator to extract the detailed features and the contrast features. At the same time, considering the characteristic distribution of different modal images, we propose a 1:4 scale input mode. Extensive experiments on the publicly available TNO dataset and CVC14 dataset show that the proposed method achieves the state-of-the-art performance. We also test the universality of LPGAN by fusing RGB and infrared images on the RoadScene dataset and medical images. In addition, LPGAN is applied to multi-spectral remote sensing image fusion. Both qualitative and quantitative experiments demonstrate that our LPGAN can not only achieve good structural similarity, but also retain richly detailed information.
- 6 May 2023
- Published by MDPI AG in Remote Sensing
- Vol. 15 (9) , 2440
Open AccessLPGAN: A LBP-Based Proportional Input Generative Adversarial Network for Image Fusion
Image fusion is the process of combining multiple input images from single or multiple imaging modalities into a fused image, which is expected to be more informative for human or machine perception as compared to any of the input images. In this paper, we propose a novel method based on deep learning for fusing infrared images and visible images, named the local binary pattern (LBP)-based proportional input generative adversarial network (LPGAN). In the image fusion task, the preservation of structural similarity and image gradient information is contradictory, and it is difficult for both to achieve good performance at the same time. To solve this problem, we innovatively introduce LBP into GANs, enabling the network to have stronger texture feature extraction and utilization capabilities, as well as anti-interference capabilities. In the feature extraction stage, we introduce a pseudo-Siamese network for the generator to extract the detailed features and the contrast features. At the same time, considering the characteristic distribution of different modal images, we propose a 1:4 scale input mode. Extensive experiments on the publicly available TNO dataset and CVC14 dataset show that the proposed method achieves the state-of-the-art performance. We also test the universality of LPGAN by fusing RGB and infrared images on the RoadScene dataset and medical images. In addition, LPGAN is applied to multi-spectral remote sensing image fusion. Both qualitative and quantitative experiments demonstrate that our LPGAN can not only achieve good structural similarity, but also retain richly detailed information.
- Open Access
- 6 April 2023
- Published by MDPI AG in Preprints.org
LPGAN: A LBP-based Proportional Input Generative Adversarial Network for Image Fusion
Image fusion is the process of combining multiple input images from single or multiple imaging modalities into a fused image, which is expected to be more informative for human or machine perception as compared to any of the input images. In this paper, we propose a novel method based on deep learning for fusing infrared images and visible images, named the LBP-based proportional input generative adversarial network (LPGAN). In the image fusion task, the preservation of structural similarity and image gradient information is contradictory, and it is difficult for both to achieve good performance at the same time. To solve this problem, we innovatively introduce Local Binary Patterns (LBP) into Generative Adversarial Networks (GANs), which effectively utilize the texture features of the source images, so that the network has stronger feature extraction ability and anti-interference ability. In the feature extraction stage, we introduce a pseudo-siamese network for the generator to extract the detailed features and the contrast features. At the same time, considering the characteristic distribution of different modal images, we propose a 1:4 scale input mode. Extensive experiments on the publicly available TNO dataset and CVC14 dataset show that the proposed method achieves the state-of-the-art performance. We also test the universality of LPGAN through the fusion of RGB and infrared images on the RoadScene dataset. In addition, LPGAN is applied to multi-spectral remote sensing image fusion. Both qualitative and quantitative experiments demonstrate that our LPGAN can not only achieve good structural similarity, but also retain rich detailed information.
- 6 April 2023
- Published by MDPI AG in Preprints.org
Open AccessLPGAN: A LBP-based Proportional Input Generative Adversarial Network for Image Fusion
Image fusion is the process of combining multiple input images from single or multiple imaging modalities into a fused image, which is expected to be more informative for human or machine perception as compared to any of the input images. In this paper, we propose a novel method based on deep learning for fusing infrared images and visible images, named the LBP-based proportional input generative adversarial network (LPGAN). In the image fusion task, the preservation of structural similarity and image gradient information is contradictory, and it is difficult for both to achieve good performance at the same time. To solve this problem, we innovatively introduce Local Binary Patterns (LBP) into Generative Adversarial Networks (GANs), which effectively utilize the texture features of the source images, so that the network has stronger feature extraction ability and anti-interference ability. In the feature extraction stage, we introduce a pseudo-siamese network for the generator to extract the detailed features and the contrast features. At the same time, considering the characteristic distribution of different modal images, we propose a 1:4 scale input mode. Extensive experiments on the publicly available TNO dataset and CVC14 dataset show that the proposed method achieves the state-of-the-art performance. We also test the universality of LPGAN through the fusion of RGB and infrared images on the RoadScene dataset. In addition, LPGAN is applied to multi-spectral remote sensing image fusion. Both qualitative and quantitative experiments demonstrate that our LPGAN can not only achieve good structural similarity, but also retain rich detailed information.
- 26 September 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 22 (22) , 21827-21836
An Accurate Line-of-Sight Rate Estimation Method Based on LSTM Recurrent Neural Network for Strapdown Imaging Seeker
Obtaining accurate line-of-sight (LOS) rates is a challenging task for strapdown imaging seekers, which are important sensor systems for unmanned aerial vehicles, missiles or other aircrafts. The traditional Kalman filter and its improved algorithm are difficult to deal with the non-Gaussian noise. To this end, we design a model-driven and data-driven filter using long short-term memory (LSTM) recurrent neural network. A decoupling algorithm is proposed to isolate LOS information from the attitude angle and generate one-step predictions. Predictions and observations are used as input for supervised learning, which solves the problem of difficult to describe noise features. Finally, we design a simulation experiment to compare the proposed LSTM model with extended Kalman filter (EKF), unscented Kalman filtering (UKF) and particle filter (PF) on the task of estimating the LOS angle and LOS rate. The experimental results show that the proposed LSTM model is better than the other three methods in terms of accuracy.
- 26 September 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 22 (22) , 21827-21836
An Accurate Line-of-Sight Rate Estimation Method Based on LSTM Recurrent Neural Network for Strapdown Imaging Seeker
Obtaining accurate line-of-sight (LOS) rates is a challenging task for strapdown imaging seekers, which are important sensor systems for unmanned aerial vehicles, missiles or other aircrafts. The traditional Kalman filter and its improved algorithm are difficult to deal with the non-Gaussian noise. To this end, we design a model-driven and data-driven filter using long short-term memory (LSTM) recurrent neural network. A decoupling algorithm is proposed to isolate LOS information from the attitude angle and generate one-step predictions. Predictions and observations are used as input for supervised learning, which solves the problem of difficult to describe noise features. Finally, we design a simulation experiment to compare the proposed LSTM model with extended Kalman filter (EKF), unscented Kalman filtering (UKF) and particle filter (PF) on the task of estimating the LOS angle and LOS rate. The experimental results show that the proposed LSTM model is better than the other three methods in terms of accuracy.