Zhiyong Pan
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Zhiyong Pan has a total of 33 co-authors. Here are the top co-authors by number of shared publications:
- Gangyi Jiang
Ningbo University
4 shared publications - Mei Yu
Ningbo University
4 shared publications - Chaoqun Xia
College of Computer Science and Artificial Intelligence, Wenzhou University, Wenzhou 325035, China
2 shared publications - Xiaoqin Zhang
Zhejiang University of Technology, Hangzhou, China
2 shared publications - Yo-Sung Ho
School of Information and Communications, Gwangju Institute of Science and Technology (GIST), Gwangju, South Korea
2 shared publications - Zhao-Min Chen
College of Computer Science and Artificial Intelligence, Wenzhou University, Wenzhou, China
2 shared publications
Latest Publications
- 7 April 2026
- Published by Springer Nature in Applied Intelligence
- Vol. 56 (5) , 190
Parameter adaptive competitive differential evolution with local search
- 7 April 2026
- Published by Springer Nature in Applied Intelligence
- Vol. 56 (5) , 190
Parameter adaptive competitive differential evolution with local search
- Open Access
- 23 August 2023
- Published by IOP Publishing in Machine Learning: Science and Technology
- Vol. 4 (3) , 035047
DIM: long-tailed object detection and instance segmentation via dynamic instance memory
Object detection and instance segmentation have been successful on benchmarks with relatively balanced category distribution (e.g. MSCOCO). However, state-of-the-art object detection and segmentation methods still struggle to generalize on long-tailed datasets (e.g. LVIS), where a few classes (head classes) dominate the instance samples, while most classes (tailed classes) have only a few samples. To address this challenge, we propose a plug-and-play module within the Mask R-CNN framework called dynamic instance memory (DIM). Specifically, we augment Mask R-CNN with an auxiliary branch for training. It maintains a dynamic memory bank storing an instance-level prototype representation for each category, and shares the classifier with the existing instance branch. With a simple metric loss, the representations in DIM can be dynamically updated by the instance proposals in the mini-batch during training. Our DIM introduces a bias toward tailed classes to the classifier learning along with a class frequency reversed sampler, which learns generalizable representations from the original data distribution, complementing the existing instance branch. Comprehensive experiments on LVIS demonstrate the effectiveness of DIM, as well as the significant advantages of DIM over the baseline Mask R-CNN.
- 23 August 2023
- Published by IOP Publishing in Machine Learning: Science and Technology
- Vol. 4 (3) , 035047
Open AccessDIM: long-tailed object detection and instance segmentation via dynamic instance memory
Object detection and instance segmentation have been successful on benchmarks with relatively balanced category distribution (e.g. MSCOCO). However, state-of-the-art object detection and segmentation methods still struggle to generalize on long-tailed datasets (e.g. LVIS), where a few classes (head classes) dominate the instance samples, while most classes (tailed classes) have only a few samples. To address this challenge, we propose a plug-and-play module within the Mask R-CNN framework called dynamic instance memory (DIM). Specifically, we augment Mask R-CNN with an auxiliary branch for training. It maintains a dynamic memory bank storing an instance-level prototype representation for each category, and shares the classifier with the existing instance branch. With a simple metric loss, the representations in DIM can be dynamically updated by the instance proposals in the mini-batch during training. Our DIM introduces a bias toward tailed classes to the classifier learning along with a class frequency reversed sampler, which learns generalizable representations from the original data distribution, complementing the existing instance branch. Comprehensive experiments on LVIS demonstrate the effectiveness of DIM, as well as the significant advantages of DIM over the baseline Mask R-CNN.
- 3 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 60 (01962892) , 1-18
Infrared Small Target Detection via Dynamic Image Structure Evolution
Infrared small target detection (IRSTD) is a challenging task, due to the scarce target feature, complex background interferences, and poor image quality. The existing studies handle the IRSTD problem by making specific distribution assumptions of target and background, which incurs two issues. First, the specific assumptions of background cannot always hold. Second, the distorted target distributions contaminated by heavy clutters and noise are difficult to model statically. This article handles the problems from the discriminative and dynamic perspectives and proposes an original mechanism named dynamic image structure evolution (DISE) and a DISE-derived single-frame IRSTD framework. First, we resolve IRSTD by a discriminative model without assuming specific background distributions, which is based on a mathematical definition of structure singularity modeling the ideal target structure. Second, to decouple the target distributions from interferences, DISE guides the distorted target to reveal potential structure singularity, while suppressing the interference signal through iterative procedures of structure collapse, intensity settlement, and collapse convergence. The three functional procedures of DISE perform their own duties regarding target enhancement and background suppression. Moreover, an innovative chain mechanism is introduced to propagate the structure field. Experiments on real datasets demonstrate the superiority of DISE against the state-of-the-art IRSTD methods.
- 3 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 60 (01962892) , 1-18
Infrared Small Target Detection via Dynamic Image Structure Evolution
Infrared small target detection (IRSTD) is a challenging task, due to the scarce target feature, complex background interferences, and poor image quality. The existing studies handle the IRSTD problem by making specific distribution assumptions of target and background, which incurs two issues. First, the specific assumptions of background cannot always hold. Second, the distorted target distributions contaminated by heavy clutters and noise are difficult to model statically. This article handles the problems from the discriminative and dynamic perspectives and proposes an original mechanism named dynamic image structure evolution (DISE) and a DISE-derived single-frame IRSTD framework. First, we resolve IRSTD by a discriminative model without assuming specific background distributions, which is based on a mathematical definition of structure singularity modeling the ideal target structure. Second, to decouple the target distributions from interferences, DISE guides the distorted target to reveal potential structure singularity, while suppressing the interference signal through iterative procedures of structure collapse, intensity settlement, and collapse convergence. The three functional procedures of DISE perform their own duties regarding target enhancement and background suppression. Moreover, an innovative chain mechanism is introduced to propagate the structure field. Experiments on real datasets demonstrate the superiority of DISE against the state-of-the-art IRSTD methods.
- 13 July 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 70 (00189456) , 1-15
Pseudoreference Subaperture Images and Microlens Image-Based Blind Light Field Image Quality Measurement
Light field image (LFI) records the radiation intensity and direction information of light in scene, offering powerful capabilities for computer vision. It is found that the distortion caused by LFI processing is obviously reflected in the two representations of LFI, that is, subaperture images (SAIs) and microlens images (MLI). Specifically, on the one hand, pseudoreference SAI (PSAI) of a distorted SAI, which is rendered with the adjacent distorted SAIs, can be used to measure the distortion of the involved SAIs. On the other hand, macropixel blocks (MPBs) of MLI can be adopted to measure LFI global angular distortion. Therefore, a PSAIs- and MLI-based blind LFI quality measurement (PM-BLFIQM) method is proposed in this article. First, the local light field (LLF) is defined to generate PSAI of the central distorted SAI in each LLF of LFI, and then a pseudo-full-reference quality measurement metric based on PSAIs is presented. Second, to measure the angular consistency of the distorted LFI, an MLI-based feature extraction scheme is proposed for MPBs and their mean removal version by using discrete cosine transform and singular value decomposition. Finally, the quality of the distorted LFI is predicted with the features extracted from LLFs and MLI. Experimental results demonstrate that both PSAI and MPB are effective for measuring the quality of distorted LFI. In addition, the results on three LFI databases show that the proposed method achieves higher Spearman rank-order correlation coefficient (SROCC) and Pearson linear correlation coefficient (PLCC) values, so it outperforms the state-of-the-art methods in terms of consistency with human visual perception.
- 13 July 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 70 (00189456) , 1-15
Pseudoreference Subaperture Images and Microlens Image-Based Blind Light Field Image Quality Measurement
Light field image (LFI) records the radiation intensity and direction information of light in scene, offering powerful capabilities for computer vision. It is found that the distortion caused by LFI processing is obviously reflected in the two representations of LFI, that is, subaperture images (SAIs) and microlens images (MLI). Specifically, on the one hand, pseudoreference SAI (PSAI) of a distorted SAI, which is rendered with the adjacent distorted SAIs, can be used to measure the distortion of the involved SAIs. On the other hand, macropixel blocks (MPBs) of MLI can be adopted to measure LFI global angular distortion. Therefore, a PSAIs- and MLI-based blind LFI quality measurement (PM-BLFIQM) method is proposed in this article. First, the local light field (LLF) is defined to generate PSAI of the central distorted SAI in each LLF of LFI, and then a pseudo-full-reference quality measurement metric based on PSAIs is presented. Second, to measure the angular consistency of the distorted LFI, an MLI-based feature extraction scheme is proposed for MPBs and their mean removal version by using discrete cosine transform and singular value decomposition. Finally, the quality of the distorted LFI is predicted with the features extracted from LLFs and MLI. Experimental results demonstrate that both PSAI and MPB are effective for measuring the quality of distorted LFI. In addition, the results on three LFI databases show that the proposed method achieves higher Spearman rank-order correlation coefficient (SROCC) and Pearson linear correlation coefficient (PLCC) values, so it outperforms the state-of-the-art methods in terms of consistency with human visual perception.
- 3 February 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Signal Processing
- Vol. 15 (3) , 672-687
Combining Tensor Slice and Singular Value for Blind Light Field Image Quality Assessment
Light field image (LFI) collects radiance from rays in different directions, offers powerful capabilities for immersive media and computer vision. As the high-dimensional data, LFI suffers from spatial as well as angular information distortions in its processing, which brings new challenges to image quality assessment (IQA). Based on the strong ability of tensor about representing high-dimensional data and distortion characteristics of LFI, this paper proposes a method of combining tensor slice and singular value for blind light field image quality assessment (TSSV-LFIQA) to effectively evaluate the quality of LFI content. Specifically, five-order tensor representation of LFI is firstly defined which contains light ray intensity, angular information and color information of the LFI. Secondly, the first slice sharpness measurement and the other slice information distribution are used to describe the tensor slice spatial feature (TSSF) of the LFI. Moreover, singular value angular feature (SVAF) is also proposed to measure the angular consistency of LFI by further unfolding the five-order tensor of LFI and analyzing the percentage of singular values. The experimental results show that benefiting from the combination of TSSF and SVAF, the proposed TSSV-LFIQA method is statistically superior to the existing IQA methods, and matches well with human subjective opinions.
- 3 February 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Signal Processing
- Vol. 15 (3) , 672-687
Combining Tensor Slice and Singular Value for Blind Light Field Image Quality Assessment
Light field image (LFI) collects radiance from rays in different directions, offers powerful capabilities for immersive media and computer vision. As the high-dimensional data, LFI suffers from spatial as well as angular information distortions in its processing, which brings new challenges to image quality assessment (IQA). Based on the strong ability of tensor about representing high-dimensional data and distortion characteristics of LFI, this paper proposes a method of combining tensor slice and singular value for blind light field image quality assessment (TSSV-LFIQA) to effectively evaluate the quality of LFI content. Specifically, five-order tensor representation of LFI is firstly defined which contains light ray intensity, angular information and color information of the LFI. Secondly, the first slice sharpness measurement and the other slice information distribution are used to describe the tensor slice spatial feature (TSSF) of the LFI. Moreover, singular value angular feature (SVAF) is also proposed to measure the angular consistency of LFI by further unfolding the five-order tensor of LFI and analyzing the percentage of singular values. The experimental results show that benefiting from the combination of TSSF and SVAF, the proposed TSSV-LFIQA method is statistically superior to the existing IQA methods, and matches well with human subjective opinions.
- 1 February 2021
- Published by Elsevier in Journal of Visual Communication and Image Representation
- Vol. 75
Strong ghost removal in multi-exposure image fusion using hole-filling with exposure congruency
- 1 February 2021
- Published by Elsevier in Journal of Visual Communication and Image Representation
- Vol. 75
Strong ghost removal in multi-exposure image fusion using hole-filling with exposure congruency
- Open Access
- 21 January 2020
- Published by MDPI AG in Applied Sciences
- Vol. 10 (3) , 732
Image Registration Algorithm Based on Convolutional Neural Network and Local Homography Transformation
In order to overcome the poor robustness of traditional image registration algorithms in illuminating and solving the problem of low accuracy of a learning-based image homography matrix estimation algorithm, an image registration algorithm based on convolutional neural network (CNN) and local homography transformation is proposed. Firstly, to ensure the diversity of samples, a sample and label generation method based on moving direct linear transformation (MDLT) is designed. The generated samples and labels can effectively reflect the local characteristics of images and are suitable for training the CNN model with which multiple pairs of local matching points between two images to be registered can be calculated. Then, the local homography matrices between the two images are estimated by using the MDLT and finally the image registration can be realized. The experimental results show that the proposed image registration algorithm achieves higher accuracy than other commonly used algorithms such as the SIFT, ORB, ECC, and APAP algorithms, as well as another two learning-based algorithms, and it has good robustness for different types of illumination imaging.
- 21 January 2020
- Published by MDPI AG in Applied Sciences
- Vol. 10 (3) , 732
Open AccessImage Registration Algorithm Based on Convolutional Neural Network and Local Homography Transformation
In order to overcome the poor robustness of traditional image registration algorithms in illuminating and solving the problem of low accuracy of a learning-based image homography matrix estimation algorithm, an image registration algorithm based on convolutional neural network (CNN) and local homography transformation is proposed. Firstly, to ensure the diversity of samples, a sample and label generation method based on moving direct linear transformation (MDLT) is designed. The generated samples and labels can effectively reflect the local characteristics of images and are suitable for training the CNN model with which multiple pairs of local matching points between two images to be registered can be calculated. Then, the local homography matrices between the two images are estimated by using the MDLT and finally the image registration can be realized. The experimental results show that the proposed image registration algorithm achieves higher accuracy than other commonly used algorithms such as the SIFT, ORB, ECC, and APAP algorithms, as well as another two learning-based algorithms, and it has good robustness for different types of illumination imaging.
- Open Access
- 26 May 2017
- Published by MDPI AG in Applied Sciences
- Vol. 7 (6) , 526
Stereoscopic Image Super-Resolution Method with View Incorporation and Convolutional Neural Networks
Super-resolution (SR) plays an important role in the processing and display of mixed-resolution (MR) stereoscopic images. Therefore, a stereoscopic image SR method based on view incorporation and convolutional neural networks (CNN) is proposed. For a given MR stereoscopic image, the left view of which is observed in full resolution, while the right view is viewed in low resolution, the SR method is implemented in two stages. In the first stage, a view difference image is defined to represent the correlation between views. It is estimated by using the full-resolution left view and the interpolated right view as input to the modified CNN. Accordingly, a high-precision view difference image is obtained. In the second stage, to incorporate the estimated right view in the first stage, a global reconstruction constraint is presented to make the estimated right view consistent with the low-resolution right view in terms of the MR stereoscopic image observation model. Experimental results demonstrated that, compared with the SR convolutional neural network (SRCNN) method and depth map based SR method, the proposed method improved the reconstructed right view quality by 0.54 dB and 1.14 dB, respectively, in the Peak Signal to Noise Ratio (PSNR), and subjective evaluation also implied that the proposed method produced better reconstructed stereoscopic images.
- 26 May 2017
- Published by MDPI AG in Applied Sciences
- Vol. 7 (6) , 526
Open AccessStereoscopic Image Super-Resolution Method with View Incorporation and Convolutional Neural Networks
Super-resolution (SR) plays an important role in the processing and display of mixed-resolution (MR) stereoscopic images. Therefore, a stereoscopic image SR method based on view incorporation and convolutional neural networks (CNN) is proposed. For a given MR stereoscopic image, the left view of which is observed in full resolution, while the right view is viewed in low resolution, the SR method is implemented in two stages. In the first stage, a view difference image is defined to represent the correlation between views. It is estimated by using the full-resolution left view and the interpolated right view as input to the modified CNN. Accordingly, a high-precision view difference image is obtained. In the second stage, to incorporate the estimated right view in the first stage, a global reconstruction constraint is presented to make the estimated right view consistent with the low-resolution right view in terms of the MR stereoscopic image observation model. Experimental results demonstrated that, compared with the SR convolutional neural network (SRCNN) method and depth map based SR method, the proposed method improved the reconstructed right view quality by 0.54 dB and 1.14 dB, respectively, in the Peak Signal to Noise Ratio (PSNR), and subjective evaluation also implied that the proposed method produced better reconstructed stereoscopic images.