Saiping Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Saiping Zhang has a total of 16 co-authors. Here are the top co-authors by number of shared publications:
- Fuzheng Yang
Xidian University
5 shared publications - Dongyang Jin
School of Electronic engineering and Automation, Guilin University of Electronic Technology, China
4 shared publications - Xiao Huo
School of Telecommunication Engineering, Xidian University, Xian, China
4 shared publications - Shuai Wan
Northwestern Polytechnical University
2 shared publications Defa Wang
School of Electronics and Information, Northwestern Polytechnical University, Xi’an, China
1 shared publication- Fei Yang
Nankai University
1 shared publication
Latest Publications
- Open Access
- 6 September 2024
- Published by Elsevier in Neurocomputing
- Vol. 610
A slimmable framework for practical neural video compression
- 6 September 2024
- Published by Elsevier in Neurocomputing
- Vol. 610
Open AccessA slimmable framework for practical neural video compression
- 27 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 73 (00189456) , 1-18
Light Field Rectification Based on Relative Pose Estimation
Hand-held light field (LF) cameras have unique advantages in computer vision such as 3D scene reconstruction and depth estimation. However, the related applications are limited by the ultra-small baseline, leading to the extremely low depth resolution in reconstruction. To solve this problem, we propose to rectify LF to obtain a large baseline. Specifically, the proposed method aligns two LFs captured by two hand-held LF cameras with a random relative pose, and extracts the corresponding row-aligned sub-aperture images (SAIs) to obtain an LF with a large baseline. For an accurate rectification, a method for pose estimation is also proposed, where the relative rotation and translation between the two LF cameras are estimated. The proposed pose estimation minimizes the degree of freedom (DoF) in the LF-point-LF-point correspondence model and explicitly solves this model in a linear way. The proposed pose estimation outperforms the state-of-the-art algorithms by providing more accurate results to support rectification. The row-aligned SAIs and significantly improved depth resolution in 3D reconstruction demonstrate the effectiveness of the proposed LF rectification.
- 27 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 73 (00189456) , 1-18
Light Field Rectification Based on Relative Pose Estimation
Hand-held light field (LF) cameras have unique advantages in computer vision such as 3D scene reconstruction and depth estimation. However, the related applications are limited by the ultra-small baseline, leading to the extremely low depth resolution in reconstruction. To solve this problem, we propose to rectify LF to obtain a large baseline. Specifically, the proposed method aligns two LFs captured by two hand-held LF cameras with a random relative pose, and extracts the corresponding row-aligned sub-aperture images (SAIs) to obtain an LF with a large baseline. For an accurate rectification, a method for pose estimation is also proposed, where the relative rotation and translation between the two LF cameras are estimated. The proposed pose estimation minimizes the degree of freedom (DoF) in the LF-point-LF-point correspondence model and explicitly solves this model in a linear way. The proposed pose estimation outperforms the state-of-the-art algorithms by providing more accurate results to support rectification. The row-aligned SAIs and significantly improved depth resolution in 3D reconstruction demonstrate the effectiveness of the proposed LF rectification.
- 9 January 2023
- Published by Optica Publishing Group in Applied Optics
- Vol. 62 (2) , 481-491
Calibrating an unfocused plenoptic camera based on parameters grouping and the light field structure point
Accurately calibrating an unfocused plenoptic camera is essential to its applications. Rapid progress has been made in this area in the past decades. In this paper, detailed analysis is first performed toward the state-of-the-art projection model. Based on the analysis, parameters in the projection model are divided into two groups. Then, based on the parameter analysis, a new, to the best of our knowledge, form of the projection model, together with a new image feature light field structure point (LF-structure-point), is proposed. The LF-structure-point provides a completely non-redundant representation of the signal structure of the recorded light field raw data and induces a virtual space, “light field structure space,” which is related to the real physical space by a 3D-to-3D projective transformation. The extracting algorithm of the LF-structure-point is also presented. Finally, based on the 3D-to-3D projective transformation and parameter grouping, a simple two-step calibration method using the LF-structure-point as the input data is then proposed and achieves satisfactory experimental results.
- 9 January 2023
- Published by Optica Publishing Group in Applied Optics
- Vol. 62 (2) , 481-491
Calibrating an unfocused plenoptic camera based on parameters grouping and the light field structure point
Accurately calibrating an unfocused plenoptic camera is essential to its applications. Rapid progress has been made in this area in the past decades. In this paper, detailed analysis is first performed toward the state-of-the-art projection model. Based on the analysis, parameters in the projection model are divided into two groups. Then, based on the parameter analysis, a new, to the best of our knowledge, form of the projection model, together with a new image feature light field structure point (LF-structure-point), is proposed. The LF-structure-point provides a completely non-redundant representation of the signal structure of the recorded light field raw data and induces a virtual space, “light field structure space,” which is related to the real physical space by a 3D-to-3D projective transformation. The extracting algorithm of the LF-structure-point is also presented. Finally, based on the 3D-to-3D projective transformation and parameter grouping, a simple two-step calibration method using the LF-structure-point as the input data is then proposed and achieves satisfactory experimental results.
- Open Access
- 16 August 2022
- Published by MDPI AG in Applied Sciences
- Vol. 12 (16) , 8179
Variable Rate Point Cloud Attribute Compression with Non-Local Attention Optimization
Point clouds are widely used as representations of 3D objects and scenes in a number of applications, including virtual and mixed reality, autonomous driving, antiques reconstruction. To reduce the cost for transmitting and storing such data, this paper proposes an end-to-end learning-based point cloud attribute compression (PCAC) approach. The proposed network adopts a sparse convolution-based variational autoencoder (VAE) structure to compress the color attribute of point clouds. Considering the difficulty of stacked convolution operations in capturing long range dependencies, the attention mechanism is incorporated in which a non-local attention module is developed to capture the local and global correlations in both spatial and channel dimensions. Towards the practical application, an additional modulation network is offered to achieve the variable rate compression purpose in a single network, avoiding the memory cost of storing multiple networks for multiple bitrates. Our proposed method achieves state-of-the-art compression performance compared to other existing learning-based methods and further reduces the gap with the latest MPEG G-PCC reference software TMC13 version 14.
- 16 August 2022
- Published by MDPI AG in Applied Sciences
- Vol. 12 (16) , 8179
Open AccessVariable Rate Point Cloud Attribute Compression with Non-Local Attention Optimization
Point clouds are widely used as representations of 3D objects and scenes in a number of applications, including virtual and mixed reality, autonomous driving, antiques reconstruction. To reduce the cost for transmitting and storing such data, this paper proposes an end-to-end learning-based point cloud attribute compression (PCAC) approach. The proposed network adopts a sparse convolution-based variational autoencoder (VAE) structure to compress the color attribute of point clouds. Considering the difficulty of stacked convolution operations in capturing long range dependencies, the attention mechanism is incorporated in which a non-local attention module is developed to capture the local and global correlations in both spatial and channel dimensions. Towards the practical application, an additional modulation network is offered to achieve the variable rate compression purpose in a single network, avoiding the memory cost of storing multiple networks for multiple bitrates. Our proposed method achieves state-of-the-art compression performance compared to other existing learning-based methods and further reduces the gap with the latest MPEG G-PCC reference software TMC13 version 14.
- 26 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 31 (10577149) , 1641-1656
Relative Pose Estimation for Light Field Cameras Based on LF-Point-LF-Point Correspondence Model
In this paper, we propose a relative pose estimation algorithm for micro-lens array (MLA)-based conventional light field (LF) cameras. First, by employing the matched LF-point pairs, we establish the LF-point-LF-point correspondence model to represent the correlation between LF features of the same 3D scene point in a pair of LFs. Then, we employ the proposed correspondence model to estimate the relative camera pose, which includes a linear solution and a non-linear optimization on manifold. Unlike prior related algorithms, which estimated relative poses based on the recovered depths of scene points, we adopt the estimated disparities to avoid the inaccuracy in recovering depths due to the ultra-small baseline between sub-aperture images of LF cameras. Experimental results on both simulated and real scene data have demonstrated the effectiveness of the proposed algorithm compared with classical as well as state-of-art relative pose estimation algorithms.
- 26 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 31 (10577149) , 1641-1656
Relative Pose Estimation for Light Field Cameras Based on LF-Point-LF-Point Correspondence Model
In this paper, we propose a relative pose estimation algorithm for micro-lens array (MLA)-based conventional light field (LF) cameras. First, by employing the matched LF-point pairs, we establish the LF-point-LF-point correspondence model to represent the correlation between LF features of the same 3D scene point in a pair of LFs. Then, we employ the proposed correspondence model to estimate the relative camera pose, which includes a linear solution and a non-linear optimization on manifold. Unlike prior related algorithms, which estimated relative poses based on the recovered depths of scene points, we adopt the estimated disparities to avoid the inaccuracy in recovering depths due to the ultra-small baseline between sub-aperture images of LF cameras. Experimental results on both simulated and real scene data have demonstrated the effectiveness of the proposed algorithm compared with classical as well as state-of-art relative pose estimation algorithms.
- Open Access
- 22 April 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 75884-75893
A Corner Detection Method for Conventional Light Field Camera by Jointly Using Line-Features
Micro-lens array (MLA) based light field cameras have been commercialized and had a wide range of applications in various fields in recent years. To promote these applications, especially for 3D reconstruction, it is crucial to calibrate light field cameras. Among the calibration steps, it is of great importance to detect corners in checkerboard images and establish the point-to-point correspondence between the checkerboard and its images. However, almost all existing algorithms detect corners in the sub-aperture images instead of raw images for conventional light field cameras. In this paper, a corner detection algorithm that can recognize corners in the raw images of conventional light field cameras was proposed. Firstly, template matching was used to detect the 3D lines in the image space of the main lens. Then, the locations of the corners were obtained by calculating the intersection of 3D line segments and reprojecting it to the raw image again. Experimental results have demonstrated that the algorithm can still achieve good performance in actual datasets with blurred and low resolution micro lens image.
- 22 April 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 75884-75893
Open AccessA Corner Detection Method for Conventional Light Field Camera by Jointly Using Line-Features
Micro-lens array (MLA) based light field cameras have been commercialized and had a wide range of applications in various fields in recent years. To promote these applications, especially for 3D reconstruction, it is crucial to calibrate light field cameras. Among the calibration steps, it is of great importance to detect corners in checkerboard images and establish the point-to-point correspondence between the checkerboard and its images. However, almost all existing algorithms detect corners in the sub-aperture images instead of raw images for conventional light field cameras. In this paper, a corner detection algorithm that can recognize corners in the raw images of conventional light field cameras was proposed. Firstly, template matching was used to detect the 3D lines in the image space of the main lens. Then, the locations of the corners were obtained by calculating the intersection of 3D line segments and reprojecting it to the raw image again. Experimental results have demonstrated that the algorithm can still achieve good performance in actual datasets with blurred and low resolution micro lens image.
- Open Access
- 8 January 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 9655-9667
Spherical Lanczos Interpolation in Planar Projection or Format Conversions of Panoramic Videos
Panoramic videos provide users with an amazing experience with immersive and interactive viewing, and are now gaining global popularity. Since panoramic videos are spherical in nature while effectively processed in planar projection formats in applications, it is crucial to implement sphere-to-plane projection, where a good interpolation algorithm is of great importance regarding generating high quality planar perspective videos. Moreover, different planar projection formats adapt to different applications according to their own characteristics. It is common to convert one projection format to another, where interpolation algorithms also make a difference on performance. In this paper, an interpolation algorithm named spherical Lanczos (SLAN) is proposed to achieve advanced performance in sphere-to-plane projection or planar projection format conversions of panoramic videos. In SLAN, we fully consider panoramic videos are naturally spherical. Reference pixels are selected carefully, and the weights are calculated correctly. Experimental results demonstrate that the SLAN interpolation algorithm is beneficial for the whole processing chain of panoramic videos both with lossless compression coding and with loss compression coding in terms of the increase in various end-to-end PSNR and the decrease in BD_rate. The cubic projection (CBP) and adjusted cubemap projection (ACP) are also evaluated to illustrate that the proposed algorithm is applicable for all kinds of planar projection format due to their inevitable deformation.
- 8 January 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 9655-9667
Open AccessSpherical Lanczos Interpolation in Planar Projection or Format Conversions of Panoramic Videos
Panoramic videos provide users with an amazing experience with immersive and interactive viewing, and are now gaining global popularity. Since panoramic videos are spherical in nature while effectively processed in planar projection formats in applications, it is crucial to implement sphere-to-plane projection, where a good interpolation algorithm is of great importance regarding generating high quality planar perspective videos. Moreover, different planar projection formats adapt to different applications according to their own characteristics. It is common to convert one projection format to another, where interpolation algorithms also make a difference on performance. In this paper, an interpolation algorithm named spherical Lanczos (SLAN) is proposed to achieve advanced performance in sphere-to-plane projection or planar projection format conversions of panoramic videos. In SLAN, we fully consider panoramic videos are naturally spherical. Reference pixels are selected carefully, and the weights are calculated correctly. Experimental results demonstrate that the SLAN interpolation algorithm is beneficial for the whole processing chain of panoramic videos both with lossless compression coding and with loss compression coding in terms of the increase in various end-to-end PSNR and the decrease in BD_rate. The cubic projection (CBP) and adjusted cubemap projection (ACP) are also evaluated to illustrate that the proposed algorithm is applicable for all kinds of planar projection format due to their inevitable deformation.