Xiaoyan Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Xiaoyan Zhang has a total of 56 co-authors. Here are the top co-authors by number of shared publications:
Kap Luk Chan
Tolendata Singapore R&D Centre Private Limited, Singapore
13 shared publicationsMartin Constable
RMIT University, Vietnam
13 shared publicationsJinze Yu
The University of Tokyo
11 shared publicationsWang Junyan
University of Southern California
11 shared publications- Jianmin Jiang
Shenzhen University, Shenzhen, China
4 shared publications Gaoyang Tang
2 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 6445-6454
SG-Net: Semantic Guidance Network for Spectral Reconstruction from RGB Images
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 6445-6454
Open AccessSG-Net: Semantic Guidance Network for Spectral Reconstruction from RGB Images
- 11 August 2025
- Published by Elsevier in Neurocomputing
- Vol. 653
NSGHG: Neural surface guided generalizable human Gaussian splatting for sparse view synthesis
- 11 August 2025
- Published by Elsevier in Neurocomputing
- Vol. 653
NSGHG: Neural surface guided generalizable human Gaussian splatting for sparse view synthesis
- 10 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 31 (10) , 6719-6736
VAT: Visibility Aware Transformer for Fine-Grained Clothed Human Reconstruction
In order to reconstruct 3D clothed human with accurate fine-grained details from sparse views, we propose a deep cooperating two-level global to fine-grained reconstruction framework that constructs robust global geometry to guide fine-grained geometry learning. The core of the framework is a novel visibility aware Transformer VAT, which bridges the two-level reconstruction architecture by connecting its global encoder and fine-grained decoder with two pixel-aligned implicit functions, respectively. The global encoder fuses semantic features of multiple views to integrate global geometric features. In the fine-grained decoder, visibility aware attention mechanism is designed to efficiently fuse multi-view and multi-scale features for mining fine-grained geometric features. The global encoder and fine-grained decoder are connected by a global embeding module to form a deep cooperation in the two-level framework, which provides global geometric embedding as a query guidance for calculating visibility aware attention in the fine-grained decoder. In addition, to extract highly aligned multi-scale features for the two-level reconstruction architecture, we design an image feature extractor MSUNet, which establishes strong semantic connections between different scales at minimal cost. Our proposed framework is end-to-end trainable, with all modules jointly optimized. We validate the effectiveness of our framework on public benchmarks, and experimental results demonstrate that our method has significant advantages over state-of-the-art methods in terms of both fine-grained performance and generalization.
- 10 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 31 (10) , 6719-6736
VAT: Visibility Aware Transformer for Fine-Grained Clothed Human Reconstruction
In order to reconstruct 3D clothed human with accurate fine-grained details from sparse views, we propose a deep cooperating two-level global to fine-grained reconstruction framework that constructs robust global geometry to guide fine-grained geometry learning. The core of the framework is a novel visibility aware Transformer VAT, which bridges the two-level reconstruction architecture by connecting its global encoder and fine-grained decoder with two pixel-aligned implicit functions, respectively. The global encoder fuses semantic features of multiple views to integrate global geometric features. In the fine-grained decoder, visibility aware attention mechanism is designed to efficiently fuse multi-view and multi-scale features for mining fine-grained geometric features. The global encoder and fine-grained decoder are connected by a global embeding module to form a deep cooperation in the two-level framework, which provides global geometric embedding as a query guidance for calculating visibility aware attention in the fine-grained decoder. In addition, to extract highly aligned multi-scale features for the two-level reconstruction architecture, we design an image feature extractor MSUNet, which establishes strong semantic connections between different scales at minimal cost. Our proposed framework is end-to-end trainable, with all modules jointly optimized. We validate the effectiveness of our framework on public benchmarks, and experimental results demonstrate that our method has significant advantages over state-of-the-art methods in terms of both fine-grained performance and generalization.
- Open Access
- 23 July 2024
- Published by MDPI AG in Remote Sensing
- Vol. 16 (15) , 2692
DDSR: Degradation-Aware Diffusion Model for Spectral Reconstruction from RGB Images
The reconstruction of hyperspectral images (HSIs) from RGB images is an attractive low-cost approach to recover hyperspectral information. However, existing approaches focus on learning an end-to-end mapping of RGB images and their corresponding HSIs with neural networks, which makes it difficult to ensure generalization due to the fact that they are trained on data with a specific degradation process. As a new paradigm of generative models, the diffusion model has shown great potential in image restoration, especially in noisy contexts. To address the unstable generalization ability of end-to-end models while exploiting the powerful ability of the diffusion model, we propose a degradation-aware diffusion model. The degradation process from HSI to RGB is modeled as a combination of multiple degradation operators, which are used to guide the inverse process of the diffusion model by utilizing a degradation-aware correction. By integrating the degradation-aware correction to the diffusion model, we obtain an efficient solver for spectral reconstruction, which is robust to different degradation patterns. Experiment results on various public datasets demonstrate that our method achieves competitive performance and shows a promising generalization ability.
- 23 July 2024
- Published by MDPI AG in Remote Sensing
- Vol. 16 (15) , 2692
Open AccessDDSR: Degradation-Aware Diffusion Model for Spectral Reconstruction from RGB Images
The reconstruction of hyperspectral images (HSIs) from RGB images is an attractive low-cost approach to recover hyperspectral information. However, existing approaches focus on learning an end-to-end mapping of RGB images and their corresponding HSIs with neural networks, which makes it difficult to ensure generalization due to the fact that they are trained on data with a specific degradation process. As a new paradigm of generative models, the diffusion model has shown great potential in image restoration, especially in noisy contexts. To address the unstable generalization ability of end-to-end models while exploiting the powerful ability of the diffusion model, we propose a degradation-aware diffusion model. The degradation process from HSI to RGB is modeled as a combination of multiple degradation operators, which are used to guide the inverse process of the diffusion model by utilizing a degradation-aware correction. By integrating the degradation-aware correction to the diffusion model, we obtain an efficient solver for spectral reconstruction, which is robust to different degradation patterns. Experiment results on various public datasets demonstrate that our method achieves competitive performance and shows a promising generalization ability.
- 20 January 2024
- Published by Springer Nature
- p. 180-192
Video-Based Self-supervised Human Depth Estimation
- 20 January 2024
- Published by Springer Nature
- p. 180-192
Video-Based Self-supervised Human Depth Estimation
- 29 November 2023
- Published by Elsevier in Computer Vision and Image Understanding
- Vol. 239
BiPR-RL: Portrait relighting via bi-directional consistent deep reinforcement learning
- 29 November 2023
- Published by Elsevier in Computer Vision and Image Understanding
- Vol. 239
BiPR-RL: Portrait relighting via bi-directional consistent deep reinforcement learning
- 31 July 2023
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 83 (22) , 62089-62106
Non-linear integration of loss terms for improved new view synthesis
- 31 July 2023
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 83 (22) , 62089-62106
Non-linear integration of loss terms for improved new view synthesis
- 24 June 2023
- Published by Springer Nature in The Visual Computer
- Vol. 39 (8) , 3247-3257
Deformable patch embedding-based shift module-enhanced transformer for panoramic action recognition
- 24 June 2023
- Published by Springer Nature in The Visual Computer
- Vol. 39 (8) , 3247-3257
Deformable patch embedding-based shift module-enhanced transformer for panoramic action recognition
- Open Access
See Blue Sky: Deep Image Dehaze Using Paired and Unpaired Training Images
The issue of image haze removal has attracted wide attention in recent years. However, most existing haze removal methods cannot restore the scene with clear blue sky, since the color and texture information of the object in the original haze image is insufficient. To remedy this, we propose a cycle generative adversarial network to construct a novel end-to-end image dehaze model. We adopt outdoor image datasets to train our model, which includes a set of real-world unpaired image dataset and a set of paired image dataset to ensure that the generated images are close to the real scene. Based on the cycle structure, our model adds four different kinds of loss function to constrain the effect including adversarial loss, cycle consistency loss, photorealism loss and paired L1 loss. These four constraints can improve the overall quality of such degraded images for better visual appeal and ensure reconstruction of images to keep from distortion. The proposed model could remove the haze of images and also restore the sky of images to be clean and blue (like captured in a sunny weather).
Open AccessSee Blue Sky: Deep Image Dehaze Using Paired and Unpaired Training Images
The issue of image haze removal has attracted wide attention in recent years. However, most existing haze removal methods cannot restore the scene with clear blue sky, since the color and texture information of the object in the original haze image is insufficient. To remedy this, we propose a cycle generative adversarial network to construct a novel end-to-end image dehaze model. We adopt outdoor image datasets to train our model, which includes a set of real-world unpaired image dataset and a set of paired image dataset to ensure that the generated images are close to the real scene. Based on the cycle structure, our model adds four different kinds of loss function to constrain the effect including adversarial loss, cycle consistency loss, photorealism loss and paired L1 loss. These four constraints can improve the overall quality of such degraded images for better visual appeal and ensure reconstruction of images to keep from distortion. The proposed model could remove the haze of images and also restore the sky of images to be clean and blue (like captured in a sunny weather).
- Open Access
- 22 October 2021
- Published by Association for Computing Machinery (ACM)
- p. 466-470
3D Reconstruction of Sequence Images Based on Improved Features
Three-dimensional reconstruction refers to the establishment of a mathematical model suitable for computer representation and processing of three-dimensional objects. It is not only the basis of processing, operating and analyzing its nature in the computer environment, but also the key technology of establishing virtual reality to express the objective world in the computer. Therefore, 3D reconstruction is a common scientific problem and core technology in the fields of computer aided geometric design (CAGD), computer graphics (CG), computer animation, computer vision, medical image processing, scientific computing and virtual reality, digital media creation and so on. In recent years, although many algorithms for 3D reconstruction of sequence images have been proposed, the accuracy for 3D reconstruction is still the main problem. In this paper, a new method for 3D reconstruction of sequence images is proposed. Firstly, the Harris feature extraction algorithm is improved to extract more accurate feature points. Then the extracted feature points are tracked in the sequence images. Lastly, the estimated sample consistency algorithm is used for 3D reconstruction after sparse and dense reconstruction. The experiments demonstrate that the accuracy of 3D reconstruction of sequence images is improved.
- 22 October 2021
- Published by Association for Computing Machinery (ACM)
- p. 466-470
Open Access3D Reconstruction of Sequence Images Based on Improved Features
Three-dimensional reconstruction refers to the establishment of a mathematical model suitable for computer representation and processing of three-dimensional objects. It is not only the basis of processing, operating and analyzing its nature in the computer environment, but also the key technology of establishing virtual reality to express the objective world in the computer. Therefore, 3D reconstruction is a common scientific problem and core technology in the fields of computer aided geometric design (CAGD), computer graphics (CG), computer animation, computer vision, medical image processing, scientific computing and virtual reality, digital media creation and so on. In recent years, although many algorithms for 3D reconstruction of sequence images have been proposed, the accuracy for 3D reconstruction is still the main problem. In this paper, a new method for 3D reconstruction of sequence images is proposed. Firstly, the Harris feature extraction algorithm is improved to extract more accurate feature points. Then the extracted feature points are tracked in the sequence images. Lastly, the estimated sample consistency algorithm is used for 3D reconstruction after sparse and dense reconstruction. The experiments demonstrate that the accuracy of 3D reconstruction of sequence images is improved.