Tao Yang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Tao Yang has a total of 21 co-authors. Here are the top co-authors by number of shared publications:
- Lei Zhang
Northwestern Polytechnical University, Xi'an, China
2 shared publications Peiran Ren
Alibaba Group
2 shared publications- Xuansong Xie
Alibaba Group
2 shared publications - Bo Ren
Nankai University
1 shared publication - Chaofeng Chen
School of Artificial Intelligence, Wuhan University, Wuhan, China
1 shared publication - Chenfeng Li
Zienkiewicz Institute for Modelling, Data and AI, Swansea University
1 shared publication
Latest Publications
- Top Cited Papers
- 1 November 2024
- Published by Springer Nature
- p. 74-91
Pixel-Aware Stable Diffusion for Realistic Image Super-Resolution and Personalized Stylization
- 1 November 2024
- Published by Springer Nature
- p. 74-91
Top Cited PapersPixel-Aware Stable Diffusion for Realistic Image Super-Resolution and Personalized Stylization
- Open Access
Synthesizing Realistic Image Restoration Training Pairs: A Diffusion Approach
In supervised image restoration tasks, one key issue is how to obtain the aligned high-quality (HQ) and low-quality (LQ) training image pairs. Unfortunately, such HQ-LQ training pairs are hard to capture in practice, and hard to synthesize due to the complex unknown degradation in the wild. While several sophisticated degradation models have been manually designed to synthesize LQ images from their HQ counterparts, the distribution gap between the synthesized and real-world LQ images remains large. We propose a new approach to synthesizing realistic image restoration training pairs using the emerging denoising diffusion probabilistic model (DDPM). First, we train a DDPM, which could convert a noisy input into the desired LQ image, with a large amount of collected LQ images, which define the target data distribution. Then, for a given HQ image, we synthesize an initial LQ image by using an off-the-shelf degradation model, and iteratively add proper Gaussian noises to it. Finally, we denoise the noisy LQ image using the pre-trained DDPM to obtain the final LQ image, which falls into the target distribution of real-world LQ images. Thanks to the strong capability of DDPM in distribution approximation, the synthesized HQ-LQ image pairs can be used to train robust models for real-world image restoration tasks, such as blind face image restoration and blind image super-resolution. Experiments demonstrated the superiority of our proposed approach to existing degradation models. Code and data will be released.
Open AccessSynthesizing Realistic Image Restoration Training Pairs: A Diffusion Approach
In supervised image restoration tasks, one key issue is how to obtain the aligned high-quality (HQ) and low-quality (LQ) training image pairs. Unfortunately, such HQ-LQ training pairs are hard to capture in practice, and hard to synthesize due to the complex unknown degradation in the wild. While several sophisticated degradation models have been manually designed to synthesize LQ images from their HQ counterparts, the distribution gap between the synthesized and real-world LQ images remains large. We propose a new approach to synthesizing realistic image restoration training pairs using the emerging denoising diffusion probabilistic model (DDPM). First, we train a DDPM, which could convert a noisy input into the desired LQ image, with a large amount of collected LQ images, which define the target data distribution. Then, for a given HQ image, we synthesize an initial LQ image by using an off-the-shelf degradation model, and iteratively add proper Gaussian noises to it. Finally, we denoise the noisy LQ image using the pre-trained DDPM to obtain the final LQ image, which falls into the target distribution of real-world LQ images. Thanks to the strong capability of DDPM in distribution approximation, the synthesized HQ-LQ image pairs can be used to train robust models for real-world image restoration tasks, such as blind face image restoration and blind image super-resolution. Experiments demonstrated the superiority of our proposed approach to existing degradation models. Code and data will be released.
- 14 February 2023
- Published by Springer Nature
- p. 738-755
Beyond a Video Frame Interpolator: A Space Decoupled Learning Approach to Continuous Image Transition
- 14 February 2023
- Published by Springer Nature
- p. 738-755
Beyond a Video Frame Interpolator: A Space Decoupled Learning Approach to Continuous Image Transition
- Open Access
Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors
A key challenge of real-world image super-resolution (SR) is to recover the missing details in low-resolution (LR) images with complex unknown degradations (e.g., downsampling, noise and compression). Most previous works restore such missing details in the image space. To cope with the high diversity of natural images, they either rely on the unstable GANs that are difficult to train and prone to artifacts, or resort to explicit references from high-resolution (HR) images that are usually unavailable. In this work, we propose Feature Matching SR (FeMaSR), which restores realistic HR images in a much more compact feature space. Unlike image-space methods, our FeMaSR restores HR images by matching distorted LR image {\it features} to their distortion-free HR counterparts in our pretrained HR priors, and decoding the matched features to obtain realistic HR images. Specifically, our HR priors contain a discrete feature codebook and its associated decoder, which are pretrained on HR images with a Vector Quantized Generative Adversarial Network (VQGAN). Notably, we incorporate a novel semantic regularization in VQGAN to improve the quality of reconstructed images. For the feature matching, we first extract LR features with an LR encoder consisting of several Swin Transformer blocks and then follow a simple nearest neighbour strategy to match them with the pretrained codebook. In particular, we equip the LR encoder with residual shortcut connections to the decoder, which is critical to the optimization of feature matching loss and also helps to complement the possible feature matching errors. Experimental results show that our approach produces more realistic HR images than previous methods. Codes are released at \url{https://github.com/chaofengc/FeMaSR}.
Open AccessReal-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors
A key challenge of real-world image super-resolution (SR) is to recover the missing details in low-resolution (LR) images with complex unknown degradations (e.g., downsampling, noise and compression). Most previous works restore such missing details in the image space. To cope with the high diversity of natural images, they either rely on the unstable GANs that are difficult to train and prone to artifacts, or resort to explicit references from high-resolution (HR) images that are usually unavailable. In this work, we propose Feature Matching SR (FeMaSR), which restores realistic HR images in a much more compact feature space. Unlike image-space methods, our FeMaSR restores HR images by matching distorted LR image {\it features} to their distortion-free HR counterparts in our pretrained HR priors, and decoding the matched features to obtain realistic HR images. Specifically, our HR priors contain a discrete feature codebook and its associated decoder, which are pretrained on HR images with a Vector Quantized Generative Adversarial Network (VQGAN). Notably, we incorporate a novel semantic regularization in VQGAN to improve the quality of reconstructed images. For the feature matching, we first extract LR features with an LR encoder consisting of several Swin Transformer blocks and then follow a simple nearest neighbour strategy to match them with the pretrained codebook. In particular, we equip the LR encoder with residual shortcut connections to the decoder, which is critical to the optimization of feature matching loss and also helps to complement the possible feature matching errors. Experimental results show that our approach produces more realistic HR images than previous methods. Codes are released at \url{https://github.com/chaofengc/FeMaSR}.
- 1 January 2016
- Published by International Press of Boston in Communications in Information and Systems
- Vol. 16 (3) , 127-146
Solid deformation by material point method
International Press of Boston - publishers of scholarly mathematical and scientific journals and books
- 1 January 2016
- Published by International Press of Boston in Communications in Information and Systems
- Vol. 16 (3) , 127-146
Solid deformation by material point method
International Press of Boston - publishers of scholarly mathematical and scientific journals and books
- 28 April 2015
- Published by Springer Nature in The Visual Computer
- Vol. 32 (4) , 523-534
Fast SPH simulation for gaseous fluids
- 28 April 2015
- Published by Springer Nature in The Visual Computer
- Vol. 32 (4) , 523-534
Fast SPH simulation for gaseous fluids