Xinyuan Zhao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Xinyuan Zhao has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
- Wei Zhao
Huawei Technologies, Shenzhen, China
2 shared publications - Fei Yin
Tsinghua University
1 shared publication - Jiahao Wang
University of Hong Kong
1 shared publication - Ran Yu
Meituan, Beijing, China
1 shared publication - Weihao Xia
University of Cambridge
1 shared publication - Wenhe Long
Shenzhen International Graduate School, Tsinghua University Shenzhen International Graduate School, China
1 shared publication
Latest Publications
- 21 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 47 (5) , 3452-3468
Learning High-Quality Dynamic Memory for Video Object Segmentation
Recently, several spatial-temporal memory-based methods have verified that storing intermediate frames with masks as memory helps segment target objects in videos. However, they mainly focus on better matching between the current frame and memory frames without paying attention to the quality of the memory. Consequently, frames with poor segmentation masks may be memorized, leading to error accumulation problems. Besides, the linear increase of memory frames with the growth of frame numbers limits the ability of the models to handle long videos. To this end, we propose a Quality-aware Dynamic Memory Network (QDMN) to evaluate the segmentation quality of each frame, allowing the memory bank to selectively store accurately segmented frames and prevent error accumulation. Then, we combine the segmentation quality with temporal consistency to dynamically update the memory bank and make the models can handle videos of arbitrary length. The above operation ensures the reliability of memory frames and improves the quality of memory at the frame level. Moreover, we observe that the memory features extracted from reliable frames still contain noise and have limited representation capabilities. To address this problem, we propose to perform memory enhancement and anchoring on the basis of QDMN to improve the quality of memory from the feature level, resulting in a more robust and effective network QDMN++. Our method achieves state-of-the-art performance on all popular benchmarks. Moreover, extensive experiments demonstrate that the proposed memory screening mechanism can be applied to any memory-based methods as generic plugins.
- 21 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 47 (5) , 3452-3468
Learning High-Quality Dynamic Memory for Video Object Segmentation
Recently, several spatial-temporal memory-based methods have verified that storing intermediate frames with masks as memory helps segment target objects in videos. However, they mainly focus on better matching between the current frame and memory frames without paying attention to the quality of the memory. Consequently, frames with poor segmentation masks may be memorized, leading to error accumulation problems. Besides, the linear increase of memory frames with the growth of frame numbers limits the ability of the models to handle long videos. To this end, we propose a Quality-aware Dynamic Memory Network (QDMN) to evaluate the segmentation quality of each frame, allowing the memory bank to selectively store accurately segmented frames and prevent error accumulation. Then, we combine the segmentation quality with temporal consistency to dynamically update the memory bank and make the models can handle videos of arbitrary length. The above operation ensures the reliability of memory frames and improves the quality of memory at the frame level. Moreover, we observe that the memory features extracted from reliable frames still contain noise and have limited representation capabilities. To address this problem, we propose to perform memory enhancement and anchoring on the basis of QDMN to improve the quality of memory from the feature level, resulting in a more robust and effective network QDMN++. Our method achieves state-of-the-art performance on all popular benchmarks. Moreover, extensive experiments demonstrate that the proposed memory screening mechanism can be applied to any memory-based methods as generic plugins.
- Open Access
- 3 March 2023
- Published by Association for Computing Machinery (ACM)
- p. 60-65
Image Colorization with Fast Fourier Convolution
We present a spectrum-based framework, utilizing low and high spectral latent features, to colorize the grayscale image. Current image colorization algorithms, despite significant achievements, constantly struggle with color bleeding and low color saturation. One of the main reasons causing such phenomena is vanilla convolutions in modern neural networks focus on local and fixed scales. To alleviate this issue, we propose a novel approach called Low and High Spectral Colorization(LoHiSC). LoHiSC is based on a new colorization network that uses fast fourier convolution(FFC), which is capable of efficiently connecting two distant locations and has a global receptive field. Particularly, our proposed LoHiSC improves FFC by splitting it into two main branches, one is cascaded with low pass spectral filter, and another with high pass spectral filter, for the reason we experimently find that colors are mainly concentrated in the low frequency part, while contours and edges in the high frequency part. Moreover, we propose a frequency domain loss which is beneficial to restore challenging artifacts of grayscale image colorization. Experiments on both synthetic dataset and real-world old photos demonstrate our method improves the stat-of-the-art and achieves compelling colorizing results.
- 3 March 2023
- Published by Association for Computing Machinery (ACM)
- p. 60-65
Open AccessImage Colorization with Fast Fourier Convolution
We present a spectrum-based framework, utilizing low and high spectral latent features, to colorize the grayscale image. Current image colorization algorithms, despite significant achievements, constantly struggle with color bleeding and low color saturation. One of the main reasons causing such phenomena is vanilla convolutions in modern neural networks focus on local and fixed scales. To alleviate this issue, we propose a novel approach called Low and High Spectral Colorization(LoHiSC). LoHiSC is based on a new colorization network that uses fast fourier convolution(FFC), which is capable of efficiently connecting two distant locations and has a global receptive field. Particularly, our proposed LoHiSC improves FFC by splitting it into two main branches, one is cascaded with low pass spectral filter, and another with high pass spectral filter, for the reason we experimently find that colors are mainly concentrated in the low frequency part, while contours and edges in the high frequency part. Moreover, we propose a frequency domain loss which is beneficial to restore challenging artifacts of grayscale image colorization. Experiments on both synthetic dataset and real-world old photos demonstrate our method improves the stat-of-the-art and achieves compelling colorizing results.