Mohan Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Mohan Zhang has a total of 5 co-authors. Here are the top co-authors by number of shared publications:
- Jing Liao
City University of Hong Kong
2 shared publications - Jinhui Yu
Zhejiang University
2 shared publications - Junsong Zhang
National Engineering Research Center for E‐learning Central China Normal University Wuhan China
1 shared publication - Kang Zhang
Hong Kong University of Science and Technology, Guangzhou, Guangzhou, China
1 shared publication - Zhitong Huang
City University of Hong Kong, Hong Kong, SAR, China
1 shared publication
Latest Publications
- Open Access
- 19 November 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 43 (6) , 1-11
LVCD: Reference-based Lineart Video Colorization with Diffusion Models
We propose the first video diffusion framework for reference-based lineart video colorization. Unlike previous works that rely solely on image generative models to colorize lineart frame by frame, our approach leverages a large-scale pretrained video diffusion model to generate colorized animation videos. This approach leads to more temporally consistent results and is better equipped to handle large motions. Firstly, we introduce Sketch-guided ControlNet which provides additional control to finetune an image-to-video diffusion model for controllable video synthesis, enabling the generation of animation videos conditioned on lineart. We then propose Reference Attention to facilitate the transfer of colors from the reference frame to other frames containing fast and expansive motions. Finally, we present a novel scheme for sequential sampling, incorporating the Overlapped Blending Module and Prev-Reference Attention , to extend the video diffusion model beyond its original fixed-length limitation for long video colorization. Both qualitative and quantitative results demonstrate that our method significantly outperforms state-of-the-art techniques in terms of frame and video quality, as well as temporal consistency. Moreover, our method is capable of generating high-quality, long temporal-consistent animation videos with large motions, which is not achievable in previous works. Our code and model are available at https://luckyhzt.github.io/lvcd.
- 19 November 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 43 (6) , 1-11
Open AccessLVCD: Reference-based Lineart Video Colorization with Diffusion Models
We propose the first video diffusion framework for reference-based lineart video colorization. Unlike previous works that rely solely on image generative models to colorize lineart frame by frame, our approach leverages a large-scale pretrained video diffusion model to generate colorized animation videos. This approach leads to more temporally consistent results and is better equipped to handle large motions. Firstly, we introduce Sketch-guided ControlNet which provides additional control to finetune an image-to-video diffusion model for controllable video synthesis, enabling the generation of animation videos conditioned on lineart. We then propose Reference Attention to facilitate the transfer of colors from the reference frame to other frames containing fast and expansive motions. Finally, we present a novel scheme for sequential sampling, incorporating the Overlapped Blending Module and Prev-Reference Attention , to extend the video diffusion model beyond its original fixed-length limitation for long video colorization. Both qualitative and quantitative results demonstrate that our method significantly outperforms state-of-the-art techniques in terms of frame and video quality, as well as temporal consistency. Moreover, our method is capable of generating high-quality, long temporal-consistent animation videos with large motions, which is not achievable in previous works. Our code and model are available at https://luckyhzt.github.io/lvcd.
- 1 December 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 28 (8) , 2926-2937
Deep Exemplar-Based Color Transfer for 3D Model
Recoloring 3D models is a challenging task that often requires professional knowledge and tedious manual efforts. In this article, we present the first deep-learning framework for exemplar-based 3D model recolor, which can automatically transfer the colors from a reference image to the 3D model texture. Our framework consists of two modules to solve two major challenges in the 3D color transfer. First, we propose a new feed-forward Color Transfer Network to achieve high-quality semantic-level color transfer by finding dense semantic correspondences between images. Second, considering 3D model constraints such as UV mapping, we design a novel 3D Texture Optimization Module which can generate a seamless and coherent texture by combining color transferred results rendered in multiple views. Experiments show that our method performs robustly and generalizes well to various kinds of models.
- 1 December 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 28 (8) , 2926-2937
Deep Exemplar-Based Color Transfer for 3D Model
Recoloring 3D models is a challenging task that often requires professional knowledge and tedious manual efforts. In this article, we present the first deep-learning framework for exemplar-based 3D model recolor, which can automatically transfer the colors from a reference image to the 3D model texture. Our framework consists of two modules to solve two major challenges in the 3D color transfer. First, we propose a new feed-forward Color Transfer Network to achieve high-quality semantic-level color transfer by finding dense semantic correspondences between images. Second, considering 3D model constraints such as UV mapping, we design a novel 3D Texture Optimization Module which can generate a seamless and coherent texture by combining color transferred results rendered in multiple views. Experiments show that our method performs robustly and generalizes well to various kinds of models.
- 22 November 2019
- Published by Springer Nature in Journal of Computer Science and Technology
- Vol. 34 (6) , 1294-1306
Artistic Augmentation of Photographs with Droplets
- 22 November 2019
- Published by Springer Nature in Journal of Computer Science and Technology
- Vol. 34 (6) , 1294-1306
Artistic Augmentation of Photographs with Droplets