Haibo Chen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Haibo Chen has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Jian Yang
Nanjing University of Science and Technology, Nanjing, China
3 shared publications - Jun Li
Southeast University
3 shared publications - Lei Zhao
College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
3 shared publications Lei Zhao
2 shared publicationsDaoLei Liang
School of Sciences, ZheJiang Sci Tech University, Hangzhou, China
1 shared publicationJian Yang
1 shared publication
Latest Publications
- 6 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 36 (1) , 133-146
ConceptCraft: One-Shot Personalized Text-to-Image Generation via Object-Background Disentanglement
Personalized text-to-image generation aims to learn new concepts from user-provided images and subsequently generate diverse scenes or styles of the concepts from input prompts. Most existing methods usually require a set of images (typically 3-5) for each concept, which can be cumbersome. Although several methods allow personalized generation with a single reference image, they often require heavy model training and suffer from many issues such as domain-specific applicability, insufficient fidelity, and limited editability. To address these problems, we propose a novel one-shot personalized text-to-image generation method called ConceptCraft, which explicitly separates the reference image into object and background regions and treats them as two distinct concepts to learn, significantly improving the personalization performance. Specifically, we incorporate two unique identifiers into the text prompts: one followed by the object’s class name and the other by the word “background”. To bind these two identifiers to the reference image’s object and background respectively, we introduce a mask-aware object preservation loss and a mask-aware background preservation loss to optimize their corresponding token embeddings under well-designed text conditions, enabling both object and background personalization. In addition, we also develop an identifier regularization scheme to enhance our editability, allowing the synthesis of personalized images across a broader range of scenes and styles without changing the identity. Extensive qualitative and quantitative experiments are conducted to verify the effectiveness and superiority of our method.
- 6 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 36 (1) , 133-146
ConceptCraft: One-Shot Personalized Text-to-Image Generation via Object-Background Disentanglement
Personalized text-to-image generation aims to learn new concepts from user-provided images and subsequently generate diverse scenes or styles of the concepts from input prompts. Most existing methods usually require a set of images (typically 3-5) for each concept, which can be cumbersome. Although several methods allow personalized generation with a single reference image, they often require heavy model training and suffer from many issues such as domain-specific applicability, insufficient fidelity, and limited editability. To address these problems, we propose a novel one-shot personalized text-to-image generation method called ConceptCraft, which explicitly separates the reference image into object and background regions and treats them as two distinct concepts to learn, significantly improving the personalization performance. Specifically, we incorporate two unique identifiers into the text prompts: one followed by the object’s class name and the other by the word “background”. To bind these two identifiers to the reference image’s object and background respectively, we introduce a mask-aware object preservation loss and a mask-aware background preservation loss to optimize their corresponding token embeddings under well-designed text conditions, enabling both object and background personalization. In addition, we also develop an identifier regularization scheme to enhance our editability, allowing the synthesis of personalized images across a broader range of scenes and styles without changing the identity. Extensive qualitative and quantitative experiments are conducted to verify the effectiveness and superiority of our method.
- 23 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 34 (10577149) , 759-771
TRTST: Arbitrary High-Quality Text-Guided Style Transfer With Transformers
Text-guided style transfer aims to repaint a content image with the target style described by a text prompt, offering greater flexibility and creativity compared to traditional image-guided style transfer. Despite the potential, existing text-guided style transfer methods often suffer from many issues, including insufficient visual quality, poor generalization ability, or a reliance on large amounts of paired training data. To address these limitations, we leverage the inherent strengths of transformers in handling multimodal data and propose a novel transformer-based framework called TRTST that not only achieves unpaired arbitrary text-guided style transfer but also significantly improves the visual quality. Specifically, TRTST explores combining a text transformer encoder with an image transformer encoder to project the input text prompt and content image into a joint embedding space and extract the desired style and content features. These features are then input into a multimodal co-attention module to stylize the image sequence based on the text sequence. We also propose a new adaptive parametric positional encoding (APPE) scheme which can adaptively produce different positional encodings to optimally match different inputs with a position encoder. In addition, to further improve content preservation, we introduce a text-guided identity loss to our model. Extensive results and comparisons are conducted to demonstrate the effectiveness and superiority of our method.
- 23 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 34 (10577149) , 759-771
TRTST: Arbitrary High-Quality Text-Guided Style Transfer With Transformers
Text-guided style transfer aims to repaint a content image with the target style described by a text prompt, offering greater flexibility and creativity compared to traditional image-guided style transfer. Despite the potential, existing text-guided style transfer methods often suffer from many issues, including insufficient visual quality, poor generalization ability, or a reliance on large amounts of paired training data. To address these limitations, we leverage the inherent strengths of transformers in handling multimodal data and propose a novel transformer-based framework called TRTST that not only achieves unpaired arbitrary text-guided style transfer but also significantly improves the visual quality. Specifically, TRTST explores combining a text transformer encoder with an image transformer encoder to project the input text prompt and content image into a joint embedding space and extract the desired style and content features. These features are then input into a multimodal co-attention module to stylize the image sequence based on the text sequence. We also propose a new adaptive parametric positional encoding (APPE) scheme which can adaptively produce different positional encodings to optimally match different inputs with a position encoder. In addition, to further improve content preservation, we introduce a text-guided identity loss to our model. Extensive results and comparisons are conducted to demonstrate the effectiveness and superiority of our method.
- 30 December 2024
- Published by Elsevier in Neurocomputing
- Vol. 620
PaintDiffusion: Towards text-driven painting variation via collaborative diffusion guidance
- 30 December 2024
- Published by Elsevier in Neurocomputing
- Vol. 620
PaintDiffusion: Towards text-driven painting variation via collaborative diffusion guidance
- 20 November 2024
- Published by Elsevier in Computer Vision and Image Understanding
- Vol. 251
UATST: Towards unpaired arbitrary text-guided style transfer with cross-space modulation
- 20 November 2024
- Published by Elsevier in Computer Vision and Image Understanding
- Vol. 251
UATST: Towards unpaired arbitrary text-guided style transfer with cross-space modulation
- Open Access
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 6878-6887
TSSAT: Two-Stage Statistics-Aware Transformation for Artistic Style Transfer
Artistic style transfer aims to create new artistic images by rendering a given photograph with the target artistic style. Existing methods learn styles simply based on global statistics or local patches, lacking careful consideration of the drawing process in practice. Consequently, the stylization results either fail to capture abundant and diversified local style patterns, or contain undesired semantic information of the style image and deviate from the global style distribution. To address this issue, we imitate the drawing process of humans and propose a Two-Stage Statistics-Aware Transformation (TSSAT) module, which first builds the global style foundation by aligning the global statistics of content and style features and then further enriches local style details by swapping the local statistics (instead of local features) in a patch-wise manner, significantly improving the stylization effects. Moreover, to further enhance both content and style representations, we introduce two novel losses: an attention-based content loss and a patch-based style loss, where the former enables better content preservation by enforcing the semantic relation in the content image to be retained during stylization, and the latter focuses on increasing the local style similarity between the style and stylized images. Extensive qualitative and quantitative experiments verify the effectiveness of our method.
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 6878-6887
Open AccessTSSAT: Two-Stage Statistics-Aware Transformation for Artistic Style Transfer
Artistic style transfer aims to create new artistic images by rendering a given photograph with the target artistic style. Existing methods learn styles simply based on global statistics or local patches, lacking careful consideration of the drawing process in practice. Consequently, the stylization results either fail to capture abundant and diversified local style patterns, or contain undesired semantic information of the style image and deviate from the global style distribution. To address this issue, we imitate the drawing process of humans and propose a Two-Stage Statistics-Aware Transformation (TSSAT) module, which first builds the global style foundation by aligning the global statistics of content and style features and then further enriches local style details by swapping the local statistics (instead of local features) in a patch-wise manner, significantly improving the stylization effects. Moreover, to further enhance both content and style representations, we introduce two novel losses: an attention-based content loss and a patch-based style loss, where the former enables better content preservation by enforcing the semantic relation in the content image to be retained during stylization, and the latter focuses on increasing the local style similarity between the style and stylized images. Extensive qualitative and quantitative experiments verify the effectiveness of our method.
- Open Access
- 10 April 2019
- Published by IOP Publishing in IOP Conference Series: Materials Science and Engineering
- Vol. 490, 042018
A quantitative trading method using deep convolution neural network
- 10 April 2019
- Published by IOP Publishing in IOP Conference Series: Materials Science and Engineering
- Vol. 490, 042018
Open AccessA quantitative trading method using deep convolution neural network