ArtCrafter: Text-Image Aligning Artistic Attribute Transfer via Embedding Reframing
- 1 July 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 32 (7) , 4959-4972
Abstract
Recent years have witnessed significant advancements in text-guided style transfer, primarily attributed to innovations in diffusion models. These models excel in conditional guidance, utilizing text or images to direct the sampling process. Traditional style transfer focuses on low-level visual features, such as brushstroke textures and color distributions, and appears more like applying an artistic filter to an image. Artistic attribute transfer, however, transcends the limitations of traditional style transfer by achieving the transfer of visual concepts from color and brushstrokes to high level aesthetic attributes such as composition, pose, and key semantic elements, resulting in more natural outcomes. Therefore, we propose an innovative text-to-image artistic attribute transfer framework named ArtCrafter. Specifically, we introduce an attention-based style extraction module, meticulously engineered to capture the subtle artistic attribute elements within an image. This module features a multi-layer architecture that leverages the capabilities of perceiver attention mechanisms to integrate fine-grained information. Additionally, we present a novel text-image aligning augmentation component that adeptly balances control over both modalities, enabling the model to efficiently map image and text embeddings into a shared feature space. We achieve this through attention operations that enable smooth information flow between modalities. Lastly, we incorporate an explicit modulation that seamlessly blends multimodal enhanced embeddings with original embeddings through an embedding reframing design, empowering the model to generate diverse outputs. Extensive experiments demonstrate that ArtCrafter yields impressive results in visual stylization, exhibiting exceptional levels of artistic attribute intensity, controllability, and diversity.Keywords
All Related Versions
Funding Information
- Shenzhen Key Laboratory of Next Generation Interactive Media Innovative Technology, China (ZDSYS20210623092001004)
- National Science and Technology Council, Taiwan (114-2221-E-006-114-MY3)
This publication has 34 references indexed in Scilit:
- IPAdapter-Instruct: Resolving Ambiguity in Image-Based Conditioning Using Instruct Prompts2025
- GRA: Detecting Oriented Objects Through Group-Wise Rotating and Attention2024
- Face-Adapter for Pre-trained Diffusion Models with Fine-Grained ID and Attribute Control2024
- Inversion-based Style Transfer with Diffusion Models2023
- High-Fidelity Guided Image Synthesis with Latent Diffusion Models2023
- InstructPix2Pix: Learning to Follow Image Editing Instructions2023
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- Blended Diffusion for Text-driven Editing of Natural Images2022
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015