DyArtbank: Diverse Artistic Style Transfer via Pre-trained Stable Diffusion and Dynamic Style Prompt Artbank
Authors:
Preprint
Abstract
Artistic style transfer aims to transfer the learned style onto an arbitrary content image. However, most existing style transfer methods can only render consistent artistic stylized images, making it difficult for users to get enough stylized images to enjoy. To solve this issue, we propose a novel artistic style transfer framework called DyArtbank, which can generate diverse and highly realistic artistic stylized images. Specifically, we introduce a Dynamic Style Prompt ArtBank (DSPA), a set of learnable parameters. It can learn and store the style information from the collection of artworks, dynamically guiding pre-trained stable diffusion to generate diverse and highly realistic artistic stylized images. DSPA can also generate random artistic image samples with the learned style information, providing a new idea for data augmentation. Besides, a Key Content Feature Prompt (KCFP) module is proposed to provide sufficient content prompts for pre-trained stable diffusion to preserve the detailed structure of the input content image. Extensive qualitative and quantitative experiments verify the effectiveness of our proposed method. Code is available: https://github.com/Jamie-Cheung/DyArtbankKeywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Towards the generalization of time series classification: A feature-level style transfer and multi-source transfer learning perspectiveKnowledge-Based Systems, 2024
- Intrinsic-style distribution matching for arbitrary style transferKnowledge-Based Systems, 2024
- Rethink arbitrary style transfer with transformer and contrastive learningComputer Vision and Image Understanding, 2024
- Progressive normalizing flow with learnable spectrum transform for style transferKnowledge-Based Systems, 2023
- Source-free Style-diversity Adversarial Domain Adaptation with Privacy-preservation for person re-identificationKnowledge-Based Systems, 2023
- Caster: Cartoon style transfer via dynamic cartoon style castingNeurocomputing, 2023
- Soft multimodal style transfer via optimal transportKnowledge-Based Systems, 2023
- Person image generation with attention-based injection networkNeurocomputing, 2021
- Pose transfer generation with semantic parsing attention network for person re-identificationKnowledge-Based Systems, 2021
- StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation2018