StyleSSP: Sampling StartPoint Enhancement for Training-free Diffusion-based Method for Style Transfer
Preprint
Abstract
Training-free diffusion-based methods have achieved remarkable success in style transfer, eliminating the need for extensive training or fine-tuning. However, due to the lack of targeted training for style information extraction and constraints on the content image layout, training-free methods often suffer from layout changes of original content and content leakage from style images. Through a series of experiments, we discovered that an effective startpoint in the sampling stage significantly enhances the style transfer process. Based on this discovery, we propose StyleSSP, which focuses on obtaining a better startpoint to address layout changes of original content and content leakage from style image. StyleSSP comprises two key components: (1) Frequency Manipulation: To improve content preservation, we reduce the low-frequency components of the DDIM latent, allowing the sampling stage to pay more attention to the layout of content images; and (2) Negative Guidance via Inversion: To mitigate the content leakage from style image, we employ negative guidance in the inversion stage to ensure that the startpoint of the sampling stage is distanced from the content of style image. Experiments show that StyleSSP surpasses previous training-free style transfer baselines, particularly in preserving original content and minimizing the content leakage from style image.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- IPAdapter-Instruct: Resolving Ambiguity in Image-Based Conditioning Using Instruct Prompts2025
- BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion2024
- Understanding the Impact of Negative Prompts: When and How Do They Take Effect?2024
- FlexiEdit: Frequency-Aware Latent Refinement for Enhanced Non-rigid Editing2024
- Null-text Inversion for Editing Real Images using Guided Diffusion Models2023
- ArtFID: Quantitative Evaluation of Neural Style Transfer2022
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- Style Transfer by Relaxed Optimal Transport and Self-Similarity2019
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018
- Understanding deep image representations by inverting them2015