Hierarchical Fashion Design with Multi-stage Diffusion Models
Preprint
Abstract
Cross-modal fashion synthesis and editing offer intelligent support to fashion designers by enabling the automatic generation and local modification of design drafts.While current diffusion models demonstrate commendable stability and controllability in image synthesis,they still face significant challenges in generating fashion design from abstract design elements and fine-grained editing.Abstract sensory expressions, \eg office, business, and party, form the high-level design concepts, while measurable aspects like sleeve length, collar type, and pant length are considered the low-level attributes of clothing.Controlling and editing fashion images using lengthy text descriptions poses a difficulty.In this paper, we propose HieraFashDiff,a novel fashion design method using the shared multi-stage diffusion model encompassing high-level design concepts and low-level clothing attributes in a hierarchical structure.Specifically, we categorized the input text into different levels and fed them in different time step to the diffusion model according to the criteria of professional clothing designers.HieraFashDiff allows designers to add low-level attributes after high-level prompts for interactive editing incrementally.In addition, we design a differentiable loss function in the sampling process with a mask to keep non-edit areas.Comprehensive experiments performed on our newly conducted Hierarchical fashion dataset,demonstrate that our proposed method outperforms other state-of-the-art competitors.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- FICE: Text-conditioned fashion-image editing with guided GAN inversionPattern Recognition, 2024
- SGDiff: A Style Guided Diffusion Model for Fashion Synthesis2023
- MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing2023
- Blended Latent DiffusionACM Transactions on Graphics, 2023
- FashionTex: Controllable Virtual Try-on with Text and Texture2023
- Toward Intelligent Fashion Design: A Texture and Shape Disentangled Generative Adversarial NetworkACM Transactions on Multimedia Computing, Communications, and Applications, 2023
- Toward Intelligent Design: An AI-Based Fashion Designer Using Generative Adversarial Networks Aided by Sketch and Rendering GeneratorsIEEE Transactions on Multimedia, 2022
- CLIPScore: A Reference-free Evaluation Metric for Image Captioning2021
- MeTRAbs: Metric-Scale Truncation-Robust Heatmaps for Absolute 3D Human Pose EstimationIEEE Transactions on Biometrics, Behavior, and Identity Science, 2020
- Long and Diverse Text Generation with Planning-based Hierarchical Variational Model2019