Visual Layout Composer: Image-Vector Dual Diffusion Model for Design Layout Generation
Authors:
- 16 June 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9222-9231
Abstract
This paper proposes an image-vector dual diffusion model for generative layout design. Distinct from prior efforts that mostly ignores visual information of elements and the whole canvas, our approach integrates the power of a pre-trained large image diffusion model to guide layout composition in a vector diffusion model by providing enhanced salient region understanding and high-level inter-element relationship reasoning. Our proposed model simultaneously operates in two domains: it generates the overall design appearance in the image domain while optimizing the size and position of each design element in the vector domain. The proposed method achieves the state-of-the-art results on several datasets and enables new layout design applications. Project webpage: https://aminshabani.github.io/visual_layout_composer.Keywords
This publication has 27 references indexed in Scilit:
- Unsupervised Domain Adaption with Pixel-Level Discriminator for Image-Aware Layout Generation2023
- PosterLayout: A New Benchmark and Approach for Content-Aware Visual-Textual Presentation Layout2023
- Towards Flexible Multi-modal Document Models2023
- LayoutDM: Discrete Diffusion Model for Controllable Layout Generation2023
- Unifying Layout Generation with a Decoupled Diffusion Model2023
- BLT: Bidirectional Layout Transformer for Controllable Layout Generation2022
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- MaskGIT: Masked Generative Image Transformer2022
- Variational Transformer Networks for Layout Generation2021
- Neural Design Network: Graphic Layout Generation with Constraints2020