LayoutTransformer: Layout Generation and Completion with Self-attention
Authors:
- 1 October 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 984-994
Abstract
We address the problem of scene layout generation for diverse domains such as images, mobile applications, documents, and 3D objects. Most complex scenes, natural or human-designed, can be expressed as a meaningful arrangement of simpler compositional graphical primitives. Generating a new layout or extending an existing layout re- quires understanding the relationships between these primitives. To do this, we propose LayoutTransformer, a novel framework that leverages self-attention to learn contextual relationships between layout elements and generate novel layouts in a given domain. Our framework allows us to generate a new layout either from an empty set or from an initial seed set of primitives, and can easily scale to support an arbitrary of primitives per layout. Furthermore, our analyses show that the model is able to automatically capture the semantic properties of the primitives. We propose simple improvements in both representation of layout primitives, as well as training methods to demonstrate competitive performance in very diverse data domains such as object bounding boxes in natural images (COCO bounding box), documents (PubLayNet), mobile applications (RICO dataset) as well as 3D shapes (Part-Net). Code and other materials will be made available at https://kampta.github.io/layout.Keywords
All Related Versions
This publication has 35 references indexed in Scilit:
- ComplementMeACM Transactions on Graphics, 2017
- Rico2017
- Semantic Image Synthesis via Adversarial Learning2017
- Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Networks2017
- Neural Scene De-rendering2017
- Contextual Priming and Feedback for Faster R-CNN2016
- Deep Residual Learning for Image Recognition2016
- Rethinking the Inception Architecture for Computer Vision2016
- Text to 3D Scene Generation with Rich Lexical Grounding2015
- Statistical context priming for object detection2002