Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models
Authors:
- 1 June 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 14519-14527
Abstract
We introduce a new local sparse attention layer that preserves two-dimensional geometry and locality. We show that by just replacing the dense attention layer of SAGAN with our construction, we obtain very significant FID, Inception score and pure visual improvements. FID score is improved from 18.65 to 15.94 on ImageNet, keeping all other parameters the same. The sparse attention patterns that we propose for our new layer are designed using a novel information theoretic criterion that uses information flow graphs. We also present a novel way to invert Generative Adversarial Networks with attention. Our method uses the attention layer of the discriminator to create an innovative loss function. This allows us to visualize the newly introduced attention heads and show that they indeed capture interesting aspects of two-dimensional geometry of real images.Keywords
All Related Versions
This publication has 5 references indexed in Scilit:
- GAN-Based Projector for Faster Recovery With Convergence Guarantees in Linear Inverse Problems2019
- Adaptively Sparse Transformers2019
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Network Coding for Distributed Storage SystemsIEEE Transactions on Information Theory, 2010
- Network information flowIEEE Transactions on Information Theory, 2000