Interactive Latent Diffusion Model
Open Access
- 12 July 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 586-596
Abstract
This paper introduces Interactive Latent Diffusion Model (IELDM), an encapsulation of a popular text-to-image diffusion model into an Evolutionary framework, allowing the users to steer the design of images toward their goals, alleviating the tedious trial-and-error process that such tools frequently require. The users can not only designate their favourite images, allowing the system to build a surrogate model based on their goals and move in the same directions, but also click on some specific parts of the images to either locally refine the image through dedicated mutation, or recombine images by choosing on each one some regions they like. Experiments validate the benefits of IELDM, especially in a situation where Latent Diffusion Model is challenged by complex input prompts.Keywords
This publication has 6 references indexed in Scilit:
- Interactive evolution and exploration within latent level-design space of generative adversarial networks2020
- Self-adjusting mutation rates with provably optimal success rules2019
- Evolving mario levels in the latent space of a deep convolutional generative adversarial network2018
- Picbreeder: A Case Study in Collaborative Evolutionary Exploration of Design SpaceEvolutionary Computation, 2011
- Interactive, Evolutionary Search in Upstream Object-Oriented Class DesignIEEE Transactions on Software Engineering, 2010
- Interactive evolutionary computation: fusion of the capabilities of EC optimization and human evaluationProceedings of the IEEE, 2001