A Prompt Log Analysis of Text-to-Image Generation Systems
Open Access
- 30 April 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 3892-3902
Abstract
Recent developments in large language models (LLM) and generative AI have unleashed the astonishing capabilities of text-to-image generation systems to synthesize high-quality images that are faithful to a given reference text, known as a “prompt”. These systems have immediately received lots of attention from researchers, creators, and common users. Despite the plenty of efforts to improve the generative models, there is limited work on understanding the information needs of the users of these systems at scale. We conduct the first comprehensive analysis of large-scale prompt logs collected from multiple text-to-image generation systems. Our work is analogous to analyzing the query logs of Web search engines, a line of work that has made critical contributions to the glory of the Web search industry and research. Compared with Web search queries, text-to-image prompts are significantly longer, often organized into special structures that consist of the subject, form, and intent of the generation tasks and present unique categories of information needs. Users make more edits within creation sessions, which present remarkable exploratory patterns. There is also a considerable gap between the user-input prompts and the captions of the images included in the open training data of the generative models. Our findings provide concrete implications on how to improve text-to-image generation systems for creation purposes.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Best Prompts for Text-to-Image Models and How to Find Them2023
- The Creativity of Text-to-Image Generation2022
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- Design Guidelines for Prompt Engineering Text-to-Image Generative Models2022
- X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers2020
- Collaborative search in electronic health recordsJournal of the American Medical Informatics Association, 2011
- Beyond the session timeout2008
- A Day in the Life of PubMed: Analysis of a Typical Day's Query LogJournal of the American Medical Informatics Association, 2007