PromptMagician: Interactive Prompt Engineering for Text-to-Image Creation
Top Cited Papers
- 25 October 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. PP (10772626) , 1-11
Abstract
Generative text-to-image models have gained great popularity among the public for their powerful capability to generate high-quality images based on natural language prompts. However, developing effective prompts for desired images can be challenging due to the complexity and ambiguity of natural language. This research proposes PromptMagician, a visual analysis system that helps users explore the image results and refine the input prompts. The backbone of our system is a prompt recommendation model that takes user prompts as input, retrieves similar prompt-image pairs from DiffusionDB, and identifies special (important and relevant) prompt keywords. To facilitate interactive prompt refinement, PromptMagician introduces a multi-level visualization for the cross-modal embedding of the retrieved images and recommended keywords, and supports users in specifying multiple criteria for personalized exploration. Two usage scenarios, a user study, and expert interviews demonstrate the effectiveness and usability of our system, suggesting it facilitates prompt engineering and improves the creativity support of the generative text-to-image model.Keywords
All Related Versions
This publication has 53 references indexed in Scilit:
- Learning a no-reference quality metric for single-image super-resolutionComputer Vision and Image Understanding, 2017
- DataTone2015
- Quantifying the Creativity Support of Digital Tools through the Creativity Support IndexACM Transactions on Computer-Human Interaction, 2014
- Combing the hairball with BioFabric: a new approach for visualization of large networksBMC Bioinformatics, 2012
- Unsupervised feature learning framework for no-reference image quality assessment2012
- FSIM: A Feature Similarity Index for Image Quality AssessmentIEEE Transactions on Image Processing, 2011
- User Interaction with Scatterplots on Small Screens - A Comparative Evaluation of Geometric-Semantic Zoom and Fisheye DistortionIEEE Transactions on Visualization and Computer Graphics, 2006
- A statistical interpretation of term specificity and its application in retrievalJournal of Documentation, 2004
- NLTK2002
- Pad1993