Visualizing Ambiguity: Analyzing Linguistic Ambiguity Resolution in Text-to-Image Models
Open Access
Abstract
Text-to-image models have demonstrated remarkable progress in generating visual content from textual descriptions. However, the presence of linguistic ambiguity in the text prompts poses a potential challenge to these models, possibly leading to undesired or inaccurate outputs. This work conducts a preliminary study and provides insights into how text-to-image diffusion models resolve linguistic ambiguity through a series of experiments. We investigate a set of prompts that exhibit different types of linguistic ambiguities with different models and the images they generate, focusing on how the models’ interpretations of linguistic ambiguity compare to those of humans. In addition, we present a curated dataset of ambiguous prompts and their corresponding images known as the Visual Linguistic Ambiguity Benchmark (V-LAB) dataset. Furthermore, we report a number of limitations and failure modes caused by linguistic ambiguity in text-to-image models and propose prompt engineering guidelines to minimize the impact of ambiguity. The findings of this exploratory study contribute to the ongoing improvement of text-to-image models and provide valuable insights for future advancements in the field.Keywords
This publication has 24 references indexed in Scilit:
- PrompTHis: Visualizing the Process and Influence of Prompt Editing During Text-to-Image CreationIEEE Transactions on Visualization and Computer Graphics, 2024
- Text-to-Image Synthesis With Generative Models: Methods, Datasets, Performance Metrics, Challenges, and Future DirectionIEEE Access, 2024
- Ambiguity in Linguistics1Studia Linguistica, 2023
- PromptMagician: Interactive Prompt Engineering for Text-to-Image CreationIEEE Transactions on Visualization and Computer Graphics, 2023
- Prompt Engineering as an Important Emerging Skill for Medical Professionals: TutorialJournal of Medical Internet Research, 2023
- Prompt Engineering with ChatGPT: A Guide for Academic WritersAnnals of Biomedical Engineering, 2023
- Learning to Prompt for Vision-Language ModelsInternational Journal of Computer Vision, 2022
- Long-range sequential dependencies precede complex syntactic production in language acquisitionProceedings Of The Royal Society B-Biological Sciences, 2022
- Generative adversarial networksCommunications of the ACM, 2020
- The Interaction of Lexical and Syntactic AmbiguityJournal of Memory and Language, 1993