Semiotics-based prompt engineering for architectural text-to-image generation processes
Open Access
- 29 July 2025
- journal article
- Published by Universidad de Cuenca in ESTOA
- Vol. 14 (28) , 121-135
Abstract
Text-to-image generative AI tools have gained significant attention in the architectural community; however, they are currently being used by trial-and-error with simple textual inputs. This is largely due to the lack of established frameworks for crafting prompts that yield semantically rich architectural outputs. This paper proposes using semiotics as an analytical method facilitating text-to-image generation processes. Two experiments were conducted to investigate the effects of semiotic analysis and adding context modifiers to prompts on the relevancy of outputs of three mainstream text-to-image generation tools (DALL-E, Midjourney, and Stable Diffusion). The results indicate the effectiveness of the proposed method and reveal opportunities and limitations of current text-to-image generative models in architecture. It is concluded that a human-centered approach to Human-AI interaction is needed to overcome issues regarding control, transparency, and data quality.Keywords
This publication has 15 references indexed in Scilit:
- Prompting AI Art: An Investigation into the Creative Skill of Prompt EngineeringInternational Journal of Human–Computer Interaction, 2024
- AI for conceptual architecture: Reflections on designing with text-to-text, text-to-image, and image-to-image generatorsFrontiers of Architectural Research, 2024
- A taxonomy of prompt modifiers for text-to-image generationBehaviour & Information Technology, 2023
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language ProcessingACM Computing Surveys, 2023
- Data collection and quality challenges in deep learning: a data-centric AI perspectiveThe VLDB Journal, 2023
- Generative Adversarial Networks in the built environment: A comprehensive review of the application of GANs across data types and scalesBuilding and Environment, 2022
- FloorplanGAN: Vector residential floorplan adversarial generationAutomation in Construction, 2022
- On GANs, NLP and Architecture: Combining Human and Machine Intelligences for the Generation and Evaluation of Meaningful DesignsTechnology|Architecture + Design, 2021
- Towards Hallucinating Machines - Designing with Computational VisionInternational Journal of Architectural Computing, 2020
- Recent advances in convolutional neural networksPattern Recognition, 2018