Exploring the Capabilities of Large Language Models for Generating Diverse Design Solutions
Authors:
- Kevin MaKMKevin Ma
- Daniele GrandiDGDaniele Grandi
- Christopher McCombCMChristopher McComb
- Kosa Goucher-LambertKGKosa Goucher-Lambert
Preprint
Abstract
Access to large amounts of diverse design solutions can support designers during the early stage of the design process. In this paper, we explore the efficacy of large language models (LLM) in producing diverse design solutions, investigating the level of impact that parameter tuning and various prompt engineering techniques can have on the diversity of LLM-generated design solutions. Specifically, LLMs are used to generate a total of 4,000 design solutions across five distinct design topics, eight combinations of parameters, and eight different types of prompt engineering techniques, comparing each combination of parameter and prompt engineering method across four different diversity metrics. LLM-generated solutions are compared against 100 human-crowdsourced solutions in each design topic using the same set of diversity metrics. Results indicate that human-generated solutions consistently have greater diversity scores across all design topics. Using a post hoc logistic regression analysis we investigate whether these differences primarily exist at the semantic level. Results show that there is a divide in some design topics between humans and LLM-generated solutions, while others have no clear divide. Taken together, these results contribute to the understanding of LLMs' capabilities in generating a large volume of diverse design solutions and offer insights for future research that leverages LLMs to generate diverse design solutions for a broad range of design tasks (e.g., inspirational stimuli).Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering DesignComputer-Aided Design, 2023
- Enabling multi-modal search for inspirational design stimuli using deep learningArtificial Intelligence for Engineering Design, Analysis and Manufacturing, 2022
- Crowdsourcing inspiration: Using crowd generated inspirational stimuli to support designer ideationDesign Studies, 2019
- Scaling up analogical innovation with crowds and AIProceedings of the National Academy of Sciences, 2019
- Design-by-analogy: experimental evaluation of a functional analogy search methodology for concept generation improvementResearch in Engineering Design, 2014
- The Impact of Analogies on Creative Concept Generation: Lessons From anIn VivoStudy in Engineering DesignCognitive Science, 2014
- Design by Analogy: A Study of the WordTree Method for Problem Re-RepresentationJournal of Mechanical Design, 2012
- The Value of Crowdsourcing: Can Users Really Compete with Professionals in Generating New Product Ideas?Journal of Product Innovation Management, 2012
- On the Benefits and Pitfalls of Analogies for Innovative Design: Ideation Performance Based on Analogical Distance, Commonness, and Modality of ExamplesJournal of Mechanical Design, 2011
- The role of timing and analogical similarity in the stimulation of idea generation in designDesign Studies, 2008