BlenderAlchemy: Editing 3D Graphics with Vision-Language Models
Authors:
Preprint
Abstract
Graphics design is important for various applications, including movie production and game design. To create a high-quality scene, designers usually need to spend hours in software like Blender, in which they might need to interleave and repeat operations, such as connecting material nodes, hundreds of times. Moreover, slightly different design goals may require completely different sequences, making automation difficult. In this paper, we propose a system that leverages Vision-Language Models (VLMs), like GPT-4V, to intelligently search the design action space to arrive at an answer that can satisfy a user's intent. Specifically, we design a vision-based edit generator and state evaluator to work together to find the correct sequence of actions to achieve the goal. Inspired by the role of visual imagination in the human design process, we supplement the visual reasoning capabilities of VLMs with "imagined" reference images from image-generation models, providing visual grounding of abstract language descriptions. In this paper, we provide empirical evidence suggesting our system can produce simple but tedious Blender editing sequences for tasks such as editing procedural materials from text and/or reference images, as well as adjusting lighting configurations for product renderings in complex scenes.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- 3D-GPT: Procedural 3D Modeling with Large Language Models2025
- Robot learning in the era of foundation models: a surveyNeurocomputing, 2025
- Woodpecker: hallucination correction for multimodal large language modelsScience China Information Sciences, 2024
- AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homes2024
- Alchemist: Parametric Control of Material Properties with Diffusion Models2024
- MatFuse: Controllable Material Generation with Diffusion Models2024
- GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation2024
- Paint3D: Paint Anything 3D With Lighting-Less Texture Diffusion Models2024
- Mathematical discoveries from program search with large language modelsNature, 2023
- Perception-driven procedural texture generation from examplesNeurocomputing, 2018