SeMv-3D: Toward Concurrency of Semantic and Multi-View Consistency in General Text-to-3D Generation
- 12 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35 (10577149) , 2955-2968
Abstract
General Text-to-3D (GT23D) generation is crucial for creating diverse 3D content across objects and scenes, yet it faces two key challenges: 1) ensuring semantic consistency between input text and generated 3D models, and 2) maintaining multi-view consistency across different perspectives within 3D. Existing approaches typically address only one of these challenges, often leading to suboptimal results in semantic fidelity and structural coherence. To overcome these limitations, we propose SeMv-3D, a novel framework that jointly enhances semantic alignment and multi-view consistency in GT23D generation. At its core, we introduce Triplane Prior Learning (TPL), which effectively learns triplane priors by capturing spatial correspondences across three orthogonal planes using a dedicated Orthogonal Attention mechanism, thereby ensuring geometric consistency across viewpoints. Additionally, we present Prior-based Semantic Aligning in Triplanes (SAT), which enables consistent any-view synthesis by leveraging attention-based feature alignment to reinforce the correspondence between textual semantics and triplane representations. Extensive experiments demonstrate that our method sets a new state-of-the-art in multi-view consistency, while maintaining competitive performance in semantic consistency compared to methods focused solely on semantic alignment. These results emphasize the remarkable ability of our approach to effectively balance and excel in both dimensions, establishing a new benchmark in the field.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (U23A20315)
- National Natural Science Foundation of China (82441006)
- National Natural Science Foundation of China (U22A2097)
- National Natural Science Foundation of China (62425208)
- National Natural Science Foundation of China (62402094)
- Shenzhen Science and Technology Program (JCYJ20240813114208012)
This publication has 55 references indexed in Scilit:
- DreamMesh: Jointly Manipulating and Texturing Triangle Meshes for Text-to-3D Generation2024
- DreamView: Injecting View-Specific Text Guidance Into Text-to-3D Generation2024
- CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model2024
- LGM: Large Multi-view Gaussian Model for High-Resolution 3D Content Creation2024
- Instant3D: Instant Text-to-3D GenerationInternational Journal of Computer Vision, 2024
- TextMesh: Generation of Realistic 3D Meshes From Text Prompts2024
- IBRNet: Learning Multi-View Image-Based Rendering2021
- Diffusion Probabilistic Models for 3D Point Cloud Generation2021
- pixelNeRF: Neural Radiance Fields from One or Few Images2021
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018