Identity-Preserving Text-to-Video Generation Guided by Simple yet Effective Spatial-Temporal Decoupled Representations
Authors:
- Yuji WangYWYuji Wang
- Moran LiMLMoran Li
- Xiaobin HuXHXiaobin Hu
- Ran YiRYRan Yi
- Jiangning ZhangJZJiangning Zhang
- Han FengHFHan Feng
- Weijian CaoWCWeijian Cao
- Yabiao WangYWYabiao Wang
- Chengjie WangCWChengjie Wang
- Lizhuang MaLMLizhuang Ma
Preprint
Abstract
Identity-preserving text-to-video (IPT2V) generation, which aims to create high-fidelity videos with consistent human identity, has become crucial for downstream applications. However, current end-to-end frameworks suffer a critical spatial-temporal trade-off: optimizing for spatially coherent layouts of key elements (e.g., character identity preservation) often compromises instruction-compliant temporal smoothness, while prioritizing dynamic realism risks disrupting the spatial coherence of visual structures. To tackle this issue, we propose a simple yet effective spatial-temporal decoupled framework that decomposes representations into spatial features for layouts and temporal features for motion dynamics. Specifically, our paper proposes a semantic prompt optimization mechanism and stage-wise decoupled generation paradigm. The former module decouples the prompt into spatial and temporal components. Aligned with the subsequent stage-wise decoupled approach, the spatial prompts guide the text-to-image (T2I) stage to generate coherent spatial features, while the temporal prompts direct the sequential image-to-video (I2V) stage to ensure motion consistency. Experimental results validate that our approach achieves excellent spatiotemporal consistency, demonstrating outstanding performance in identity preservation, text relevance, and video quality. By leveraging this simple yet robust mechanism, our algorithm secures the runner-up position in 2025 ACM MultiMedia Challenge.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Concat-ID: Towards Universal Identity-Preserving Video Synthesis2025
- MoonShot: Towards Controllable Video Generation and Editing with Motion-Aware Multimodal ConditionsInternational Journal of Computer Vision, 2025
- Still-Moving: Customized Video Generation without Customized Video DataACM Transactions on Graphics, 2024
- A Survey on Video Diffusion ModelsACM Computing Surveys, 2024
- MotionBooth: Motion-Aware Customized Text-to-Video Generation2024
- Optimized HRNet for image semantic segmentationExpert Systems with Applications, 2021