CustomVideo: Customizing Text-to-Video Generation With Multiple Subjects
Top Cited Papers
- 27 February 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. PP (15209210) , 1-14
Abstract
Customized text-to-video generation aims to generate high-quality videos guided by text prompts and subject references. Current approaches for personalizing text-to-video generation suffer from tackling multiple subjects, which is a more challenging and practical scenario. In this work, our aim is to promote multi-subject guided text-to-video customization. We propose CustomVideo, a novel framework that can generate identity-preserving videos with the guidance of multiple subjects. To be specific, firstly, we encourage the co-occurrence of multiple subjects via composing them in a single image. Further, upon a basic text-to-video diffusion model, we design a simple yet effective attention control strategy to disentangle different subjects in the latent space of diffusion model. Moreover, to help the model focus on the specific area of the object, we segment the object from given reference images and provide a corresponding object mask for attention learning. Also, we collect a multi-subject text-to-video generation dataset as a comprehensive benchmark. Extensive qualitative, quantitative, and user study results demonstrate the superiority of our method compared to previous state-of-the-art approaches.Keywords
This publication has 34 references indexed in Scilit:
- LaVie: High-Quality Video Generation with Cascaded Latent Diffusion ModelsInternational Journal of Computer Vision, 2024
- Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video GenerationInternational Journal of Computer Vision, 2024
- FastComposer: Tuning-Free Multi-subject Image Generation with Localized AttentionInternational Journal of Computer Vision, 2024
- DiffSynth: Latent In-Iteration Deflickering for Realistic Video Synthesis2024
- Attention Calibration for Disentangled Text-to-Image Personalization2024
- Align Your Latents: High-Resolution Video Synthesis with Latent Diffusion Models2023
- DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation2023
- Multi-Concept Customization of Text-to-Image Diffusion2023
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN22022