HD-Custom: Efficient Hierarchical Disentanglement for Coarse-to-Fine Concept Customization in Subject Video Generation

Authors:
Publisher website
Google Scholar
Abstract
Subject-customized video generation aims to synthesize videos that adapt to user-specified concepts, ensuring identity consistency while faithfully following textual prompts. The key challenge lies in balancing concept fidelity and prompt following: insufficient adaptation leads to underfitting, losing subject details, whereas excessive adaptation causes overfitting, weakening motion dynamics and scene diversity. To address this, we propose HD-Custom, a test-time fine-tuning framework that leverages hierarchical disentanglement and a coarse-to-fine strategy to efficiently adapt pretrained text-to-video models. Specifically, the coarse stage disentangles category and attribute cues across layers to establish robust subject representations, while the fine stage applies targeted lightweight adaptations to refine appearance details with minimal interference to pretrained priors. A token disentanglement loss and data augmentation are further introduced to enhance attribute–category separation. We also construct HD-Bench, a benchmark explicitly partitioning in-domain and out-of-domain concepts. Extensive experiments on HD-Bench show that HD-Custom achieves a superior balance between concept fidelity and prompt following while requiring substantially fewer trainable parameters and iterations, outperforming state-of-the-art methods.
Funding Information
  • National Natural Science Foundation of China (62471445)
  • State Key Laboratory of Multimedia Information Processing Open Fund (SKLMIP-KF-2025-04)
  • Public Computing Cloud at Communication University of China

This publication has 30 references indexed in Scilit: