Timestep Embedding Tells: It’s Time to Cache for Video Diffusion Model
Authors:
Top Cited Papers
- 10 June 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7353-7363
Abstract
As a fundamental backbone for video generation, diffusion models are challenged by low inference speed due to the sequential nature of denoising. Previous methods speed up the models by caching and reusing model outputs at uniformly selected timesteps. However, such a strategy neglects the fact that differences among model outputs are not uniform across timesteps, which hinders selecting the appropriate model outputs to cache, leading to a poor balance between inference efficiency and visual quality. In this study, we introduce Timestep Embedding Aware Cache (TeaCache), a training-free caching approach that estimates and leverages the fluctuating differences among model outputs across timesteps. Rather than directly using the time-consuming model outputs, TeaCache focuses on model inputs, which have a strong correlation with the modeloutputs while incurring negligible computational cost. TeaCache first modulates the noisy inputs using the timestep embeddings to ensure their differences better approximating those of model outputs. TeaCache then introduces a rescaling strategy to refine the estimated differences and utilizes them to indicate output caching. Experiments show that TeaCache achieves up to 4.41× acceleration over Open-Sora-Plan with negligible (-0.07% Vbench score) degradation of visual quality.Keywords
Funding Information
- National Natural Science Foundation of China
- National Natural Science Foundation of China
This publication has 20 references indexed in Scilit:
- DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic ModelsMachine Intelligence Research, 2025
- FRDiff : Feature Reuse for Universal Training-Free Acceleration of Diffusion Models2024
- PIXART-$\Sigma $: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation2024
- Adversarial Diffusion Distillation2024
- Cache Me if You Can: Accelerating Diffusion Models through Block Caching2024
- Dream Video: Composing Your Dream Videos with Customized Subject and Motion2024
- On Distillation of Guided Diffusion Models2023
- Post-Training Quantization on Diffusion Models2023
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018