Zhuofan Xia
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Zhuofan Xia has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- Gao Huang
Tsinghua University, Beijing, China
4 shared publications - Dongchen Han
Tsinghua University
3 shared publications - Shiji Song
Tsinghua University
3 shared publications - Yizeng Han
Department of Automation, BNRist, Tsinghua University, Beijing, China
3 shared publications - Duo Li
Tsinghua University
2 shared publications - Ji Li
Microsoft Research Asia
2 shared publications
Latest Publications
- 3 June 2025
- Published by Elsevier in Neural Networks
- Vol. 190, 107654
From ReLU to GeMU: Activation functions in the lens of cone projection
- 3 June 2025
- Published by Elsevier in Neural Networks
- Vol. 190, 107654
From ReLU to GeMU: Activation functions in the lens of cone projection
- 22 November 2024
- Published by Springer Nature
- p. 424-441
Efficient Diffusion Transformer with Step-Wise Dynamic Attention Mediators
- 22 November 2024
- Published by Springer Nature
- p. 424-441
Efficient Diffusion Transformer with Step-Wise Dynamic Attention Mediators
- Top Cited Papers
- 1 November 2024
- Published by Springer Nature
- p. 124-140
Agent Attention: On the Integration of Softmax and Linear Attention
- 1 November 2024
- Published by Springer Nature
- p. 124-140
Top Cited PapersAgent Attention: On the Integration of Softmax and Linear Attention
- Open Access
Efficient Diffusion Transformer with Step-wise Dynamic Attention Mediators
This paper identifies significant redundancy in the query-key interactions within self-attention mechanisms of diffusion transformer models, particularly during the early stages of denoising diffusion steps. In response to this observation, we present a novel diffusion transformer framework incorporating an additional set of mediator tokens to engage with queries and keys separately. By modulating the number of mediator tokens during the denoising generation phases, our model initiates the denoising process with a precise, non-ambiguous stage and gradually transitions to a phase enriched with detail. Concurrently, integrating mediator tokens simplifies the attention module's complexity to a linear scale, enhancing the efficiency of global attention processes. Additionally, we propose a time-step dynamic mediator token adjustment mechanism that further decreases the required computational FLOPs for generation, simultaneously facilitating the generation of high-quality images within the constraints of varied inference budgets. Extensive experiments demonstrate that the proposed method can improve the generated image quality while also reducing the inference cost of diffusion transformers. When integrated with the recent work SiT, our method achieves a state-of-the-art FID score of 2.01. The source code is available at https://github.com/LeapLabTHU/Attention-Mediators.
Open AccessEfficient Diffusion Transformer with Step-wise Dynamic Attention Mediators
This paper identifies significant redundancy in the query-key interactions within self-attention mechanisms of diffusion transformer models, particularly during the early stages of denoising diffusion steps. In response to this observation, we present a novel diffusion transformer framework incorporating an additional set of mediator tokens to engage with queries and keys separately. By modulating the number of mediator tokens during the denoising generation phases, our model initiates the denoising process with a precise, non-ambiguous stage and gradually transitions to a phase enriched with detail. Concurrently, integrating mediator tokens simplifies the attention module's complexity to a linear scale, enhancing the efficiency of global attention processes. Additionally, we propose a time-step dynamic mediator token adjustment mechanism that further decreases the required computational FLOPs for generation, simultaneously facilitating the generation of high-quality images within the constraints of varied inference budgets. Extensive experiments demonstrate that the proposed method can improve the generated image quality while also reducing the inference cost of diffusion transformers. When integrated with the recent work SiT, our method achieves a state-of-the-art FID score of 2.01. The source code is available at https://github.com/LeapLabTHU/Attention-Mediators.