BlossomRec: Block-level Fused Sparse Attention Mechanism for Sequential Recommendations
Open Access
- 12 April 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 6389-6399
Abstract
Transformer structures have been widely used in sequential recommender systems (SRS). However, as user interaction histories increase, computational time and memory requirements also grow. This is mainly caused by the standard attention mechanism. Although there exist many methods employing efficient attention and SSM-based models, these approaches struggle to effectively model long sequences and may exhibit unstable performance on short sequences. To address these challenges, we design a sparse attention mechanism, BlossomRec, which models both long-term and short-term user interests through attention computation to achieve stable performance across sequences of varying lengths. Specifically, we categorize user interests in recommendation systems into long-term and short-term interests, and compute them using two distinct sparse attention patterns, with the results combined through a learnable gated output. Theoretically, it significantly reduces the number of interactions participating in attention computation. Extensive experiments on four public datasets demonstrate that BlossomRec, when integrated with state-of-the-art Transformer-based models, achieves comparable or even superior performance while significantly reducing memory usage, providing strong evidence of BlossomRec's efficiency and effectiveness. The code is available at https://github.com/Applied-Machine-Learning-Lab/WWW2026_BlossomRec.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (No.62502404)
- Hong Kong Research Grants Council (Research Impact Fund No.R1015-23)
- Hong Kong Research Grants Council (Collaborative Research Fund No.C1043-24GF)
- Hong Kong Research Grants Council (General Research Fund No.11218325)
- Institute of Digital Medicine of City University of Hong Kong (No.9229503)
- Alibaba (CCF-Alimama Tech Kangaroo Fund No. 2024002)
- DIDI (CCF-Didi Gaia Scholars Research Fund)
- Huawei (Huawei Innovation Research Program)
- Tencent (Tencent Rhino-Bird Focused Research Program)
- Tencent (Tencent University Cooperation Project)
- Bytedance (Bytedance)
- Kuaishou (CCF-Kuaishou Large Model Explorer Fund)
- Kuaishou (Kuaishou University Cooperation Project)
This publication has 35 references indexed in Scilit:
- RoFormer: Enhanced transformer with Rotary Position EmbeddingNeurocomputing, 2023
- Time Interval Aware Self-Attention for Sequential Recommendation2020
- SDM2019
- Triton: an intermediate language and compiler for tiled neural network computations2019
- Self-Attentive Sequential Recommendation2018
- CSAN2018
- Deep reinforcement learning for page-wise recommendations2018
- Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning2018
- When Recurrent Neural Networks meet the Neighborhood for Session-Based Recommendation2017
- Deep Neural Networks for YouTube Recommendations2016