AlphaFuse: Learn ID Embeddings for Sequential Recommendation in Null Space of Language Embeddings
Authors:
- Guoqing HuGHGuoqing Hu
- An ZhangAZAn Zhang
- Shuo LiuSLShuo Liu
- Zhibo CaiZCZhibo Cai
- Xun YangXYXun Yang
- Xiang WangXWXiang Wang
Preprint
Abstract
Recent advancements in sequential recommendation have underscored the potential of Large Language Models (LLMs) for enhancing item embeddings. However, existing approaches face three key limitations: 1) the degradation of the semantic space when high-dimensional language embeddings are mapped to lower-dimensional ID embeddings, 2) the underutilization of language embeddings, and 3) the reliance on additional trainable parameters, such as an adapter, to bridge the gap between the semantic and behavior spaces.In this paper, we introduce AlphaFuse, a simple but effective language-guided learning strategy that addresses these challenges by learning ID embeddings within the null space of language embeddings. Specifically, we decompose the semantic space of language embeddings via Singular Value Decomposition (SVD), distinguishing it into a semantic-rich row space and a semantic-sparse null space. Collaborative signals are then injected into the null space, while preserving the rich semantics of the row space. AlphaFuse prevents degradation of the semantic space, integrates the retained language embeddings into the final item embeddings, and eliminates the need for auxiliary trainable modules, enabling seamless adaptation to any sequential recommendation framework. We validate the effectiveness and flexibility of AlphaFuse through extensive experiments on three benchmark datasets, including cold-start user and long-tail settings, showcasing significant improvements in both discriminative and diffusion-based generative sequential recommenders. Our codes and datasets are available at https://github.com/Hugo-Chinn/AlphaFuse.Keywords
All Related Versions
- Version 1, 2025-04-27, ArXiv
- Version 2, 2025-04-29, ArXiv
- Published version: , 1614.
This publication has 21 references indexed in Scilit:
- Are ID Embeddings Necessary? Whitening Pre-trained Text Embeddings for Effective Sequential Recommendation2024
- Representation Learning with Large Language Models for Recommendation2024
- Text Is All You Need: Learning Language Representations for Sequential Recommendation2023
- MELT: Mutual Enhancement of Long-Tailed User and Item for Sequential Recommendation2023
- Towards Universal Sequence Representation Learning for Recommender Systems2022
- An embedding and interactions learning approach for ID feature in deep recommender systemExpert Systems with Applications, 2022
- Self-supervised Graph Learning for Recommendation2021
- CITIES: Contextual Inference of Tail-Item Embeddings for Sequential Recommendation2020
- Self-Attentive Sequential Recommendation2018
- Image-Based Recommendations on Styles and Substitutes2015