An Unified Search and Recommendation Foundation Model for Cold-Start Scenario
Authors:
- Yuqi GongYGYuqi Gong
- Xichen DingXDXichen Ding
- Yehui SuYSYehui Su
- Kaiming ShenKSKaiming Shen
- Zhongyi LiuZLZhongyi Liu
- Guannan ZhangGZGuannan Zhang
Preprint
Abstract
In modern commercial search engines and recommendation systems, data from multiple domains is available to jointly train the multi-domain model. Traditional methods train multi-domain models in the multi-task setting, with shared parameters to learn the similarity of multiple tasks, and task-specific parameters to learn the divergence of features, labels, and sample distributions of individual tasks. With the development of large language models, LLM can extract global domain-invariant text features that serve both search and recommendation tasks. We propose a novel framework called S\&R Multi-Domain Foundation, which uses LLM to extract domain invariant features, and Aspect Gating Fusion to merge the ID feature, domain invariant text features and task-specific heterogeneous sparse features to obtain the representations of query and item. Additionally, samples from multiple search and recommendation scenarios are trained jointly with Domain Adaptive Multi-Task module to obtain the multi-domain foundation model. We apply the S\&R Multi-Domain foundation model to cold start scenarios in the pretrain-finetune manner, which achieves better performance than other SOTA transfer learning methods. The S\&R Multi-Domain Foundation model has been successfully deployed in Alipay Mobile Application's online services, such as content query recommendation and service card recommendation, etc.Keywords
All Related Versions
- Version 1, 2023-09-16, ArXiv
- Published version: , 4595.
This publication has 11 references indexed in Scilit:
- When Search Meets Recommendation: Learning Disentangled Search Representation for Recommendation2023
- Prototypical Contrastive Learning and Adaptive Interest Selection for Candidate Generation in Recommendations2022
- Multi-Aspect Dense Retrieval2022
- A Model-Agnostic Causal Learning Framework for Recommendation using Search Data2022
- A Category-aware Multi-interest Model for Personalized Product Search2022
- Joint Learning of E-commerce Search and Recommendation with a Unified Graph Neural Network2022
- USER2021
- Infer Implicit Contexts in Real-time Online-to-Offline Recommendation2019
- Deep Learning for Matching in Search and Recommendation2018
- Learning a Hierarchical Embedding Model for Personalized Product Search2017