An Unified Search and Recommendation Foundation Model for Cold-Start Scenario
Open Access
- 21 October 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 4595-4601
Abstract
In modern commercial search engines and recommendation systems, data from multiple domains is available to jointly train the multi-domain model. Traditional methods train multi-domain models in the multi-task setting, with shared parameters to learn the similarity of multiple tasks, and task-specific parameters to learn the divergence of features, labels, and sample distributions of individual tasks. With the development of large language models, LLM can extract global domain-invariant text features that serve both search and recommendation tasks. We propose a novel framework called S&R Multi-Domain Foundation, which uses LLM to extract domain invariant features, and Aspect Gating Fusion to merge the ID feature, domain invariant text features and task-specific heterogeneous sparse features to obtain the representations of query and item. Additionally, samples from multiple search and recommendation scenarios are trained jointly with Domain Adaptive Multi-Task module to obtain the multi-domain foundation model. We apply the S&R Multi-Domain foundation model to cold start scenarios in the pretrain-finetune manner, which achieves better performance than other SOTA transfer learning methods. The S&R Multi-Domain Foundation model has been successfully deployed in Alipay Mobile Application's online services, such as content query recommendation and service card recommendation, etc.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- A Category-aware Multi-interest Model for Personalized Product Search2022
- Joint Learning of E-commerce Search and Recommendation with a Unified Graph Neural Network2022
- USER2021
- One Model to Serve All2021
- Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations2020
- Learning a Joint Search and Recommendation Model from User-Item Interactions2020
- Infer Implicit Contexts in Real-time Online-to-Offline Recommendation2019
- Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts2018
- Learning a Hierarchical Embedding Model for Personalized Product Search2017
- Multitask LearningMachine Learning, 1997