Collaborative Cross-modal Fusion with Large Language Model for Recommendation
Open Access
- 21 October 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1565-1574
Abstract
Despite the success of conventional collaborative filtering (CF) approaches for recommendation systems, they exhibit limitations in leveraging semantic knowledge within the textual attributes of users and items. Recent focus on the application of large language models for recommendation (LLM4Rec) has highlighted their capability for effective semantic knowledge capture. However, these methods often overlook the collaborative signals in user behaviors. Some simply instruct-tune a language model, while others directly inject the embeddings of a CF-based model, lacking a synergistic fusion of different modalities. To address these issues, we propose a framework of Collaborative Cross-modal Fusion with Large Language Models, termed CCF-LLM, for recommendation. In this framework, we translate the user-item interactions into a hybrid prompt to encode both semantic knowledge and collaborative signals, and then employ an attentive cross-modal fusion strategy to effectively fuse latent embeddings of both modalities. Extensive experiments demonstrate that CCF-LLM outperforms existing methods by effectively utilizing semantic and collaborative signals in the LLM4Rec context.Keywords
All Related Versions
Funding Information
- Ministry of Education, Singapore (T2EP20122-004)
This publication has 28 references indexed in Scilit:
- GenRec: Large Language Model for Generative Recommendation2024
- Large Language Models are Zero-Shot Rankers for Recommender Systems2024
- Self-Attentive Sequential Recommendation2018
- A Survey of Collaborative Filtering-Based Recommender Systems: From Traditional Methods to Hybrid Methods Based on Social NetworksIEEE Access, 2018
- Handling data sparsity in collaborative filtering using emotion and semantic based features2011
- Factorization Machines2010
- Matrix Factorization Techniques for Recommender SystemsComputer, 2009
- On social networks and collaborative recommendation2009
- Fast online SVD revisions for lightweight recommender systems2003
- A fuzzy K-nearest neighbor algorithmIEEE Transactions on Systems, Man, and Cybernetics, 1985