CTRL: Connect Collaborative and Language Model for CTR Prediction
Open Access
- 22 November 2025
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Recommender Systems
- Vol. 4 (2) , 1-23
Abstract
Traditional click-through rate (CTR) prediction models convert the tabular data into one-hot vectors and leverage the collaborative relations among features for inferring the user’s preference over items. This modeling paradigm discards essential semantic information. Though some works like P5 and KAR have explored the potential of using Pre-trained Language Models (PLMs) to extract semantic signals for CTR prediction, they are computationally expensive and suffer from low efficiency. Besides, the beneficial collaborative relations are not considered, hindering the recommendation performance. To solve these problems, in this article, we propose a novel framework CTRL , which is industrial-friendly and model-agnostic with superior inference efficiency. Specifically, the original tabular data is first converted into textual data. Both tabular data and converted textual data are regarded as two different modalities and are separately fed into the collaborative CTR model and PLM. A cross-modal knowledge alignment procedure is performed to fine-grained align and integrate the collaborative and semantic signals, and the lightweight collaborative model can be deployed online for efficient serving after fine-tuning with supervised signals. Experimental results on three public datasets show that CTRL outperforms the state-of-the-art (SOTA) CTR models significantly. Moreover, we further verify its effectiveness on a large-scale industrial recommender system.Keywords
All Related Versions
This publication has 39 references indexed in Scilit:
- Exploring Text-Transformers in AAAI 2021 Shared Task: COVID-19 Fake News Detection in English2021
- A recent overview of the state-of-the-art elements of text classificationExpert Systems with Applications, 2018
- Deep Interest Network for Click-Through Rate Prediction2018
- Deep & Cross Network for Ad Click Predictions2017
- Wide & Deep Learning for Recommender Systems2016
- Practical Lessons from Predicting Clicks on Ads at Facebook2014
- Optimal real-time bidding for display advertising2014
- Factorization Machines2010
- Jensen-Shannon divergence and Hilbert space embedding2004
- The Regression Analysis of Binary SequencesJournal of the Royal Statistical Society Series B: Statistical Methodology, 1958