Multi-Grained Topological Pre-Training of Language Models in Sponsored Search
Open Access
- 18 July 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2189-2193
Abstract
Relevance models measure the semantic closeness between queries and the candidate ads, widely recognized as the nucleus of sponsored search systems. Conventional relevance models solely rely on the textual data within the queries and ads, whose performance is hindered by the scarce semantic information in these short texts. Recently, user behavior graphs have been incorporated to provide complementary information beyond pure textual semantics.Despite the promising performance, behavior-enhanced models suffer from exhausting resource costs due to the extra computations introduced by explicit topological aggregations. In this paper, we propose a novel Multi-Grained Topological Pre-Training paradigm, MGTLM, to teach language models to understand multi-grained topological information in behavior graphs, which contributes to eliminating explicit graph aggregations and avoiding information loss. Extensive experimental results over online and offline settings demonstrate the superiority of our proposal.Keywords
Funding Information
- NSFC (62172443)
This publication has 9 references indexed in Scilit:
- Semi-Supervised Variational User Identity Linkage via Noise-Aware Self-LearningIEEE Transactions on Knowledge and Data Engineering, 2023
- Improving Relevance Modeling via Heterogeneous Behavior Graph Learning in Bing Ads2022
- Geometric Disentangled Collaborative Filtering2022
- Leveraging Bidding Graphs for Advertiser-Aware Relevance Modeling in Sponsored Search2021
- SimCSE: Simple Contrastive Learning of Sentence Embeddings2021
- Adversarial Learning for Weakly-Supervised Social Network AlignmentProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Dynamic-Structured Semantic Propagation Network2018
- A Latent Semantic Model with Convolutional-Pooling Structure for Information Retrieval2014
- Learning semantic representations using convolutional neural networks for web search2014