CAN
Authors:
Open Access
- 11 February 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 57-65
Abstract
Feature interaction has been recognized as an important problem in machine learning, which is also very essential for click-through rate (CTR) prediction tasks. In recent years, Deep Neural Networks (DNNs) can automatically learn implicit nonlinear interactions from original sparse features, and therefore have been widely used in industrial CTR prediction tasks. However, the implicit feature interactions learned in DNNs cannot fully retain the complete representation capacity of the original and empirical feature interactions (e.g., cartesian product) without loss. For example, a simple attempt to learn the combination of feature A and feature B < A, B > as the explicit cartesian product representation of new features can outperform previous implicit feature interaction models including factorization machine (FM)-based models and their variations. This indicates there is still a big gap between explicit and implicit feature interaction models. However, to learn all the explicit feature interaction (cartesian product) representations requires a very large sample size along with N times of original parameter space (where N is quite large in most industrial applications). In this paper, we propose a Co-Action Network (CAN) to approximate the explicit pairwise feature interactions without introducing too many additional parameters. More specifically, giving feature A and its associated feature B, their feature interaction is modeled by learning two sets of parameters: 1) the embedding of feature A, and 2) a Multi-Layer Perceptron (MLP) to represent feature B. The approximated feature interaction can be obtained by passing the embedding of feature A through the MLP network of feature B. We refer to such pairwise feature interaction as feature co-action, and such a Co-Action Network unit can provide a very powerful capacity to fitting complex feature interactions. In addition, FM can be viewed as a special case of the CAN unit when the MLP is a single layer with only one output. Experimental results on public and industrial datasets show that CAN outperforms state-of-the-art CTR models and the cartesian product method. Moreover, CAN has been deployed in the display advertisement system in Alibaba, obtaining 12% improvement on CTR and 8% on Revenue Per Mille (RPM), which is a great improvement to the business. The code for experiments in this paper is open-sourced\footnotehttps://github.com/CAN-Paper/Co-Action-Network.Keywords
This publication has 16 references indexed in Scilit:
- Deep Session Interest Network for Click-Through Rate Prediction2019
- Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction2019
- Deep Interest Evolution Network for Click-Through Rate PredictionProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Product-Based Neural Networks for User Response Prediction over Multi-Field Categorical DataACM Transactions on Information Systems, 2018
- Deep Interest Network for Click-Through Rate Prediction2018
- xDeepFM2018
- Meta-Graph Based Recommendation Fusion over Heterogeneous Information Networks2017
- Wide & Deep Learning for Recommender Systems2016
- Field-aware Factorization Machines for CTR Prediction2016
- Factorization Machines2010