Graph-based Regularization on Embedding Layers for Recommendation
Open Access
- 5 September 2020
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 39 (1) , 1-27
Abstract
Neural networks have been extensively used in recommender systems. Embedding layers are not only necessary but also crucial for neural models in recommendation as a typical discrete task. In this article, we argue that the widely used l 2 regularization for normal neural layers (e.g., fully connected layers) is not ideal for embedding layers from the perspective of regularization theory in Reproducing Kernel Hilbert Space. More specifically, the l 2 regularization corresponds to the inner product and the distance in the Euclidean space where correlations between discrete objects (e.g., items) are not well captured. Inspired by this observation, we propose a graph-based regularization approach to serve as a counterpart of the l 2 regularization for embedding layers. The proposed regularization incurs almost no extra computational overhead especially when being trained with mini-batches. We also discuss its relationships to other approaches (namely, data augmentation, graph convolution, and joint learning) theoretically. We conducted extensive experiments on five publicly available datasets from various domains with two state-of-the-art recommendation models. Results show that given a kNN (k-nearest neighbor) graph constructed directly from training data without external information, the proposed approach significantly outperforms the l 2 regularization on all the datasets and achieves more notable improvements for long-tail users and items.Keywords
Funding Information
- National Key Research and Development Program of China (2018AAA0101902)
- MOE-ChinaMobile Program (MCM20170503)
- NSFC (61532001)
This publication has 51 references indexed in Scilit:
- Collaborative Knowledge Base Embedding for Recommender Systems2016
- A Dynamic Recurrent Model for Next Basket Recommendation2016
- Collaborative Denoising Auto-Encoders for Top-N Recommender Systems2016
- Semantic Path based Personalized Recommendation on Weighted Heterogeneous Information Networks2015
- DeepWalk2014
- Personalized entity recommendation2014
- Kernels for Vector-Valued Functions: A ReviewFoundations and Trends in Machine Learning, 2012
- Amazon.com recommendations: item-to-item collaborative filteringIEEE Internet Computing, 2003
- Training with Noise is Equivalent to Tikhonov RegularizationNeural Computation, 1995
- A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by SplinesThe Annals of Mathematical Statistics, 1970