CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Models
Open Access
- 12 March 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Management of Data
- Vol. 2 (1) , 1-28
Abstract
Recently, the growing memory demands of embedding tables in Deep Learning Recommendation Models (DLRMs) pose great challenges for model training and deployment. Existing embedding compression solutions cannot simultaneously meet three key design requirements: memory efficiency, low latency, and adaptability to dynamic data distribution. This paper presents CAFE, a Compact, Adaptive, and Fast Embedding compression framework that addresses the above requirements. The design philosophy of CAFE is to dynamically allocate more memory resources to important features (called hot features), and allocate less memory to unimportant ones. In CAFE, we propose a fast and lightweight sketch data structure, named HotSketch, to capture feature importance and report hot features in real time. For each reported hot feature, we assign it a unique embedding. For the non-hot features, we allow multiple features to share one embedding by using hash embedding technique. Guided by our design philosophy, we further propose a multi-level hash embedding framework to optimize the embedding tables of non-hot features. We theoretically analyze the accuracy of HotSketch, and analyze the model convergence against deviation. Extensive experiments show that CAFE significantly outperforms existing embedding compression methods, yielding 3.92% and 3.68% superior testing AUC on Criteo Kaggle dataset and CriteoTB dataset at a compression ratio of 10000x. The source codes of CAFE are available at GitHub.Keywords
All Related Versions
This publication has 60 references indexed in Scilit:
- Evolving Interest with Feature Co-action Network for CTR PredictionData Science and Engineering, 2023
- A Survey of Personalized News RecommendationData Science and Engineering, 2023
- Multimodal Interactive Network for Sequential RecommendationJournal of Computer Science and Technology, 2023
- Context-Aware Semantic Type Identification for Relational AttributesJournal of Computer Science and Technology, 2023
- AutoSrh: An Embedding Dimensionality Search Framework for Tabular Data PredictionIEEE Transactions on Knowledge and Data Engineering, 2022
- Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems2021
- Kraken: Memory-Efficient Continual Learning for Large-Scale Real-Time Recommendations2020
- Per-Flow Traffic Measurement Through Randomized Counter SharingIEEE/ACM Transactions on Networking, 2012
- Probabilistic lossy countingACM SIGCOMM Computer Communication Review, 2008
- An improved data stream summary: the count-min sketch and its applicationsJournal of Algorithms, 2005