FlashEmbedding
Authors:
Open Access
- 24 August 2021
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 9-16
Abstract
We present FlashEmbedding, a hardware/software co-design solution for storing embedding tables on SSDs for large-scale recommendation inference under memory capacity-limited systems. FlashEmbedding leverages an embedding semantic-aware SSD, an embedding-oriented software cache, and pipeline techniques to improve the overall performance. We evaluate the performance of FlashEmbedding with our FPGA-based prototype SSD on a real-world public dataset. FlashEmbedding achieves up to 17.44× lower latency in embedding lookups and 2.89× lower end-to-end latency than baseline solution in a memory capacity-limted system.Keywords
This publication has 10 references indexed in Scilit:
- MERCI: efficient embedding reduction on commodity hardware via sub-query memoization2021
- RecSSD: near data processing for solid state drive based recommendation inference2021
- Reducing tail latency of DNN-based recommender systems using in-storage processing2020
- The Architectural Implications of Facebook's DNN-Based Personalized Recommendation2020
- 13.5 A 128Gb 1b/Cell 96-Word-Line-Layer 3D Flash Memory to Improve Random Read Latency with tPROG=75µs and tR=4µs2020
- TensorDIMM2019
- FlatFlash: Exploiting the Byte-Accessibility of SSDs within a Unified Memory-Storage Hierarchy2019
- Billion-scale Commodity Embedding for E-commerce Recommendation in Alibaba2018
- 2B-SSD: The Case for Dual, Byte- and Block-Addressable Solid-State Drives2018
- Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective2018