Samvit Kaul
Research Subjects
Samvit Kaul has a total of 10 co-authors. Here are the top co-authors by number of shared publications:
- Adwait Jog
University of Virginia, Charlottesville, Virginia, USA
1 shared publication - Anand Sivasubramaniam
The Pennsylvania State University, University Park, PA, USA
1 shared publication - Chita R. Das
The Pennsylvania State University, State College, Pennsylvania, USA
1 shared publication - Kiwan Maeng
Pennsylvania State University, University Park, PA, USA
1 shared publication - Mahmut Kandemir
The Pennsylvania State University, University Park, PA, USA
1 shared publication - Meena Arunachalam
AMD, Inc, Santa Clara, USA
1 shared publication
Latest Publications
- Open Access
- 17 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-15
Optimizing CPU Performance for Recommendation Systems At-Scale
Deep Learning Recommendation Models (DLRMs) are very popular in personalized recommendation systems and are a major contributor to the data-center AI cycles. Due to the high computational and memory bandwidth needs of DLRMs, specifically the embedding stage in DLRM inferences, both CPUs and GPUs are used for hosting such workloads. This is primarily because of the heavy irregular memory accesses in the embedding stage of computation that leads to significant stalls in the CPU pipeline. As the model and parameter sizes keep increasing with newer recommendation models, the computational dominance of the embedding stage also grows, thereby, bringing into question the suitability of CPUs for inference. In this paper, we first quantify the cause of irregular accesses and their impact on caches and observe that off-chip memory access is the main contributor to high latency. Therefore, we exploit two well-known techniques: (1) Software prefetching, to hide the memory access latency suffered by the demand loads and (2) Overlapping computation and memory accesses, to reduce CPU stalls via hyperthreading to minimize the overall execution time. We evaluate our work on a single-core and 24-core configuration with the latest recommendation models and recently released production traces. Our integrated techniques speed up the inference by up to 1.59x, and on average by 1.4x.
- 17 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-15
Open AccessOptimizing CPU Performance for Recommendation Systems At-Scale
Deep Learning Recommendation Models (DLRMs) are very popular in personalized recommendation systems and are a major contributor to the data-center AI cycles. Due to the high computational and memory bandwidth needs of DLRMs, specifically the embedding stage in DLRM inferences, both CPUs and GPUs are used for hosting such workloads. This is primarily because of the heavy irregular memory accesses in the embedding stage of computation that leads to significant stalls in the CPU pipeline. As the model and parameter sizes keep increasing with newer recommendation models, the computational dominance of the embedding stage also grows, thereby, bringing into question the suitability of CPUs for inference. In this paper, we first quantify the cause of irregular accesses and their impact on caches and observe that off-chip memory access is the main contributor to high latency. Therefore, we exploit two well-known techniques: (1) Software prefetching, to hide the memory access latency suffered by the demand loads and (2) Overlapping computation and memory accesses, to reduce CPU stalls via hyperthreading to minimize the overall execution time. We evaluate our work on a single-core and 24-core configuration with the latest recommendation models and recently released production traces. Our integrated techniques speed up the inference by up to 1.59x, and on average by 1.4x.