Aquabolt-XL HBM2-PIM, LPDDR5-PIM With In-Memory Processing, and AXDIMM With Acceleration Buffer

Authors:

Abstract
Processing-in-Memory has been proposed to improve performance of bandwidth intensive workloads as well as save energy reduction due to reduced compute-memory data movement. To realize PIM, Programmable computing units were integrated with memory cores on a HBM2 device to enable parallel processing and minimize data movement. A GPU system equipped with Samsung Aquabolt-XL HBM2-PIM devices improved micro-kernel General Matrix-Vector multiplication and speech recognition applications by 8.9 and 3.5 times, respectively, and reduced energy consumption by over 60%. In a Xilinx AlveoU280 system, micro-kernel GEMV and ADD workload performances improved by 2.8X, and Long Short-Term Memory workload improved by 2.54X. Simulations show that performance gain of over 2.3X may be attained in a system with LP5-PIM for certain transformer based speech recognition with energy reduction of 86%. In addition, AXDIMM, a DIMM-level PIM with acceleration buffers, exhibits an 80% performance improvement and 42.6% energy savings over a regular RDIMM system.