Abstract
Due to the broad successes of deep learning, many CPU-centric artificial intelligent computing systems employ specialized devices such as GPUs, FPGAs, and ASICs, which can be named as Deep Learning Processing Units (DLPUs), for processing computation-intensive deep learning tasks. The separation between the scalar control operations mapped on CPUs and the vector computation operations mapped on DLPUs causes the frequent and costly interactions between CPUs and DLPUs, leading to the Interaction Wall. Moreover, the increasing algorithm complexity and DLPU computation speed would further aggravate the interaction wall substantially.
Funding Information
  • National Key Research and Development Program of China (2017YFA0700902)
  • National Natural Science Foundation of China (61925208, U19B2019, U20A20227)
  • Beijing Natural Science Foundation (JQ18013)
  • Strategic Priority Research Program of Chinese Academy of Science (XDB32050200)
  • Beijing Academy of Artificial Intelligence
  • Beijing Nova Program of Science and Technology (Z191100001119093)
  • CAS Project for Young Scientists in Basic Research (YSBR-029)
  • Youth Innovation Promotion Association of the Chinese Academy of Sciences
  • XPLORE PRIZE