TensorMap: A Deep RL-Based Tensor Mapping Framework for Spatial Accelerators
- 9 May 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 73 (8) , 1899-1912
Abstract
The mapping of tensor computation is a complex and important process for spatial accelerators. Today's mapping works depend on hand-tuned kernel libraries or search-based heuristics from human experts. The former is time-intensive while the latter easily leads to sub-optimal performance. In this paper, we propose TensorMap, a deep reinforcement learning (RL)-based mapping framework for tensor computations on spatial accelerators. We propose a sequential generation mode for mapping optimization and construct a coarse-grained action space to reduce the complexity of the mapping search space. An efficient policy network is devised to optimize mapping primitives in the RL-based search. We then propose a stop signal that is sampled from Bernoulli distribution to facilitate multi-level loop unrolling for spatial accelerators. Finally, a genetic algorithm is employed to further refine the optimized mappings. In the experiments, we demonstrate TensorMap's ability for different spatial accelerators with various tensor computations. On TPU, TensorMap provides 2.6 $\times$ , 2.7 $\times$ , and 2.4 $\times$ better energy-delay product (EDP) on average compared with FlexTensor, Ansor, and AMOS respectively. On Eyeriss, TensorMap provides 2.1 $\times$ , 1.8 $\times$ , and 1.7 $\times$ better EDP on average compared with FlexTensor, Ansor, and AMOS respectively.
Keywords
Funding Information
- National Key R&D Program of China (2022ZD0115304)
- Natural Science Foundation of China (62072479, 62332021)
- Major Program of Guangdong Basic and Applied Research (2019B030302002)
- Natural Science Foundation of Guangdong Province (2024B1515020011)
- Guangzhou Science and Technology Projects (202201011388)
- Xiaomi Young Talents Program
- Pazhou Lab (PZL2023KF0001)
This publication has 28 references indexed in Scilit:
- Leveraging Domain Information for the Efficient Automated Design of Deep Learning Accelerators2023
- MAGMA: An Optimization Framework for Mapping Multiple DNNs on Multiple Accelerator Cores2022
- A graph placement methodology for fast chip designNature, 2021
- CoSA: Scheduling by Constrained Optimization for Spatial Accelerators2021
- HASCO: Towards Agile HArdware and Software CO-design for Tensor Computation2021
- Tiramisu: A Polyhedral Compiler for Expressing Fast and Portable Code2019
- MobileNetV2: Inverted Residuals and Linear Bottlenecks2018
- Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks2016
- Deep Residual Learning for Image Recognition2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015