GSPO: A Graph Substitution and Parallelization Joint Optimization Framework for DNN Inference
Open Access
- 23 June 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Abstract
This work proposes GSPO, an automatic unified framework that jointly applies graph substitution and parallelization for DNN inference. GSPO uses a joint optimization computation graph (JOCG) to represent graph substitution and parallelization at the operator level. Then, a novel cost model customized for joint optimization is used to evaluate the computation graph execution time quickly. With the graph partition and backtracking search algorithm, GSPO can find the optimal joint optimization solution within an acceptable search time. Compared to existing frameworks applying graph substitution or parallelization, GSPO can achieve up to 27.1% end-to-end performance improvement and reduce search time by up to 94.3%.Keywords
This publication has 6 references indexed in Scilit:
- Optimizing DNN computation graph using graph substitutionsProceedings of the VLDB Endowment, 2020
- TASO2019
- Learning Transferable Architectures for Scalable Image Recognition2018
- Deep Residual Learning for Image Recognition2016
- Rethinking the Inception Architecture for Computer Vision2016
- An experimental comparison of min-cut/max- flow algorithms for energy minimization in visionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2004