Accelerating Parallel Structures in DNNs via Parallel Fusion and Operator Co-Optimization
Open Access
- 16 June 2025
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
Abstract
Parallel structures have become a key pattern in deep neural networks (DNNs), offering improved efficiency and scalability. However, existing machine learning compilers (MLCs) face challenges in optimizing these structures due to limited parallel fusion scope and insufficient analysis of intra-operator characteristics. This paper introduces Magneto, a framework designed to accelerate DNN inference by co-optimizing parallel operators. Magneto broadens the fusion scope and incorporates a specialized co-tuning algorithm to optimize operators jointly. Our approach addresses the unique challenges inherent in optimizing parallel structures, enabling significant performance improvements across various hardware platforms. Experimental results show that Magneto outperforms state-of-the-art NVIDIA TensorRT and AMD MIGraphX, achieving geometric mean speedups of 2.27 × and 2.88 ×, respectively.Keywords
This publication has 16 references indexed in Scilit:
- MLIR: Scaling Compiler Infrastructure for Domain Specific Computation2021
- GPT-3: Its Nature, Scope, Limits, and ConsequencesMinds and Machines, 2020
- TASO2019
- Learning to optimize halide with tree search and random programsACM Transactions on Graphics, 2019
- Triton: an intermediate language and compiler for tiled neural network computations2019
- Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts2018
- Learning Transferable Architectures for Scalable Image Recognition2018
- Aggregated Residual Transformations for Deep Neural Networks2017
- Rethinking the Inception Architecture for Computer Vision2016
- HalideACM SIGPLAN Notices, 2013