OptiFX: Automatic Optimization for Convolutional Neural Networks with Aggressive Operator Fusion on GPUs
Open Access
- 30 June 2025
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
- Vol. 22 (2) , 1-27
Abstract
Convolutional Neural Networks (CNNs) are fundamental to advancing computer vision technologies. As CNNs become more complex and larger, optimizing model inference remains a critical challenge in both industry and academia. On modern GPU platforms, CNN operators are typically memory-bound, leading to significant performance degradation due to memory wall effects. While recent advancements have utilized operator fusion–merging multiple operators into one–to enhance inference performance, the fusion of multiple region-based operators like convolution is seldom addressed. This article introduces AFusion , a novel operator fusion technique aimed at improving inference performance, and OptiFX, an automatic optimization framework based on this approach. OptiFX employs a cost-based backtracking search to identify optimal sub-graphs for fusion and utilizes template-based code generation to create efficient kernels for these fused sub-graphs. We evaluate OptiFX across seven prominent CNN architectures–GoogLeNet, ResNet, DenseNet, MobileNet, SqueezeNet, NasNet, and UNet–on Nvidia A6000 Ada, RTX 4090, and Jetson AGX Orin platforms. Our results demonstrate that OptiFX significantly outperforms existing methods, achieving average speedups of \(2.91\times\) , \(3.30\times\) , and \(2.09\times\) in accelerating inference performance on these platforms, respectively.
Keywords
Funding Information
- National Science and Technology Major Project (2023ZD0120502)
- National Natural Science Foundation of China (62302479, 62232015, 62090024, and 62372055)
- Laboratory for Advanced Computing and Intelligence Engineering, the China Postdoctoral Science Foundation (2024M750258, and 2023M733566)
- CCF-Tencent Rhino-Bird Open Research Fund, State Key Lab of Processors, Institute of Computing Technology, CAS (CLQ202411)
- Innovation Funding of ICT, CAS (E361010, and E261110)
- Australian Research Council (DP250104934)
This publication has 30 references indexed in Scilit:
- A high-performance dataflow-centric optimization framework for deep learning inference on the edgeJournal of Systems Architecture, 2024
- Optimizing deep neural networks on intelligent edge accelerators via flexible-rate filter pruningJournal of Systems Architecture, 2022
- MLIR: Scaling Compiler Infrastructure for Domain Specific Computation2021
- Unleashing the Low-Precision Computation Potential of Tensor Cores on GPUs2021
- Jittor: a novel deep learning framework with meta-operators and unified graph executionScience China Information Sciences, 2020
- Accelerating Deep Learning Inference with Cross-Layer Data Reuse on GPUs2020
- Densely Connected Convolutional Networks2017
- Deep Residual Learning for Image Recognition2016
- Going deeper with convolutions2015
- ImageNet: A large-scale hierarchical image database2009