QiMeng-GEMM: Automatically Generating High-Performance Matrix Multiplication Code by Exploiting Large Language Models
Authors:
Open Access
- 11 April 2025
- journal article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (21) , 22982-22990
Abstract
As a crucial operator in numerous scientific and engineering computing applications, the automatic optimization of General Matrix Multiplication (GEMM) with full utilization of ever-evolving hardware architectures (e.g. GPUs and RISC-V) is of paramount importance. While Large Language Models (LLMs) can generate functionally correct code for simple tasks, they have yet to produce high-performance code. The key challenge resides in deeply understanding diverse hardware architectures and crafting prompts that effectively unleash the potential of LLMs to generate high-performance code. In this paper, we propose a novel prompt mechanism called QiMeng-GEMM which enables LLMs to comprehend the architectural characteristics of different hardware platforms and automatically search for the optimization combinations for GEMM. The key of QiMeng-GEMM is a set of informative, adaptive, and iterative meta-prompts. Based on this, a searching strategy for optimal combinations of meta-prompts is used to iteratively generate high-performance code. Extensive experiments conducted on 4 leading LLMs, various paradigmatic hardware platforms, and representative matrix dimensions unequivocally demonstrate QiMeng-GEMM’s superior performance in auto-generating optimized GEMM code. Compared to vanilla prompts, our method achieves a performance enhancement of up to 113×. Even when compared to human experts, our method can reach 115% of cuBLAS on NVIDIA GPUs and 211% of OpenBLAS on RISC-V CPUs. Notably, while human experts often take months to optimize GEMM, our approach reduces the development cost by over 240×.Keywords
This publication has 11 references indexed in Scilit:
- IrGEMM: An Input-Aware Tuning Framework for Irregular GEMM on ARM and X86 CPUsIEEE Transactions on Parallel and Distributed Systems, 2024
- CodePlan: Repository-Level Coding using LLMs and PlanningProceedings of the ACM on Software Engineering, 2024
- CodeT5+: Open Code Large Language Models for Code Understanding and Generation2023
- Competition-level code generation with AlphaCodeScience, 2022
- Flexible Performant GEMM Kernels on GPUsIEEE Transactions on Parallel and Distributed Systems, 2021
- Efficient Auto-Tuning of Parallel Programs with Interdependent Tuning Parameters via Auto-Tuning Framework (ATF)ACM Transactions on Architecture and Code Optimization, 2021
- Will RISC-V revolutionize computing?2020
- A new golden age for computer architectureCommunications of the ACM, 2019
- Finding good optimization sequences covering program spaceACM Transactions on Architecture and Code Optimization, 2013
- Autotuning GEMM Kernels for the Fermi GPUIEEE Transactions on Parallel and Distributed Systems, 2012