Cooperative Warp Execution in Tensor Core for RISC-V GPGPU
Authors:
- 1 March 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1422-1436
Abstract
The rise of Deep Neural Networks (DNNs) has amplified the demand for efficient computation, with General Matrix Multiply (GEMM) operations at their core. While ASICs are efficient but inflexible, GPUs, especially NVIDIA GPUs, equipped with tensor cores, provide a flexible yet high-performance solution for GEMM-based workloads. Previous research and optimizations have largely centered on NVIDIA’s architecture and programming model, which, while effective, can obscure the rationale behind certain design decisions and limit flexibility for further improvements in tensor core designs. In this paper, we present the design and integration of a tensor core into the open-source RISC-V Vortex GPGPU platform, along with a suite of intrinsics designed for GEMM kernel generation. The analysis conducted on the integration elucidates the connections between GPU system architectural parameters and tensor core configuration. We find that the tensor core is severely under-utilized in many cases and that increased compute capacity does not always imply better performance. Hence, we propose a novel technique, cooperative warp execution in tensor core, which leverages hardware-supported warp cooperation within the tensor core to reduce memory requirements for GEMM operations and boost performance over the baseline tensor core implementation by up to 3x.Keywords
This publication has 24 references indexed in Scilit:
- Benchmarking and Dissecting the Nvidia Hopper GPU Architecture2024
- From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference2023
- Analyzing and Leveraging Decoupled L1 Caches in GPUs2021
- GPNPU: Enabling Efficient Hardware-Based Direct Convolution with Multi-Precision Support in GPU Tensor Cores2020
- Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling2020
- SIGMA: A Sparse and Irregular GEMM Accelerator with Flexible Interconnects for DNN Training2020
- Improving GPU Cache Hierarchy Performance with a Fetch and Replacement Cache2018
- NVIDIA Tensor Core Programmability, Performance & Precision2018
- WIR: Warp Instruction Reuse to Minimize Repeated Computations in GPUs2018
- MIAOW - An open source RTL implementation of a GPGPU2015