Andrei Ivanov
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Andrei Ivanov has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Torsten Hoefler
ETH Zurich, Zurich, Switzerland
3 shared publications - Tal Ben-Nun
Lawrence Livermore National Laboratory
2 shared publications - Timo Schneider
ETH Zurich, Zurich, Switzerland
2 shared publications - Afif Boudaoud
ETH Zurich, Zurich, Switzerland
1 shared publication - Basilio B. Fraguela
Universidade da Coruña, CITIC, Computer Architecture Group, A Coruña, Spain
1 shared publication - Diego Andrade
Universidade da Coruña
1 shared publication
Latest Publications
- Open Access
- 27 June 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
- Vol. 23 (2) , 1-25
STen: Productive and Efficient Sparsity in PyTorch
As deep learning models grow, sparsity is becoming an increasingly critical component of deep neural networks, enabling improved performance and reduced storage. However, existing frameworks offer poor support for sparsity. Specialized sparsity engines focus exclusively on sparse inference, while general frameworks primarily focus on sparse tensors in classical formats and neglect the broader sparsification pipeline necessary for using sparse models, especially during training. Further, existing frameworks are not easily extensible: adding a new sparse tensor format or operator is challenging and time-consuming. To address this, we propose STen, a sparsity programming model and interface for PyTorch whose key design insight is the decoupling of sparsity layouts, operators, and sparsifiers into composable, first-class abstractions that users can independently define and combine. An automatic dispatch mechanism selects the best available sparse implementation and transparently falls back to dense execution, allowing STen to support virtually all sparsification methods while enabling rapid prototyping without sacrificing performance for optimized paths. We demonstrate the versatility of STen by expressing existing sparsification techniques within its abstraction, achieving a code size reduction of over 2×. Finally, we develop a novel, high-performance grouped n : m sparsity layout for CPU inference at moderate sparsity, accelerating end-to-end BERT BASE inference by up to 3.2×. STen brings high performance and ease of use, making sparsity readily accessible for existing PyTorch models.
- 27 June 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
- Vol. 23 (2) , 1-25
Open AccessSTen: Productive and Efficient Sparsity in PyTorch
As deep learning models grow, sparsity is becoming an increasingly critical component of deep neural networks, enabling improved performance and reduced storage. However, existing frameworks offer poor support for sparsity. Specialized sparsity engines focus exclusively on sparse inference, while general frameworks primarily focus on sparse tensors in classical formats and neglect the broader sparsification pipeline necessary for using sparse models, especially during training. Further, existing frameworks are not easily extensible: adding a new sparse tensor format or operator is challenging and time-consuming. To address this, we propose STen, a sparsity programming model and interface for PyTorch whose key design insight is the decoupling of sparsity layouts, operators, and sparsifiers into composable, first-class abstractions that users can independently define and combine. An automatic dispatch mechanism selects the best available sparse implementation and transparently falls back to dense execution, allowing STen to support virtually all sparsification methods while enabling rapid prototyping without sacrificing performance for optimized paths. We demonstrate the versatility of STen by expressing existing sparsification techniques within its abstraction, achieving a code size reduction of over 2×. Finally, we develop a novel, high-performance grouped n : m sparsity layout for CPU inference at moderate sparsity, accelerating end-to-end BERT BASE inference by up to 3.2×. STen brings high performance and ease of use, making sparsity readily accessible for existing PyTorch models.
- Open Access
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 137-151
PerfDojo: Automated ML Library Generation for Heterogeneous Architectures
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 137-151
Open AccessPerfDojo: Automated ML Library Generation for Heterogeneous Architectures
- Open Access
- 11 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-14
VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores
The increasing success and scaling of Deep Learning models demands higher computational efficiency and power. Sparsification can lead to both smaller models as well as higher compute efficiency, and accelerated hardware is becoming available. However, exploiting it efficiently requires kernel implementations, pruning algorithms, and storage formats, to utilize hardware support of specialized sparse vector units. An example of those are the NVIDIA's Sparse Tensor Cores (SPTCs), which promise a 2× speedup. However, SPTCs only support the 2:4 format, limiting achievable sparsity ratios to 50%. We present the V:N:M format, which enables the execution of arbitrary N:M ratios on SPTCs. To efficiently exploit the resulting format, we propose Spatha, a high-performance sparse-library for DL routines. We show that Spatha achieves up to 37× speedup over cuBLAS. We also demonstrate a second-order pruning technique that enables sparsification to high sparsity ratios with V:N:M and little to no loss in accuracy in modern transformers.
- 11 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-14
Open AccessVENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores
The increasing success and scaling of Deep Learning models demands higher computational efficiency and power. Sparsification can lead to both smaller models as well as higher compute efficiency, and accelerated hardware is becoming available. However, exploiting it efficiently requires kernel implementations, pruning algorithms, and storage formats, to utilize hardware support of specialized sparse vector units. An example of those are the NVIDIA's Sparse Tensor Cores (SPTCs), which promise a 2× speedup. However, SPTCs only support the 2:4 format, limiting achievable sparsity ratios to 50%. We present the V:N:M format, which enables the execution of arbitrary N:M ratios on SPTCs. To efficiently exploit the resulting format, we propose Spatha, a high-performance sparse-library for DL routines. We show that Spatha achieves up to 37× speedup over cuBLAS. We also demonstrate a second-order pruning technique that enables sparsification to high sparsity ratios with V:N:M and little to no loss in accuracy in modern transformers.