Benjamin Michalowicz
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Benjamin Michalowicz has a total of 38 co-authors. Here are the top co-authors by number of shared publications:
- Dhabaleswar Kumar Panda
The Ohio State University, Columbus, USA
5 shared publications - Stephen Poole
Los Alamos National Laboratory, Los Alamos, USA
4 shared publications - Hari Subramoni
The Ohio State University, Columbus, USA
3 shared publications - Michael Beebe
Texas Tech University, Lubbock, USA
3 shared publications - Wendy Poole
Los Alamos National Laboratory, Los Alamos, USA
3 shared publications - Kaushik Kandadi Suresh
Department of Computer Science and Engineering, The Ohio State University, Columbus, Ohio, USA
2 shared publications
Latest Publications
- Open Access
- 10 September 2026
- Published by Elsevier in SSRN Electronic Journal
CEVec: Cache-Efficient Vectorized Reduction Collectives
The Message Passing Interface (MPI) is the de facto programming model for developing parallel and distributed programs on high-performance computing (HPC) clusters. Among MPI collectives, reduction primitives are critical for the performance of both traditional HPC workloads and emerging AI/DL applications. Modern multi/many-core systems are equipped with hundreds of cores, deep memory hierarchies, and numerous Single Instruction Multiple Data (SIMD) units. Reduction collectives, which involve both communication and computation, are greatly affected by how efficiently we leverage such resources. However, current state-of-the-art reduction collectives suffer from two fundamental bottlenecks: inefficient data traversal, which is not cache-friendly for large messages, and poor SIMD utilization. We address these challenges by proposing CEVec: Cache-Efficient Vectorized Reduction Collectives. CEVec systematically combines three traversal schemes and two complementary SIMD execution strategies with NUMA-awareness. Unlike prior efforts that treat traversal order, SIMD usage, and NUMA-awareness as isolated techniques, CEVec integrates them into a unified framework for scalable reduction. At the microbenchmark level, CEVec achieves up to 19.9x lower latency for Reduce and Allreduce compared to state-of-the-art MPI implementations. At the application level, CEVec delivers up to 3.5x speedup in miniAMR with 4096 processes on NVIDIA Grace-Grace superchips, and up to 1.8x improvement in distributed deep learning training with Horovod on 512 nodes.
- 10 September 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessCEVec: Cache-Efficient Vectorized Reduction Collectives
The Message Passing Interface (MPI) is the de facto programming model for developing parallel and distributed programs on high-performance computing (HPC) clusters. Among MPI collectives, reduction primitives are critical for the performance of both traditional HPC workloads and emerging AI/DL applications. Modern multi/many-core systems are equipped with hundreds of cores, deep memory hierarchies, and numerous Single Instruction Multiple Data (SIMD) units. Reduction collectives, which involve both communication and computation, are greatly affected by how efficiently we leverage such resources. However, current state-of-the-art reduction collectives suffer from two fundamental bottlenecks: inefficient data traversal, which is not cache-friendly for large messages, and poor SIMD utilization. We address these challenges by proposing CEVec: Cache-Efficient Vectorized Reduction Collectives. CEVec systematically combines three traversal schemes and two complementary SIMD execution strategies with NUMA-awareness. Unlike prior efforts that treat traversal order, SIMD usage, and NUMA-awareness as isolated techniques, CEVec integrates them into a unified framework for scalable reduction. At the microbenchmark level, CEVec achieves up to 19.9x lower latency for Reduce and Allreduce compared to state-of-the-art MPI implementations. At the application level, CEVec delivers up to 3.5x speedup in miniAMR with 4096 processes on NVIDIA Grace-Grace superchips, and up to 1.8x improvement in distributed deep learning training with Horovod on 512 nodes.
- Open Access
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 238-252
Toward a Unified GPU-Aware OpenSHMEM Specification
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 238-252
Open AccessToward a Unified GPU-Aware OpenSHMEM Specification
- Open Access
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1075-1087
OpenSHMEM MLIR: A Dialect for Compile-Time Optimization of One-Sided Communications
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1075-1087
Open AccessOpenSHMEM MLIR: A Dialect for Compile-Time Optimization of One-Sided Communications
- Open Access
- 18 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-6
OpenSHMEM Performance on Bluefield-3 Data Processing Units (DPUs)
- 18 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Open AccessOpenSHMEM Performance on Bluefield-3 Data Processing Units (DPUs)
- 22 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Micro
- Vol. 45 (2) , 8-17
Understanding and Characterizing Communication Characteristics for Distributed Transformer Models
The transformer architecture has revolutionized many applications such as large language models. This progress has been largely enabled by distributed training, yet communication remains a significant bottleneck. This paper examines the communication behavior of transformer models, focusing on how different parallelism schemes in multi-node/multi-GPU training communicate data. We use GPT-based language models as a case study due to their prevalence. We validate our empirical results using analytical models. Our analysis reveals practical insights and potential areas for further optimization in framework and HPC middleware design.
- 22 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Micro
- Vol. 45 (2) , 8-17
Understanding and Characterizing Communication Characteristics for Distributed Transformer Models
The transformer architecture has revolutionized many applications such as large language models. This progress has been largely enabled by distributed training, yet communication remains a significant bottleneck. This paper examines the communication behavior of transformer models, focusing on how different parallelism schemes in multi-node/multi-GPU training communicate data. We use GPT-based language models as a case study due to their prevalence. We validate our empirical results using analytical models. Our analysis reveals practical insights and potential areas for further optimization in framework and HPC middleware design.
- Open Access
- 23 July 2023
- Published by Association for Computing Machinery (ACM)
- p. 94-101
DPU-Bench: A Micro-Benchmark Suite to Measure Offload Efficiency Of SmartNICs
Smart Network Interface Cards (SmartNIC) have experienced massive growth in popularity over the last few years such as the NVIDIA BlueField-2 Data Processing Unit (DPU). Being equipped with their own set of cores and memory allows them to perform actions beyond a regular NIC, and HPC researchers are designing new ways to use them. For example, offloading communication to one enables the CPU "host" to perform more computationally heavy tasks. However, one question remains: How much of that work can be distributed among processes placed on the SmartNIC before facing performance degradation? We present DPU-Bench: A low-level micro-benchmark suite using IB-Verbs primitives to enable HPC users to examine the number of processes to be placed on one or more SmartNICs in order to efficiently offload a given communication pattern. We examine direct algorithms in this paper at a medium scale with different work assignment mechanisms and give insights into the trends found with varying numbers of worker processes and message sizes.
- 23 July 2023
- Published by Association for Computing Machinery (ACM)
- p. 94-101
Open AccessDPU-Bench: A Micro-Benchmark Suite to Measure Offload Efficiency Of SmartNICs
Smart Network Interface Cards (SmartNIC) have experienced massive growth in popularity over the last few years such as the NVIDIA BlueField-2 Data Processing Unit (DPU). Being equipped with their own set of cores and memory allows them to perform actions beyond a regular NIC, and HPC researchers are designing new ways to use them. For example, offloading communication to one enables the CPU "host" to perform more computationally heavy tasks. However, one question remains: How much of that work can be distributed among processes placed on the SmartNIC before facing performance degradation? We present DPU-Bench: A low-level micro-benchmark suite using IB-Verbs primitives to enable HPC users to examine the number of processes to be placed on one or more SmartNICs in order to efficiently offload a given communication pattern. We examine direct algorithms in this paper at a medium scale with different work assignment mechanisms and give insights into the trends found with varying numbers of worker processes and message sizes.
- Open Access
- 21 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 477-487
Enabling Reconfigurable HPC through MPI-based Inter-FPGA Communication
- 21 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 477-487
Open AccessEnabling Reconfigurable HPC through MPI-based Inter-FPGA Communication