Christian Terboven

Power consumption of the main memory in modern heterogeneous high-performance computing (HPC) constitutes a significant part of the total power consumption of a node. This motivates energy-efficient solutions targeting the memory domain as well. Practitioners need reliable energy measurement techniques for analyzing energy and power consumption of applications and performance optimizations. Running Average Power Limit (RAPL) is a common choice, as it provides uncomplicated access to the energy measurements. While RAPL's accuracy has been studied and validated on homogeneous memory platforms, no work we are aware of investigated its accuracy on heterogeneous memory platforms, specifically with high-capacity memory (HCM). This paper describes the process of measuring the memory power consumption externally using riser cards in detail. We validate RAPL's accuracy by comparing results obtained from Intel's Ice Lake-SP system equipped with DDR4 DRAM and Intel Optane Persistent Memory Modules (PMM). In addition, we verify the accuracy of our instrumentation setup by comparing the results from an older Broadwell system with the results in the literature. We show that the RAPL values on a heterogeneous memory system report a higher offset from the reference measurements. The difference is more pronounced at lower memory load for all memory types. Also, we find that RAPL readings are inconsistent between multiple sockets and over time. Based on the evaluated scenarios, we conclude that RAPL overestimates the actual power consumption on heterogeneous memory systems and provide a discussion on the possible causes of this effect.

In High-Performance Computing, new use cases are currently emerging in which classical numerical simulations are coupled with machine learning as a surrogate for complex physical models that are expensive to compute. In the context of simulating reactive thermo-fluid systems, the idea to replace current state-of-the-art tabulated chemistry with machine learning inference is an active field of research. For this purpose, a simplified OpenFOAM application is coupled with an artificial neural network. In this work, we present a case study focusing solely on the performance of the coupled OpenFOAM-ML application. Our coupling approach features a heterogeneous cluster architecture combining pure CPU nodes and nodes equipped with two Nvidia V100 GPUs. We evaluate our approach by comparing the inference performance and the communication our approach induces with various machine learning frameworks. Additionally, we also compare the GPUs with NEC Vector Engine Type 10B regarding inference performance.

Hybrid MPI–OpenMP applications employ message-passing interface (MPI)-enabled process-level, distributed computations on many compute nodes in conjunction with OpenMP shared-memory, thread-level parallelism for the most efficient computation. This poses challenges on the dynamic MPI correctness tool MUST and TypeART, its memory allocation tracking sanitizer extension based on the LLVM compiler framework. In particular, at the thread-level granularity of a process, MPI calls and memory allocations, which are both tracked for our analysis, can occur concurrently. To correctly handle this situation, we: 1) extended our compiler extension to handle OpenMP and 2) introduced thread-safety mechanisms to our runtime libraries, thus keeping the tracking of data consistent and avoiding data races. Our approach exhibits acceptable runtime and memory overheads, both typically below 30%.