A PCIe Congestion-Aware Performance Model for Densely Populated Accelerator Servers
Authors:
- Maxime MartinassoMMMaxime Martinasso
- Grzegorz KwasniewskiGKGrzegorz Kwasniewski
- Sadaf R. AlamSASadaf R. Alam
- Thomas C. SchulthessTSThomas C. Schulthess
- Torsten HoeflerTHTorsten Hoefler
- 16 March 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 739-749
Abstract
MeteoSwiss, the Swiss national weather forecast institute, has selected densely populated accelerator servers as their primary system to compute weather forecast simulation. Servers with multiple accelerator devices that are primarily connected by a PCI-Express (PCIe) network achieve a significantly higher energy efficiency. Memory transfers between accelerators in such a system are subjected to PCIe arbitration policies. In this paper, we study the impact of PCIe topology and develop a congestion-aware performance model for PCIe communication. We present an algorithm for computing congestion factors of every communication in a congestion graph that characterizes the dynamic usage of network resources by an application. Our model applies to any PCIe tree topology. Our validation results on two different topologies of 8 GPU devices demonstrate that our model achieves an accuracy of over 97% within the PCIe network. We demonstrate the model on a weather forecast application to identify the best algorithms for its communication patterns among GPUs.Keywords
This publication has 10 references indexed in Scilit:
- Topology-Aware GPU Selection on Multi-GPU Nodes2016
- STELLA2015
- GPUSync: A Framework for Real-Time GPU Management2013
- PARTANSACM Transactions on Architecture and Code Optimization, 2013
- Optimization principles for collective neighborhood communications2012
- A Contention-Aware Performance Model for HPC-Based Networks: A Case Study of the InfiniBand NetworkLecture Notes in Computer Science, 2011
- Exploring the multiple-GPU design space2009
- Terascale direct numerical simulations of turbulent combustion using S3DComputational Science & Discovery, 2009
- Scalable Parallel Programming with CUDAQueue, 2008
- Multicommodity network flows—A surveyNetworks, 1978