Cygnus - World First Multihybrid Accelerated Cluster with GPU and FPGA Coupling
Authors:
Open Access
- 29 August 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-8
Abstract
In this paper, we describe the concept, system architecture, supporting system software, and applications on our world-first supercomputer with multihybrid accelerators using GPU and FPGA coupling, named Cygnus, which runs at Center for Computational Sciences, University of Tsukuba. A special group of 32 nodes is configured as a multihybrid accelerated computing system named Albireo part although Cygnus is constructed with over 80 computation nodes as a GPU-accelerated PC cluster. Each node of the Albireo part is equipped with four NVIDIA V100 GPU cards and two Intel Stratix10 FPGA cards in addition to two sockets of Intel Xeon Gold CPU where all nodes are connected by four lanes of InfiniBand HDR100 interconnection HCA in the full bisection bandwidth of NVIDIA HDR200 switches. Beside this ordinary interconnection network, all FPGA cards in Albireo part are connected by a special 2-Dimensional Torus network with direct optical links on each FPGA for constructing a very high throughput and low latency of FPGA-centric interconnection network. To the best of our knowledge, Cygnus is the world’s first production-level PC cluster to realize multihybrid acceleration with the GPU and FPGA combination. Unlike other GPU-accelerated clusters, users can program parallel codes where each process exploits both or either of the GPU and/or FPGA devices based on the characteristics of their applications. We developed various supporting system software such as inter-FPGA network routing system, DMA engine for GPU-FPGA direct communication managed by FPGA, and multihybrid accelerated programming framework because the programming method of such a complicated system has not been standardized. Further, we developed the first real application on Cygnus for fundamental astrophysics simulation to fully utilize GPU and FPGA together for very efficient acceleration. We describe the overall concept and construction of the Cygnus cluster with a brief introduction of the several underlying hardware and software research studies that have already been published. We summarize how such a concept of GPU/FPGA coworking will usher in a new era of accelerated supercomputing.Keywords
This publication has 10 references indexed in Scilit:
- Multi-FPGA Designs and Scaling of HPC Challenge Benchmarks via MPI and Circuit-Switched Inter-FPGA NetworksarXiv, 2022
- Multi-hetero Acceleration by GPU and FPGA for Astrophysics Simulation on oneAPI Environment2022
- Performance Evaluation of OpenCL-Enabled Inter-FPGA Optical Link Communication Framework CIRCUS and SMI2021
- OpenCL-enabled Parallel Raytracing for Astrophysical Application on Multiple FPGAs with Optical Links2020
- Accelerating Radiative Transfer Simulation with GPU-FPGA Cooperative Computation2020
- Performance Evaluation of Pipelined Communication Combined with Computation in OpenCL Programming on FPGA2020
- Multi-Hybrid Accelerated Simulation by GPU and FPGA on Radiative Transfer Simulation in AstrophysicsJournal of Information Processing, 2020
- Streaming message interface2019
- GPU-FPGA Heterogeneous Computing with OpenCL-Enabled Direct Memory Access2019
- Accelerating Space Radiative Transfer on FPGA using OpenCL2018