David Goz

New challenges in Astronomy and Astrophysics (AA) are urging the need for many exceptionally computationally intensive simulations. “Exascale” (and beyond) computational facilities are mandatory to address the size of theoretical problems and data coming from the new generation of observational facilities in AA. Currently, the High-Performance Computing (HPC) sector is undergoing a profound phase of innovation, in which the primary challenge to the achievement of the “Exascale” is the power consumption. The goal of this work is to give some insights about performance and energy footprint of contemporary architectures for a real astrophysical application in an HPC context. We use a state-of-the-art N-body application that we re-engineered and optimized to exploit the heterogeneous underlying hardware fully. We quantitatively evaluate the impact of computation on energy consumption when running on four different platforms. Two of them represent the current HPC systems (Intel-based and equipped with NVIDIA GPUs), one is a micro-cluster based on ARM-MPSoC, and one is a “prototype towards Exascale” equipped with ARM-MPSoCs tightly coupled with FPGAs. We investigate the behavior of the different devices where the high-end GPUs excel in terms of time-to-solution while MPSoC-FPGA systems outperform GPUs in power consumption. Our experience reveals that considering FPGAs for computationally intensive application seems very promising, as their performance is improving to meet the requirements of scientific applications. This work can be a reference for future platform development for astrophysics applications where computationally intensive calculations are required.

New challenges in Astronomy and Astrophysics (AA) are urging the need for a large number of exceptionally computationally intensive simulations. "Exascale" (and beyond) computational facilities are mandatory to address the size of theoretical problems and data coming from the new generation of observational facilities in AA. Currently, the High Performance Computing (HPC) sector is undergoing a profound phase of innovation, in which the primary challenge to the achievement of the "Exascale" is the power-consumption. The goal of this work is to give some insights about performance and energy footprint of contemporary architectures for a real astrophysical application in an HPC context. We use a state-of-the-art N-body application that we re-engineered and optimized to exploit the heterogeneous underlying hardware fully. We quantitatively evaluate the impact of computation on energy consumption when running on four different platforms. Two of them represent the current HPC systems (Intel-based and equipped with NVIDIA GPUs), one is a micro-cluster based on ARM-MPSoC, and one is a "prototype towards Exascale" equipped with ARM-MPSoCs tightly coupled with FPGAs. We investigate the behavior of the different devices where the high-end GPUs excel in terms of time-to-solution while MPSoC-FPGA systems outperform GPUs in power consumption. Our experience reveals that considering FPGAs for computationally intensive application seems very promising, as their performance is improving to meet the requirements of scientific applications. This work can be a reference for future platforms development for astrophysics applications where computationally intensive calculations are required.

The aim of this work is to quantitatively evaluate the impact of computation on the energy consumption on ARM MPSoC platforms, exploiting CPUs, embedded GPUs and FPGAs. One of them possibly represents the future of High Performance Computing systems: a prototype of an Exascale supercomputer. Performance and energy measurements are made using a state-of-the-art direct $N$-body code from the astrophysical domain. We provide a comparison of the time-to-solution and energy delay product metrics, for different software configurations. We have shown that FPGA technologies can be used for application kernel acceleration and are emerging as a promising alternative to "traditional" technologies for HPC, which purely focus on peak-performance than on power-efficiency.

This work arises on the environment of the ExaNeSt project aiming at design and development of an exascale ready supercomputer with low energy consumption profile but able to support the most demanding scientific and technical applications. The ExaNeSt compute unit consists of densely-packed low-power 64-bit ARM processors, embedded within Xilinx FPGA SoCs. SoC boards are heterogeneous architecture where computing power is supplied both by CPUs and GPUs, and are emerging as a possible low-power and low-cost alternative to clusters based on traditional CPUs. A state-of-the-art direct $N$-body code suitable for astrophysical simulations has been re-engineered in order to exploit SoC heterogeneous platforms based on ARM CPUs and embedded GPUs. Performance tests show that embedded GPUs can be effectively used to accelerate real-life scientific calculations, and that are promising also because of their energy efficiency, which is a crucial design in future exascale platforms.

The architecture of Exascale computing facilities, which involves millions of heterogeneous processing units, will deeply impact on scientific applications. Future astrophysical HPC applications must be designed to make such computing systems exploitable. The ExaNeSt H2020 EU-funded project aims to design and develop an exascale ready prototype based on low-energy-consumption ARM64 cores and FPGA accelerators. We participate to the design of the platform and to the validation of the prototype with cosmological N-body and hydrodynamical codes suited to perform large-scale, high-resolution numerical simulations of cosmic structures formation and evolution. We discuss our activities on astrophysical applications to take advantage of the underlying architecture.

We present predictions of Spectral Energy Distributions (SEDs), from the UV to the FIR, of simulated galaxies at z = 0. These were obtained by post-processing the results of an N-body+hydro simulation of a cosmological box of side 25 Mpc, that uses the Multi-Phase Particle Integrator (MUPPI) for star formation and stellar feedback, with the grasil-3d radiative transfer code, that includes reprocessing of UV light by dust. Physical properties of our sample of ∼500 galaxies resemble observed ones, though with some tension at small and large stellar masses. Comparing predicted SEDs of simulated galaxies with different samples of local galaxies, we find that these resemble observed ones, when normalised at 3.6 μm. A comparison with the Herschel Reference Survey shows that average SEDs of galaxies, divided in bins of Star Formation Rate (SFR), are reproduced in shape and absolute normalization to within a factor of ∼2, while average SEDs of galaxies divided in bins of stellar mass show tensions that are an effect of the difference of simulated and observed galaxies in the stellar mass – SFR plane. We use our sample to investigate the correlation of IR luminosity in Spitzer and Herschel bands with several galaxy properties. SFR is the quantity that best correlates with IR light up to 160 μm, while at longer wavelengths better correlations are found with molecular mass and, at 500 μm, with dust mass. However, using the position of the FIR peak as a proxy for cold dust temperature, we assess that heating of cold dust is mostly determined by SFR, with stellar mass giving only a minor contribution. We finally show how our sample of simulated galaxies can be used as a guide to understand the physical properties and selection biases of observed samples.

We present a quantification of the properties of bars in two N-body+ smoothed particle hydrodynamics cosmological simulations of spiral galaxies, named GA and AqC. The initial conditions were obtained using the zoom-in technique and represent two dark matter haloes of 2–3 × 10¹² M_⊙, available at two different resolutions. The resulting galaxies are presented in the companion paper of Murante et al. We find that the GA galaxy has a bar of length 8.8 kpc, present at the two resolution levels even though with a slightly different strength. Classical bar signatures (e.g. pattern of streaming motions, high m = 2 Fourier mode with roughly constant phase) are consistently found at both resolutions. Though a close encounter with a merging satellite at z ∼ 0.6 (mass ratio 1: 50) causes a strong, transient spiral pattern and some heating of the disc, we find that bar instability is due to secular process, caused by a low Toomre parameter Q ≲ 1 due to accumulation of mass in the disc. The AqC galaxy has a slightly different history: it suffers a similar tidal disturbance due to a merging satellite at z ∼ 0.5 but with a mass ratio of 1: 32, that triggers a bar in the high-resolution simulation, while at low resolution the merging is found to take place at a later time, so that both secular evolution and merging are plausible triggers for bar instability.

We present results of cosmological simulations of disc galaxies carried out with the gadget-3 TreePM+SPH code, where star formation and stellar feedback are described using our MUlti Phase Particle Integrator model. This description is based on a simple multiphase model of the interstellar medium at unresolved scales, where mass and energy flows among the components are explicitly followed by solving a system of ordinary differential equations. Thermal energy from supernovae is injected into the local hot phase, so as to avoid that it is promptly radiated away. A kinetic feedback prescription generates the massive outflows needed to avoid the overproduction of stars. We use two sets of zoomed-in initial conditions of isolated cosmological haloes with masses (2–3) × 10¹² M_⊙, both available at several resolution levels. In all cases we obtain spiral galaxies with small bulge-over-total stellar mass ratios (B/T ∼ 0.2), extended stellar and gas discs, flat rotation curves and realistic values of stellar masses. Gas profiles are relatively flat, molecular gas is found to dominate at the centre of galaxies, with star formation rates following the observed Schmidt–Kennicutt relation. Stars kinematically belonging to the bulge form early, while disc stars show a clear inside-out formation pattern and mostly form after redshift z = 2. However, the baryon conversion efficiencies in our simulations differ from the relation given by Moster et al. at a 3σ level, thus indicating that our stellar discs are still too massive for the dark matter halo in which they reside. Results are found to be remarkably stable against resolution. This further demonstrates the feasibility of carrying out simulations producing a realistic population of galaxies within representative cosmological volumes, at a relatively modest resolution.