Petascale direct numerical simulation of turbulent channel flow on up to 786K cores
Open Access
- 17 November 2013
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-11
Abstract
We present results of performance optimization for direct numerical simulation (DNS) of wall bounded turbulent flow (channel flow). DNS is a technique in which the fluid flow equations are solved without subgrid modeling. Of particular interest are high Reynolds number (Re) turbulent flows over walls, because of their importance in technological applications. Simulating high Re turbulence is a challenging computational problem, due to the high spatial and temporal resolution requirements. An optimized code was developed using spectral methods, the method of choice for turbulent flows. Optimization was performed to address three major issues: efficiency of banded matrix linear algebra, cache reuse and memory access, and communication for the global data transposes. Results show that performance is highly dependent on characteristics of the communication network, rather than singlecore performance. In our tests, it exhibits approximately 80% strong scaling parallel efficiency at 786K cores relative to performance on 65K cores.Keywords
Funding Information
- Petroleum Research Atlantic Canada (832634)
- Office of Cyberinfrastructure (OCI-0749223)
This publication has 14 references indexed in Scilit:
- Dissipation and enstrophy in isotropic turbulence: Resolution effects and scaling in direct numerical simulationsPhysics of Fluids, 2008
- What are we learning from simulating wall turbulence?Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2007
- Scaling of the velocity fluctuations in turbulent channels up to Reτ=2003Physics of Fluids, 2006
- The Design and Implementation of FFTW3Proceedings of the IEEE, 2005
- A Critical Evaluation of the Resolution Properties of B-Spline and Compact Finite Difference MethodsJournal of Computational Physics, 2001
- Direct numerical simulation of turbulent channel flow up to Reτ=590Physics of Fluids, 1999
- Direct Numerical Simulation of a Turbulent Reactive Plume on a Parallel ComputerJournal of Computational Physics, 1996
- Spectral methods for the Navier-Stokes equations with one infinite and two periodic directionsJournal of Computational Physics, 1991
- Spectral Methods in Fluid Dynamics1988
- Turbulence statistics in fully developed channel flow at low Reynolds numberJournal of Fluid Mechanics, 1987