Tricking AI chips into Simulating the Human Brain: A Detailed Performance Analysis
Authors:
- Lennart P. L. LandsmeerLLLennart P. L. Landsmeer
- Max C. W. EngelenMEMax C. W. Engelen
- Rene MiedemaRMRene Miedema
- Christos StrydisCSChristos Strydis
Preprint
Abstract
Challenging the Nvidia monopoly, dedicated AI-accelerator chips have begun emerging for tackling the computational challenge that the inference and, especially, the training of modern deep neural networks (DNNs) poses to modern computers. The field has been ridden with studies assessing the performance of these contestants across various DNN model types. However, AI-experts are aware of the limitations of current DNNs and have been working towards the fourth AI wave which will, arguably, rely on more biologically inspired models, predominantly on spiking neural networks (SNNs). At the same time, GPUs have been heavily used for simulating such models in the field of computational neuroscience, yet AI-chips have not been tested on such workloads. The current paper aims at filling this important gap by evaluating multiple, cutting-edge AI-chips (Graphcore IPU, GroqChip, Nvidia GPU with Tensor Cores and Google TPU) on simulating a highly biologically detailed model of a brain region, the inferior olive (IO). This IO application stress-tests the different AI-platforms for highlighting architectural tradeoffs by varying its compute density, memory requirements and floating-point numerical accuracy. Our performance analysis reveals that the simulation problem maps extremely well onto the GPU and TPU architectures, which for networks of 125,000 cells leads to a 28x respectively 1,208x speedup over CPU runtimes. At this speed, the TPU sets a new record for largest real-time IO simulation. The GroqChip outperforms both platforms for small networks but, due to implementing some floating-point operations at reduced accuracy, is found not yet usable for brain simulation.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- Training Spiking Neural Networks Using Lessons From Deep LearningProceedings of the IEEE, 2023
- A domain-specific supercomputer for training deep neural networksCommunications of the ACM, 2020
- Think Fast: A Tensor Streaming Processor (TSP) for Accelerating Deep Learning Workloads2020
- NengoDL: Combining Deep Learning and Neuromorphic Modelling MethodsNeuroinformatics, 2019
- Arbor — A Morphologically-Detailed Neural Network Simulation Library for Contemporary High-Performance Computing Architectures2019
- A Mini Review of Neuromorphic Architectures and ImplementationsIEEE Transactions on Electron Devices, 2016
- The Inferior Olive of the C57BL/6J Mouse: A Chemoarchitectonic StudyThe Anatomical Record, 2014
- The NEURON Book2006
- Cortex: Statistics and Geometry of Neuronal Connectivity1998
- Networks of spiking neurons: The third generation of neural network modelsNeural Networks, 1997