Efficient Orchestrated AI Workflows Execution on Scale-Out Spatial Architecture
- 8 October 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Artificial Intelligence
- Vol. 1 (2) , 229-243
Abstract
Given the increasing complexity of AI applications, traditional spatial architectures frequently fall short. Our analysis identifies a pattern of interconnected, multi-faceted tasks encompassing both AI and general computational processes. In response, we have conceptualized “Orchestrated AI Workflows,” an approach that integrates various tasks with logic-driven decisions into dynamic, sophisticated workflows. Specifically, we find that the intrinsic Dual Dynamicity of Orchestrated AIWorkflows, namely dynamic execution times and frequencies of Task Blocks, can be effectively represented using the Orchestrated Workflow Graph. Furthermore, the intrinsic Dual Dynamicity poses challenges to existing spatial architecture, namely Indiscriminate Resource Allocation, Reactive Load Rebalancing, and Contagious PEA Idleness. To overcome these challenges, we present Octopus, a scale-out spatial architecture and a suite of advanced scheduling strategies optimized for executing Orchestrated AI Workflows, such as the Discriminate Dual-Scheduling Mechanism, Adaptive Task Block Processing Unit (TBU) Scheduling Strategy, and Proactive Cluster Scheduling Strategy. Our evaluations demonstrate that Octopus significantly outperforms traditional architectures in handling the dynamic demands of Orchestrated AI Workflows, and possesses robust scalability in large scale hardware such as wafer-scale chip.Keywords
All Related Versions
Funding Information
- National Science and Technology Major Project (2022ZD011520)
- NSFC (62125403, 92164301)
- Beijing ST Project (Z221100007722023)
- Beijing National Research Center For Information Science And Technology
- Beijing Advanced Innovation Center for Integrated Circuits
This publication has 45 references indexed in Scilit:
- SWG: an architecture for sparse weight gradient computationScience China Information Sciences, 2024
- HTDcr: a job execution framework for high-throughput computing on supercomputersScience China Information Sciences, 2023
- Reconfigurable spatial-parallel stochastic computing for accelerating sparse convolutional neural networksScience China Information Sciences, 2023
- Hetu: a highly efficient automatic parallel distributed deep learning systemScience China Information Sciences, 2022
- Scheduling multi-tenant cloud workflow tasks with resource reliabilityScience China Information Sciences, 2022
- An energy-efficient dynamically reconfigurable cryptographic engine with improved power/EM-side-channel-attack resistanceScience China Information Sciences, 2022
- Designing a 2048-Chiplet, 14336-Core Waferscale Processor2021
- A fully-customized dataflow engine for 3D earthquake simulation with a complex topographyScience China Information Sciences, 2021
- Communication and performance evaluation of 3-ary n-cubes onto network-on-chipsScience China Information Sciences, 2021
- CGRA-ME: A unified framework for CGRA modelling and exploration2017