Qize Yang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Qize Yang has a total of 27 co-authors. Here are the top co-authors by number of shared publications:
- Shouyi Yin
Tsinghua University
4 shared publications - Yang Hu
Tsinghua University
4 shared publications - Chao Li
Shanghai Jiao Tong University, Shanghai, China
2 shared publications - Huizheng Wang
School of Integrated Circuits, Tsinghua University, Beijing, China
2 shared publications - Jinyi Deng
Tsinghua University, Beijing, Beijing, China
2 shared publications - Shaojun Wei
Tsinghua University
2 shared publications
Latest Publications
- 4 November 2025
- Published by Springer Nature
- p. 34-46
Segmentation-Aware Optimization of Collective for Waferscale Chips
- 4 November 2025
- Published by Springer Nature
- p. 34-46
Segmentation-Aware Optimization of Collective for Waferscale Chips
- Open Access
- 20 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 49-64
PD Constraint-aware Physical/Logical Topology Co-Design for Network on Wafer
- 20 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 49-64
Open AccessPD Constraint-aware Physical/Logical Topology Co-Design for Network on Wafer
- 8 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Artificial Intelligence
- Vol. 1 (2) , 229-243
Efficient Orchestrated AI Workflows Execution on Scale-Out Spatial Architecture
Given the increasing complexity of AI applications, traditional spatial architectures frequently fall short. Our analysis identifies a pattern of interconnected, multi-faceted tasks encompassing both AI and general computational processes. In response, we have conceptualized “Orchestrated AI Workflows,” an approach that integrates various tasks with logic-driven decisions into dynamic, sophisticated workflows. Specifically, we find that the intrinsic Dual Dynamicity of Orchestrated AIWorkflows, namely dynamic execution times and frequencies of Task Blocks, can be effectively represented using the Orchestrated Workflow Graph. Furthermore, the intrinsic Dual Dynamicity poses challenges to existing spatial architecture, namely Indiscriminate Resource Allocation, Reactive Load Rebalancing, and Contagious PEA Idleness. To overcome these challenges, we present Octopus, a scale-out spatial architecture and a suite of advanced scheduling strategies optimized for executing Orchestrated AI Workflows, such as the Discriminate Dual-Scheduling Mechanism, Adaptive Task Block Processing Unit (TBU) Scheduling Strategy, and Proactive Cluster Scheduling Strategy. Our evaluations demonstrate that Octopus significantly outperforms traditional architectures in handling the dynamic demands of Orchestrated AI Workflows, and possesses robust scalability in large scale hardware such as wafer-scale chip.
- 8 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Artificial Intelligence
- Vol. 1 (2) , 229-243
Efficient Orchestrated AI Workflows Execution on Scale-Out Spatial Architecture
Given the increasing complexity of AI applications, traditional spatial architectures frequently fall short. Our analysis identifies a pattern of interconnected, multi-faceted tasks encompassing both AI and general computational processes. In response, we have conceptualized “Orchestrated AI Workflows,” an approach that integrates various tasks with logic-driven decisions into dynamic, sophisticated workflows. Specifically, we find that the intrinsic Dual Dynamicity of Orchestrated AIWorkflows, namely dynamic execution times and frequencies of Task Blocks, can be effectively represented using the Orchestrated Workflow Graph. Furthermore, the intrinsic Dual Dynamicity poses challenges to existing spatial architecture, namely Indiscriminate Resource Allocation, Reactive Load Rebalancing, and Contagious PEA Idleness. To overcome these challenges, we present Octopus, a scale-out spatial architecture and a suite of advanced scheduling strategies optimized for executing Orchestrated AI Workflows, such as the Discriminate Dual-Scheduling Mechanism, Adaptive Task Block Processing Unit (TBU) Scheduling Strategy, and Proactive Cluster Scheduling Strategy. Our evaluations demonstrate that Octopus significantly outperforms traditional architectures in handling the dynamic demands of Orchestrated AI Workflows, and possesses robust scalability in large scale hardware such as wafer-scale chip.
- Open Access
- 1 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Integrated Circuits and Systems
- Vol. 1 (4) , 178-195
TMAC: Training-Targeted Mapping and Architecture Co-Exploration for Wafer-Scale Chips
Transformer-based large language models (LLMs) have made significant strides in the field of artificial intelligence (AI). However, training these LLMs imposes immense demands on computational power and bandwidth for hardware systems. Wafer-scale chips (WSCs) offer a promising solution, yet they struggle with limited on-chip memory and complex tensor partitioning. To fully harness the high-bandwidth, low-latency on-chip interconnect benefits of WSCs and to alleviate the on-chip memory limitations, a specialized mapping and architecture co-exploration method is essential. Despite existing efforts in memory optimization and mapping, current approaches fall short for WSC scenarios. To bridge this gap, we introduce TMAC, an architecture-mapping co-exploration framework that integrates recomputation into the design space, fully exploiting optimization opportunities overlooked by existing works. Further, TMAC takes advantage of the superior on-chip interconnect performance of WSCs by incorporating a more flexible tensor partition scheme. TMAC then introduces a novel operator-centric encoding scheme (OCES) designed to comprehensively describe the mapping space for training LLMs. Unlike previous studies that focus solely on communication volume analysis based on mapping, TMAC explores the design space by evaluating the combined impact of mapping and architecture on training performance. However, fully accounting for these untapped optimization opportunities increases the complexity of the design space. To address this, we streamline the simulation process, reducing the time needed for exploration. Compared to AccPar, Deepspeed and Megatron, TMAC delivers a $3.1\times$ , $2.9\times$ , $1.6\times$ performance gain. In terms of memory usage, TMAC requires $3.6\times$ , $3.1\times$ less memory than AccPar and Deepspeed, respectively and is comparable to Megatron's full recomputation method.
- 1 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Integrated Circuits and Systems
- Vol. 1 (4) , 178-195
Open AccessTMAC: Training-Targeted Mapping and Architecture Co-Exploration for Wafer-Scale Chips
Transformer-based large language models (LLMs) have made significant strides in the field of artificial intelligence (AI). However, training these LLMs imposes immense demands on computational power and bandwidth for hardware systems. Wafer-scale chips (WSCs) offer a promising solution, yet they struggle with limited on-chip memory and complex tensor partitioning. To fully harness the high-bandwidth, low-latency on-chip interconnect benefits of WSCs and to alleviate the on-chip memory limitations, a specialized mapping and architecture co-exploration method is essential. Despite existing efforts in memory optimization and mapping, current approaches fall short for WSC scenarios. To bridge this gap, we introduce TMAC, an architecture-mapping co-exploration framework that integrates recomputation into the design space, fully exploiting optimization opportunities overlooked by existing works. Further, TMAC takes advantage of the superior on-chip interconnect performance of WSCs by incorporating a more flexible tensor partition scheme. TMAC then introduces a novel operator-centric encoding scheme (OCES) designed to comprehensively describe the mapping space for training LLMs. Unlike previous studies that focus solely on communication volume analysis based on mapping, TMAC explores the design space by evaluating the combined impact of mapping and architecture on training performance. However, fully accounting for these untapped optimization opportunities increases the complexity of the design space. To address this, we streamline the simulation process, reducing the time needed for exploration. Compared to AccPar, Deepspeed and Megatron, TMAC delivers a $3.1\times$ , $2.9\times$ , $1.6\times$ performance gain. In terms of memory usage, TMAC requires $3.6\times$ , $3.1\times$ less memory than AccPar and Deepspeed, respectively and is comparable to Megatron's full recomputation method.