Yuanbo Wen

Automated processor design, which significantly reduces manual efforts and accelerates design cycles, has been a key focus of recent research. Although prior work has achieved progress in designing simple processors, such as single-issue processors, their performance falls short of modern superscalar processors that execute multiple instructions per cycle. Current state-of-the-art methods fail to design superscalar processors due to their inability to handle inter-instruction dependencies effectively, which limits parallelism and results in inefficient execution. This work proposes a novel framework for automated superscalar processor design by introducing a hardware-friendly model: Stateful Binary Speculation Diagram (State-BSD). We observe that processor performance can be significantly improved by reusing the processor’s internal states to learn the data dependency. To address the challenge of limited hardware resources while maintaining high prediction accuracy, the proposed State-BSD includes three key components: 1) a lightweight state selector, trained with the simulated annealing method, which identifies the most reusable instructions and stores them in a small buffer, 2) an efficient state-replacer to dynamically replace unused states in the buffer under limited buffer size, and 3) a highly accurate state speculator which predicts interinstruction data dependencies using the buffered states. This work demonstrates the first automated design of superscalar processors with hardware-level optimizations. The processor designed using the proposed automated design approach achieves a 431.28× performance improvement with only 4.25× more area.

Optimizing deep learning inference, particularly reducing the execution latency of tensor computations at small batch sizes, is crucial for the successful and widespread adoption of deep neural network (DNN) models. However, current deep learning compilers and hand-tuned libraries often fail to achieve high hardware efficiency when executing small-batch workloads. The primary reason is the inherently sequential nature of reductions (e.g., along the hidden dimension in the flattened GEMM for LLM decoding), which is difficult to parallelize and therefore fails to fully utilize available hardware resources. In this article, we propose Swift, a novel search-based approach for efficiently generating high-performance programs for GPUs by maximizing hardware utilization. The key insight is that reduction parallelization can be incorporated into a unified representation alongside the existing tile structure, significantly expanding the search space for high-performance programs. Concretely, by enumerating all possible parallel mappings of loops, we first generate a large search space that contains high-performance programs. Then, to efficiently explore the extended search space, we employ subspace shifting exploration to identify promising regions, effectively prune large portions of the less-promising search space. We conduct experiments on three distinct GPU architectures using a diverse set of benchmarks representative of typical application scenarios. Experimental results demonstrate that Swift achieves an average speedup of 1.19× over the state-of-the-art compiler-based approaches. Moreover, compared with vendor-provided hand-tuned libraries, Swift achieves an average speedup of 2.40×.

Compilers, while essential, are notoriously complex systems that demand prohibitively expensive human expertise to develop and maintain. The recent advancements in Large Language Models (LLMs) offer a compelling new paradigm: Neural Compilation, which could potentially simplify compiler development for new architectures and facilitate the discovery of innovative optimization techniques. However, several critical obstacles impede its practical adoption. Firstly, a significant lack of dedicated benchmarks and robust evaluation methodologies hinders objective assessment and tracking of progress in the field. Secondly, systematically enhancing the reliability and performance of LLM-generated assembly remains a critical challenge. Addressing these challenges, this paper introduces NeuComBack, a novel benchmark dataset specifically designed for IR-to-assembly compilation. Leveraging this dataset, we first define a foundational Neural Compilation workflow and conduct a comprehensive evaluation of the capabilities of recent frontier LLMs on Neural Compilation, establishing new performance baselines. We further propose a self-evolving prompt optimization method that enables LLMs to iteratively evolve their internal prompt strategies by extracting insights from prior self-debugging traces, thereby enhancing their neural compilation capabilities. Experiments demonstrate that our method significantly improves both the functional correctness and the performance of LLM-generated assembly code. Compared to baseline prompts, the functional correctness rates improved from 44% to 64% on x86_64 and from 36% to 58% on aarch64, respectively. More significantly, among the 16 correctly generated x86_64 programs using our method, 14 (87.5%) surpassed clang-O3 performance.

Customized processors are essential for domain-specific applications such as the Internet of Things (IoT) and multimedia embedded systems, yet their design often requires extensive expert intervention. Traditional approaches, including hardware design using encapsulated abstractions (e.g., Chisel) and high-level synthesis (HLS) from languages like C or SystemC, reduce some manual efforts but remain either costly or suboptimal. Recent explorations into leveraging large language models (LLMs) to generate register-transfer level (RTL) from natural language specifications have shown promise, but these methods still struggle with generating complex and high-performance processors, mainly due to the complicated low-level details in the RTL code. In this work, we introduce AGON, a novel framework designed to facilitate the development of customized processor RTL from instruction set architecture (ISA) documents using LLMs. The framework comprises two layers: a functional description layer and a hardware implementation layer. At the functional layer, AGON employs a nano-operator (nOP)-based intermediate representation (IR) that abstracts basic instruction operations, thereby reducing the semantic gap between natural language and the RTL code. This abstraction significantly shortens the descriptive code required for LLM generation, improving the generation accuracy in a single pass. At the hardware layer, AGON offers three abstraction levels (i.e., instruction, ISA, and processor) along with rule-based primitives to systematically lower the nOP-based IR into a fully optimized processor implementation. This decoupled design not only ensures correctness-by-construction but also enables automated, power, performance, and area (PPA)-aware performance optimization. We evaluate AGON by designing high-performance out-of-order (OoO) processors that correctly execute practical programs. Experimental results demonstrate that processors generated with AGON achieve an average speed of $4.51\times $ on specific tasks compared to expert-designed general-purpose central processing units (CPUs) while requiring minimal design effort.

Chiplet-based DNN accelerators have been extensively explored to save design and manufacturing costs. Previous works regard all chiplets as identical and employ uniform workload partitioning strategies. These workload partitioning strategies overlook various real-world factors that contribute to remarkable performance variations among chiplets, including manufacturing process variation, thermal condition, physical placement, and power supply condition. When considering these performance variations, a variation-aware workload partitioning can achieve superior performance. This article introduces VariPar, a systematic framework to employ variation-aware partitioning strategy in chiplet-based DNN accelerators. VariPar models performance variations for each chiplet and partition workloads accordingly. VariPar includes a simulator with multifactor variation modeling and a heuristic search engine to generate near-optimal partitioning within a reasonable time. Experiment results show that VariPar achieves $1.45{\times }$ performance and $1.82{\times }$ energy efficiency improvement on average when compared to uniform partitioning (UP) strategy.