Naifeng Jing

Multi-task transformers have garnered widespread attention for fast task adaptation at edge. However, in addition to matrix-vector multiplication with static and dynamic weights in the data-intensive transformer, task adaptation introduces new challenge of weight updating. Besides, while sparsity can help mitigate the rapid growth in parameter size and computational demand, it requires great hardware flexibility. Compute-in-memory (CIM) is widely explored to overcome the memory bottleneck in neural network acceleration, mainly with single type of memory devices, but struggles to balance the storage density with computing flexibility across multiple tasks. Meanwhile, the conventional pipeline parallel dataflow for high throughput suffers from the underutilization with bubbles caused by variable input sequence length and model sparsity. In this paper, HEIRS-PS, a hybrid three-dimension RRAM- and SRAM-CIM architecture, is proposed for the pipelined multi-task transformer acceleration with sparsity. With the 3D integrated RRAM-CIM and SRAM-CIM arrays, a novel hybrid- CIM paradigm is proposed with the collaborative addition to achieve both high storage density and computing flexibility for multi-task inference. Moreover, by fully leveraging the hybrid-CIM structure in HEIRS-PS, a new dynamic workload scheduling method is proposed with a layer regrouping method to balance the computing latency across pipeline stages. By eliminating the bubbles, the model inference latency is reduced with higher throughput. Experiment shows that the proposed HEIRS-PS architecture improves the energy efficiency by up to 13.3× and 8.68×, compared with state-of-the-art (SOTA) RRAM-CIM and SRAM-CIM works on Llama-1.3B. Moreover, HEIRS-PS also reduces the computing latency by up to 29.5% with the balanced latency across pipeline stages.

Accurate IR drop analysis in 3D integrated circuits (3D ICs) is critical for ensuring power integrity and physical design convergence. However, conventional IR drop simulation flows suffer from extensive runtime and complicated flows, making them difficult for early-stage design evaluation and large-scale design iterations. To address this challenge, we propose DIONet3D, a novel dual-input dual-output neural network tailored for IR drop prediction in 3D ICs. DIONet3D takes feature maps from both top and bottom dies as input, incorporates a shared backbone with U-Net structure, and outputs IR drop maps for both dies simultaneously. To enhance the modeling of 3D power-delivery effects, we introduce a physics-guided 3D-Enhanced PDN representation, in which planar PDN stripes are converted into spatially continuous features through Gaussian kernel-based smoothing, while vertical HB/TSV connections are encoded to capture cross-die power delivery. This hybrid representation enables the network to capture both inter die and intra die features effectively while balancing the computation efficiency and feature representation efficiency. Furthermore, a compound loss function is proposed, combining multiple loss functions with dynamic weighting to balance multi-task convergence and improve structural consistency. Experimental results demonstrate that our proposed method outperforms the state-of-the-art convolutional neural network (CNN) methods and achieves good generalization performance. Compared to commercial EDA simulation tools, DIONet3D achieves high structural and spatial similarity, with up to 309× runtime speedup and 89× memory reduction under CPU inference. Our results demonstrate that DIONet3D offers an accurate and lightweight surrogate solution for 3D IC IR drop evaluation.

DRAM-based Processing-in-Memory (PIM) addresses the “memory wall” by executing computations directly inside main memory. However, memory interleaving and virtual memory limit contiguous data size visible to PIM units, constraining PIM task granularity. Fine-grained PIM tasks incur significant offloading overhead that negates PIM performance benefits. To mitigate this, existing PIM systems drastically isolate PIM memory or disable memory interleaving. These design choices, however, decrease the CPU memory bandwidth and introduce extra data transfer, leading to an additional “system memory wall” that degrades CPU performance and must be resolved to realize PIM’s full potential. In this work, we propose GUMPIM, a PIM system that allows interleaved CPU pages and non-interleaved PIM pages to coexist in a U nitary and M alleable memory space with G uaranteed PIM page allocation. GUMPIM enables zero-copy during PIM task offloading and maintains CPU memory bandwidth while ensuring low PIM offloading overhead. First, we propose a dual-track memory management mechanism consisting of independent page allocation and address translation for CPU and PIM pages. Second, we design GUMPIM interface hardware on PIM-enabled DRAMs to provide a dynamic address mapping for the different data layouts of CPU and PIM pages. Third, we propose a PIM-assisted page migration mechanism that transparently migrates pages while preserving CPU access bandwidth, thereby enabling guaranteed and accelerated PIM page allocation. GUMPIM requires no changes to commodity DRAM standards; all hardware modifications are limited to the DRAM side, ensuring full compatibility with existing CPUs and enabling immediate deployment on current HBMx- and LPDDRx-based PIM platforms. Our results show only <0.1% performance degradation for CPU workloads on GUMPIM, in contrast to the 25.8% degradation on PIM systems with memory interleaving turned off. For PIM workloads, GUMPIM reduces memory allocation and CPU-part computation times by 2.7× and 4.93×, respectively, yielding an end-to-end 2.3× speedup over a state-of-the-art baseline system.

Near-memory processing (NMP) is a promising way to overcome the memory wall in large language models (LLMs). However, dataflow optimization in NMP is fundamentally constrained, as existing analyses cannot efficiently handle the new distributed vault/channel organization. We propose the Remote-Access-Free (RAF) dataflow, which uses tensor rotation to abstract this distributed organization and eliminate all intra-operator remote access. On top of RAF, we apply analytical optimization to minimize intra-operator local access, thereby achieving the intra-operator communication lower bound. We then introduce data partitioning that removes inter-operator remote access and enable operator fusion to minimize inter-operator local access, so that the inter-operator communication lower bound is also reached. Experimental results show that RAF reduces energy by 50.4%, 39.0%, and 37.8%, and delivers speedups of 3.98×, 1.72×, and 1.57× over IANUS, H²LLM, and OptiPIM, respectively.

On accelerators, data reuse is critical, prompting optimal tensor operator dataflow analyses. However, conventional tensor dataflow analyses mainly focus on square-shaped tensors, whereas in attention-based models, especially in long sequence tasks, tensors often exhibit narrow-and-long shapes, necessitating analyses that differ from conventional approaches. In this study, we first identify the guiding principles inherent in conventional dataflow optimizations. Then we propose dimensional collapse tiling (DCT), a novel tensor tiling scheme to exploit data reuse in narrow-and-long tensors under limited buffer sizes. We introduce the DCT method and explain why DCT is superior in dataflow optimization by using attainable data movement analysis. DCT is well-suited for the latest fused tensor operator dataflows and can establish a novel inter-operator dataflow that enables efficient fusion of multiple operators connected at any length. Finally, we broaden the scope of dataflow exploration by incorporating DCT and develop an exploration framework that supports operator fusion, aiming to identify the optimal dataflow on spatial accelerators with a given on-chip buffer size. Experimental results show that DCT delivers 4.69×, 1.99×, and 2.12× speedup and 55.3%, 24.7%, and 16.3% energy savings on average for tiny, edge, and cloud accelerators, respectively, compared with the state-of-the-art dataflow FLAT.

The increasing adoption of artificial intelligence has spurred the development of specialized deep neural network (DNN) accelerators. Among them, computing-in-memory (CIM) architectures are promising for their in-situ computation capability, which alleviates the computation and data movement bottlenecks of modern DNNs. However, the diversity of models and hardware designs makes it challenging to fully exploit CIM accelerators. Existing approaches often rely on manual mapping or provide limited automation, struggling to integrate general-purpose optimizations with CIM-specific features. In this work, we present HARMONY, a hardware-aware compilation framework for CIM accelerators. At its core is a hardware intermediate representation (IR) that unifies computational and memory abstractions. Based on this IR, HARMONY introduces an automatic mapping algorithm that identifies offloadable operators and constructs a hybrid software–hardware IR. This enables systematic integration of general-purpose and CIM-specific scheduling primitives within a unified search space, which is efficiently explored using reinforcement learning (RL). Extensive evaluations show that HARMONY supports a broader set of operators than existing CIM compilers and consistently delivers substantial performance and energy improvements across diverse DNN workloads. These results demonstrate that HARMONY provides both generality and efficiency, making it a practical compilation solution for CIM accelerators.

While 3D-stacked static random-access memory (SRAM) architectures have demonstrated prominent performance speedup for tensor computing by exploiting higher bandwidth, larger buffer, and reduced latency, they suffer from thermal challenges owing to the vertical stacking nature of chips. In this article, we identify that tensor dataflow may further exacerbate the thermal issues, so we propose T3D, the first thermal-aware tensor framework for 3D-stacked SRAM architectures, leveraging tensor dataflow characteristics to significantly enhance thermal performance. Specifically, we first perform a quantitative formulation to identify the most energy-efficient tensor dataflow with given 3-D constraints, effectively reducing heat generation without performance loss. Then, we develop a thermal-aware 3-D architectural floor plan to improve heat spreading by optimizing the spatial arrangement of multiple SRAM macros with varying power overheads, which is caused by mismatched data access rates of tensor computing. Experimental results show that our proposed T3D can reduce the peak chip temperature by $12.9~^{\circ }$ C on certain LLM and DNN workloads over the state-of-the-art 3-D solutions.

With the increasing complexity of chiplet-based architectures and the growing adoption of heterogeneous computing, efficient and high-precision simulation frameworks are essential for evaluating chiplet interconnect performance. This paper presents an implementation of the UCIe protocol (Universal Chiplet Interconnect Express) within the gem5 simulation environment, providing a comprehensive system-level modeling framework for UCIe-based interconnects. The proposed UCIe link model supports full-stack protocol simulation, including transaction-level processing, die-to-die adaptation, and physical layer interactions. A flit-based packing mechanism is introduced to enable accurate transmission modeling, and an Ack/Nak-based retry mechanism is introduced to ensure robust data integrity. An optimized event-driven scheduling strategy is incorporated to address performance bottlenecks observed in traditional UCIe simulations. Experimental evaluations validate the accuracy and efficiency of the proposed UCIe link model by comparing its latency with theoretical values under different computational workloads. The results show that the model closely matches the theoretical predictions, with a latency deviation of less than 0.5% in most cases. Additionally, performance comparisons between UCIe interconnect, PCIe interconnect, and direct memory transfer reveal that UCIe incurs only a minor protocol overhead (less than 0.7%), making it a practical and scalable solution for multi-chipset interconnects. The proposed UCIe simulation framework provides a high-precision virtual verification platform for chiplet system design, enabling low-cost, high-accuracy performance evaluations, and lays the foundation for future optimizations in large-scale chiplet-based architectures.