Weiguang Sheng

Near-memory processing (NMP) is a promising way to overcome the memory wall in large language models (LLMs). However, dataflow optimization in NMP is fundamentally constrained, as existing analyses cannot efficiently handle the new distributed vault/channel organization. We propose the Remote-Access-Free (RAF) dataflow, which uses tensor rotation to abstract this distributed organization and eliminate all intra-operator remote access. On top of RAF, we apply analytical optimization to minimize intra-operator local access, thereby achieving the intra-operator communication lower bound. We then introduce data partitioning that removes inter-operator remote access and enable operator fusion to minimize inter-operator local access, so that the inter-operator communication lower bound is also reached. Experimental results show that RAF reduces energy by 50.4%, 39.0%, and 37.8%, and delivers speedups of 3.98×, 1.72×, and 1.57× over IANUS, H²LLM, and OptiPIM, respectively.

Multichiplet deep neural network (DNN) accelerators have evolved as promising modular solutions, offering enhanced performance, scalability, and cost-effectiveness. These architectures, however, suffer from the escalating communication bottleneck with increasing scale, primarily stemming from rising hop count and worsening link underutilization. The bottleneck is exacerbated by conventional fixed interconnection networks’ nonadaptability to diverse DNN dataflows. Moreover, an efficient routing tailored for large-scale networks with deadlock-freedom is needed for performance. To address these scalability challenges, leveraging the low-latency links and abundant interconnection resources with reconfigurability in the active interposer, we propose HiRe, a hierarchical reconfigurable network-routing co-design architecture for large-scale multichiplet DNN accelerators. The architecture introduces reconfigurable nodes (RNs) across on-chip and interchiplet hierarchical networks, enabling dynamic bypassing and network reconfiguration. Based on the network, it incorporates an efficient deadlock-free routing that combines simulated annealing (SA)-based communication scheduling with greedy path selection. Through the network-routing co-design, HiRe reduces the hop count and enhances the link utilization. The HiRe architecture is implemented and synthesized in a 55-nm CMOS process. Experimental results demonstrate that HiRe achieves a 14.3%–45.2% EDP reduction and a 14.1%–35.0% latency reduction compared to state-of-the-art (SOTA) innovations, effectively mitigating the large-scale communication bottleneck.

Due to the high local current densities, Through Silicon Vias (TSVs) are susceptible to electromigration (EM) degradation, which reduces the reliability of integrated circuits. Unlike traditional methods for TSV modeling and simulation, this paper introduces a unified hard constraint physics-informed learning neural network approach, called HCPINN, for the transient analysis of electromigration in TSVs by solving the conventional mass balance equation. The proposed method allows simultaneous consideration of atomic depletion and accumulation, effective resistance degradation, electric current evolution, and stress distribution. Specifically, we propose a hard constraint method for solving partial differential equations (PDEs) with general boundary conditions (BCs) for transient TSV electromigration analysis. By using the extra fields derived from the mixed finite element method, we reconstruct the corresponding PDEs by transforming general BCs into linear forms. Based on this derivation, we embed general BCs of mass balance equation into the proposed ansatz and employ sub-networks for the approximation on general BCs. The main neural network is responsible for training the internal part of the problem domain without adding loss terms with BCs, overcoming the convergence issue due to unbalanced gradients among different loss terms. Besides, we theoretically demonstrate that this reformulation of general BCs can stabilize the training process. Experimental results indicate that the proposed HCPINN exhibits superior performance and reduces boundary error in TSV electromigration analysis. Compared to the finite element method, the proposed network achieves approximately more than 70 times faster inference with a minimal mean squared error increase of less than 0.1% under different TSV geometries and varying current density profiles.

The increasing adoption of artificial intelligence has spurred the development of specialized deep neural network (DNN) accelerators. Among them, computing-in-memory (CIM) architectures are promising for their in-situ computation capability, which alleviates the computation and data movement bottlenecks of modern DNNs. However, the diversity of models and hardware designs makes it challenging to fully exploit CIM accelerators. Existing approaches often rely on manual mapping or provide limited automation, struggling to integrate general-purpose optimizations with CIM-specific features. In this work, we present HARMONY, a hardware-aware compilation framework for CIM accelerators. At its core is a hardware intermediate representation (IR) that unifies computational and memory abstractions. Based on this IR, HARMONY introduces an automatic mapping algorithm that identifies offloadable operators and constructs a hybrid software–hardware IR. This enables systematic integration of general-purpose and CIM-specific scheduling primitives within a unified search space, which is efficiently explored using reinforcement learning (RL). Extensive evaluations show that HARMONY supports a broader set of operators than existing CIM compilers and consistently delivers substantial performance and energy improvements across diverse DNN workloads. These results demonstrate that HARMONY provides both generality and efficiency, making it a practical compilation solution for CIM accelerators.

DRAM latency remains a critical bottleneck in the performance of modern computing systems. However, the latency is excessively conservative due to the timing margins imposed by DRAM vendors to accommodate rare worst-case scenarios, such as weak cells and high temperatures. In this study, we introduce a temperature- and process-variation-aware timing detection and adaptation DRAM (TPDA-DRAM) architecture that dynamically mitigates timing margins at runtime. TPDA-DRAM leverages innovative in-situ cross-coupled detectors to monitor voltage differences between bitline pairs inside DRAM arrays, ensuring precise detection of timing margins. Additionally, the proposed detector inherently accelerates the precharge operation of DRAM, thereby reducing the precharge latency by up to 62.5%. Building upon this architecture, we propose two variation-aware timing adaptation schemes: 1) a process-variation-aware adaptation (PVA) scheme that accelerates access to weak cells, mitigating process-induced timing margins, and 2) a temperature-variation-aware adaptation (TVA) scheme that leverages temperature information and the restoration truncation technique to reduce DRAM latency, mitigating temperature-induced timing margins. Evaluations on an eight-core computing system show that TPDA-DRAM improves average performance by 21.8% and energy efficiency by 18.2%.

Large-area chips using advanced processes are facing challenges in yield rate and limitations in lithographic area. Chiplet technology has emerged as a promising solution with improved flexibility, scalability, and yield rate. However, even if each chiplet is internally deadlock-free, the inter-chiplet network may still encounter deadlock issues. Ensuring the modular design and topology agnostic of each chiplet, while achieving better performance under limited inter-chiplet bandwidth, is the main challenge of routing algorithms. This article introduces IPDR, an inter-chiplet priority-driven deadlock resolution mechanism specifically designed for inter-chiplet networks, where the inter-chiplet link often constitutes a critical system bottleneck. IPDR imposes no restrictions on virtual channel (VC) allocation or turn strategies during normal state, avoiding performance penalties from conservative deadlock avoidance methods. Upon detecting a deadlock at boundary routers, IPDR prioritizes the inter-chiplet traffic by allowing the blocked inbound packet to advance and occupy a dedicated IPDR buffer. The IPDR buffer in each boundary router guarantees that at least one bubble exists in the dependency chain during an inter-chiplet deadlock, thereby breaking the deadlock dependency. To ensure correctness and prevent the formation of new deadlocks during recovery, virtual network (VN) separation rules are enforced throughout the recovery process. This design enables efficient deadlock resolution with minimal performance degradation during recovery state. Moreover, a fine-grained packet packing is implemented in IPDR to improve the bandwidth utilization of small packets. Compared with the state-of-the-art related work, IPDR achieves 7.30%–66.62% latency reduction at the cost of 0.77%–6.98% area overhead.

Face-to-face bonded 3-D (F2F 3D) technology, with the potential to significantly reduce chip area while enhancing performance, stands as one of the most promising ways to extend Moore’s Law. However, current 3-D physical design flows are often modifications of 2-D design flows and rely on technical personnel to manually modify technical files. Furthermore, existing research on 3-D design flow primarily focuses on module implementation, with very few studies addressing hierarchical design methods for large-scale chips. In this article, we first introduce a 3-D physical design flow which concurrently optimizes the timing of both the logic tier and the memory tier, achieving synchronized physical design for both tiers. Then, we develop a bottom-up hierarchical 3-D physical design flow to extend the 3-D design flow to large-scale chip design. Through coordinated power planning, clock tree design, and interconnect unit design, we enhance the power, performance, and area (PPA) metrics of the entire chip. Using our RTL-to-GDS physical design flow, we successfully implemented a 28-nm CMOS logic-on-memory (LoM) 3-D coarse-grained reconfigurable architecture (CGRA) chip with over 50 million gates. Experimental results demonstrate that our 3-D flow improves timing by 16.1% while reducing voltage drop by 38.6% compared to the 2-D design. In addition, the power-delay product (PDP) of the 3-D chip decreases by 10.2%, showcasing better performance.