Wei Li

The speed and quality of global placement (GP) are crucial for FPGA performance. The existing nonlinear placers provide higher computational efficiency than heuristic algorithms, but their insufficient attention to the initialization of GP distributions limits rapid convergence towards high-quality global optimal solutions. To resolve this, we propose a GP framework, DrlGoFPGA 2.0, combining LUT, FF, DSP, and RAM (LFDR) instance placement generation based on multi-agent graph transformer network (MAGTN) and LFDR initial placement fine-tuning based on nonlinear placement. A MAGTN model for LFDR initial placement is designed, which utilizes distributed training to circumvent the issues of sub-optimal placement models and prolonged training times due to the non-uniform distribution of large-scale LFDR sites. A lightweight LFDR line-network relationships (LNR) graph supporting MAGTN co-decision is created by separating, renumbering, and deduplicating the LNR graph containing all instances, and inputting the LFDR LNR graph into MAGTN to obtain the connections between LFDRs. The experimental results on ISPD’2016/2017 benchmarks show that compared with the state-of-the-art (SOTA) nonlinear placer, DrlGoFPGA 2.0 can improve GP speed by 27.3% ∼ 82.1%, reduce half-perimeter wirelength by 0.6% ∼ 4.6%, and routed wirelength by 0.4% ∼ 2.8%.

End-to-end latency alone cannot guide migration to a new accelerator: a slowdown reveals neither which operators account for the gap nor which optimizations will transfer across hardware. We present HET, a white-box benchmark and attribution framework for inference migration. HET combines (i) counterfactual operator performance attribution (COPA), which estimates each operator's contribution to a migration gap with a surrogate-assisted antithetic Shapley estimator, and (ii) a model-operator-hardware knowledge graph (MOH-KG), which represents structural and performance relations and drives a KG-guided PPO optimizer (KG-A2O). The benchmark contains 34 models and 6,244 profiled operator instances spanning five workload categories and five computing platforms. The graph predictor achieves 14.3% mean relative error (MRE), compared with 25.6% for Roofline and 32.0% for DeepBench. COPA reaches 0.98% attribution MRE with 500 samples, while its surrogate reduces query latency by 288-8,252×. In the evaluated A100-to-Ascend 910B migration case, the complete optimization plan yields a 15.6% speedup and a 66% cost reduction while meeting the taskquality threshold.

Efficient coordination of heterogeneous AI chips (GPU/NPU/DCU) in data centers is crucial for Large Language Model (LLM) training, but this process is hindered by architectural mismatches, protocol fragmentation, and network partitioning. Existing solutions fail to achieve unified resource management across different chips, resulting in severe resource fragmentation and reduced allreduce efficiency. To overcome these limitations, this paper proposes the unified coordination framework UniOrch, which not only integrates three core functionalities, including hardware abstraction, software standardization, and communication coordination, but also enables the training and inference of large models across heterogeneous AI chips. UniOrch's hardware-agnostic bare-metal cloud eliminates virtualization overhead through Border Gateway Protocol Ethernet Virtual Private Network (BGP EVPN) overlay networks and gateway-based chip integration; its PyTorch-based adaptation layer masks hardware differences and reduces migration costs; the Transformer Collective Communication Library (TCCL) unifies NCCL, HCCL, and OpenMPI protocols to support seamless hybrid parallel training. Furthermore, the framework's core scheduling mechanism is the Heterogeneous Hybrid Estimation Model (HHEM), which employs a two-stage cost model combining static analysis with dynamic runtime feedback to dynamically allocate computing power based on Transformer task loads, ensuring cross-chip task synchronization, resource pooling, and dynamic allocation. Deployment verification in real-world production environments (e.g., China Construction Bank) shows that UniOrch achieves significant improvements: resource utilization of heterogeneous AI infrastructure is increased by 35%, cross-chip latency is reduced by 42%, accuracy loss in heterogeneous environments is <1.5%, the GLM-130B model achieves near-linear scalability across heterogeneous AI chips (with 116.7 tokens processed per card per second). Notably, it realizes the first cross-vendor LLM training on diversified hybrid NPU-GPU clusters with convergence error <0.8%.

Optimizing deep learning inference, particularly reducing the execution latency of tensor computations at small batch sizes, is crucial for the successful and widespread adoption of deep neural network (DNN) models. However, current deep learning compilers and hand-tuned libraries often fail to achieve high hardware efficiency when executing small-batch workloads. The primary reason is the inherently sequential nature of reductions (e.g., along the hidden dimension in the flattened GEMM for LLM decoding), which is difficult to parallelize and therefore fails to fully utilize available hardware resources. In this article, we propose Swift, a novel search-based approach for efficiently generating high-performance programs for GPUs by maximizing hardware utilization. The key insight is that reduction parallelization can be incorporated into a unified representation alongside the existing tile structure, significantly expanding the search space for high-performance programs. Concretely, by enumerating all possible parallel mappings of loops, we first generate a large search space that contains high-performance programs. Then, to efficiently explore the extended search space, we employ subspace shifting exploration to identify promising regions, effectively prune large portions of the less-promising search space. We conduct experiments on three distinct GPU architectures using a diverse set of benchmarks representative of typical application scenarios. Experimental results demonstrate that Swift achieves an average speedup of 1.19× over the state-of-the-art compiler-based approaches. Moreover, compared with vendor-provided hand-tuned libraries, Swift achieves an average speedup of 2.40×.

Due to the end of Moore’s Law and Dennard Scaling, Domain-Specific Accelerators (DSAs) have come to a Cambrian explosion. Especially when advancing into the intelligent era, more and more DSAs are integrated into System-on-Chips (SoCs) as intellectual property (IP) blocks to provide high performance and efficiency. Currently, IPs usually expose IP-dependent hardware interfaces, requiring SoCs to manage them as isolated devices with software running on the host CPU. However, such software-managed heterogeneity in CPU-centric SoCs leads to low IP utilization. This inefficiency arises from the dependence on software optimization, coupled with the control and data exchange overheads. To improve IP utilization of heterogeneous SoCs, in this article, we rearchitect the SoC as a processor (i.e., SaaP) to orchestrate hardware heterogeneity. SaaP features an orchestration pipeline where DSAs are integrated as execution units and managed directly by the hardware pipeline to conceal the hardware heterogeneity from software. Moreover, SaaP redesigns the register file and data paths to implement an IP-level data-forwarding mechanism, avoiding the costly control and data exchange in the CPU-centric execution model. Block data dependence among different DSAs is carefully resolved to exploit mixed-level parallelism and inter-IP data exchange. SaaP abstracts tasks as mixed-scale instructions, where each instruction can be mapped to different IPs. Experimental results show that compared against Xavier on six fully software-optimized benchmarks from different domains, SaaP-rearchitected Xavier achieves a $2.08{\times }$ speedup, with an 8.21% area reduction and only 2.98% increase in power consumption.