JZJie Zhao

Jie Zhao

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Jie Zhao has a total of 77 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Memory-Safe Hierarchical Planning for Hybrid-Parallel Transformer Training

    by Zuoyan Zhang, Zhiqiang Zhang, Nanqin Wang, Jinyi Wang, Jie Zhao

    Training decoder-style Transformer models at scale requires composing data, tensor, pipeline, context, and, for sparse models, expert parallelism under tight device memory limits. Without a reliable pre-profiling admission model, algebraically legal but memory-infeasible configurations can consume profiling budget without yielding useful throughput measurements. Choosing a pipeline degree also leaves a structured layer-to-stage mapping problem unresolved. This article presents Shockwave, a hierarchical planner that operates through the exposed launcher and partition interfaces of Megatron-style dense and sparse Transformer runtimes. Shockwave first builds a conservative memory-feasible region with an analytical model and backend-calibrated safety margins. It then profiles a balanced baseline and a few single-dimension anchors to learn which parallel degrees improve throughput most on the target runtime and uses that ranking to select a global strategy. Finally, it solves a mixed-integer program for stage-aware layer allocation under a fixed 1F1B pipeline schedule. Implemented as a thin control layer on top of Megatron-LM and MindSpeed, Shockwave matches or exceeds expert-curated plans, achieving geometric mean throughput gains of 1.08× over expert-tuned Megatron hybrid configurations on GPUs and 1.04× over expert MindSpeed recipes on NPUs. Relative to the evaluated automatic baselines, it improves throughput by 1.33× on GPUs and 1.10× on NPUs while reducing configuration search time by 4.59× and 5.60×.

  • Open Access

    Global Scheduling of Transient Parameter Materialization for Sharded Training

    by Zuoyan Zhang, Jinyi Wang, Zhuo Tang, Jie Zhao

    Parameter-sharded training reduces device-memory use by storing model parameters as shards and reconstructing full parameters only before the computations that consume them. This materialization is implemented through frequent all-gathers, which can become exposed waits in each forward–backward training iteration. Hiding these waits is not only a communication-volume problem. It requires deciding where to launch each all-gather and how long the materialized full-parameter buffer should remain live. All-gather prefetching can overlap communication with earlier computation but extends buffer lifetime. Retaining a buffer after its forward-pass use can eliminate a re-gather before backward-pass reuse, but consumes transient memory headroom needed by other materialization decisions. We present Chorus , a compiler-runtime framework for transient parameter materialization in memory-constrained sharded training. Chorus addresses a limitation of local runtime policies that make decisions at hooks, module boundaries, or configuration thresholds. They can issue individual prefetches or suppress individual releases, but they do not jointly compare future all-gather opportunities, buffer lifetimes, and memory-headroom conflicts across the iteration. Chorus lowers the forward and backward passes into a whole-iteration graph that makes all-gathers, use-site waits, and releases explicit. It then builds a profile-guided block model and solves a mixed-integer scheduling problem that jointly chooses all-gather launch positions and iteration-local retention decisions under time-varying memory headroom. The selected schedule is realized by graph rewriting, which moves communication while preserving use-site waits, fuses compatible prefetch operations, and removes redundant releases and re-gathers. Cross-iteration persistent retention is finalized during warmup using measured runtime memory behavior. We implement Chorus on top of torch.compile and evaluate it on 7B to 13B dense Transformer models and one MoE model across 4 to 16 A100 GPUs. In the ZeRO-style stack, Chorus improves throughput by 1.10× over DeepCompile across the primary workloads. In the FSDP-style stack, the Chorus implementation on SimpleFSDP improves throughput by 1.14× over SimpleFSDP across the corresponding dense-model workloads. The benefits are greatest in network-constrained settings, where all-gather stalls are more exposed. Chorus stays within the configured memory budget and achieves a 1.35× compilation-time speedup over DeepCompile.

  • Open Access

    LLM-Based Automatic Architecture Optimization for AI Model HLS Implementation on FPGA

    by Zhe Xiao, Mingyu Liu, Xu He, Haoying Wu, Jie Zhao, Yao Wang, Libo Huang

  • Open Access

    A Decoupled Analytical Model for Tile Size Selection in Affine Programs

    by Shihan Yuan, Zuoyan Zhang, Guanghui Song, Junhui Peng, Feng Wang, Zhuo Tang, Kenli Li, Jie Zhao

    Existing tile size selection approaches are tightly coupled with compiler transformation pipelines, often leading to inaccurate modeling of cache behavior and limited effectiveness for non-rectangular tile shapes. This article presents TileMind , a decoupled analytical model that combines compile-time and runtime information for tile size selection in affine programs. It introduces a transformation-aware pre-tiling step that enables the decoupled selector to remain consistent with compiler transformations while extracting compile-time metadata. The extracted metadata is then combined with profiled runtime characteristics to construct a richer yet tractable feasible domain, within which a nonlinear objective for tile size selection is formulated. This objective is subsequently transformed into a binary product linearization problem, with its nonlinear constraints also linearized for efficient optimization. Finally, an intra-tile optimization aligns computation with data layout to enhance data reuse within tiles. Across two multi-core Intel CPUs, TileMind achieves 1.49× (sequential) and 1.33× (parallel) mean speedups on twenty PolyBench kernels, and 2.08–3.54× speedups on three deep learning workloads over the state-of-the-art analytical model Pluto-tss . Compared with TVM’s latest autotuner MetaSchedule, TileMind delivers 1.35–1.46× mean speedups while reducing tuning overhead by 2–4 orders of magnitude. While demonstrating effectiveness on selecting tile sizes for non-rectangular tile shapes and compatibility with PPCG, Pluto, and TVM, we further provide proof-of-concept results on GPUs, illustrating the potential portability of TileMind across architectures.

  • A combined language and polyhedral approach to heterogeneous parallelism

    by Jie Zhao

    Une approche combinée langage-polyédrique pour la programmation parallèle hétérogène: De nos jours, l'optimisation des compilateurs est de plus en plus mise à l'épreuve par la diversité des langages de programmation et l'hétérogénéité des architectures. Le modèle polyédrique est un puissant cadre mathématique permettant aux programmes d’exploiter la parallélisation automatique et l’optimisation de la localité, jouant un rôle important dans le domaine de l’optimisation des compilateurs. Une limite de longue date du modèle réside dans sa restriction aux programmes affines à contrôle statique, ce qui a entraîné une demande émergente de prise en charge d'extensions non affines. Cela est particulièrement aigu dans le contexte d'architectures hétérogènes où une variété de noyaux de calcul doivent être analysés et transformés pour répondre aux contraintes des accélérateurs matériels et pour gérer les transferts de données à travers des espaces mémoire. Nous explorons plusieurs extensions non affines du modèle polyhédral, dans le contexte d'un langage intermédiaire bien défini combinant des éléments affines et syntaxiques. D'un côté, nous expliquons comment les transformations et la génération de code pour des boucles avec des limites de boucle dynamiques non dépendantes des données et dynamiques sont intégrées dans un cadre polyédrique, élargissant ainsi le domaine applicable de la compilation polyédrique dans le domaine des applications non affines. D'autre part, nous décrivons l'intégration du pavage en recouvrement pour les calculs de pochoir dans un cadre polyhédral général, en automatisant les transformations non affines dans la compilation polyhédrique. Nous évaluons nos techniques sur des architectures de CPU et de GPU, en validant l'efficacité des optimisations en effectuant une comparaison approfondie des performances avec des frameworks et des librairies écrites à la pointe de la technologie.

  • Dual-Pronged Deep Learning Preprocessing on Heterogeneous Platforms With CPU, Accelerator and CSD

    by Jia Wei, Xingjun Zhang, Witold Pedrycz, Longxiang Wang, Jie Zhao

    For image-related deep learning tasks, the first step often involves reading data from external storage and performing preprocessing on the CPU. As accelerator speed increases and the number of single compute node accelerators increases, the computing and data transfer capabilities gap between accelerators and CPUs gradually increases. Data reading and preprocessing become progressively the bottleneck of these tasks. Our work, DDLP, addresses the data computing and transfer bottleneck of deep learning preprocessing using Computable Storage Devices (CSDs). DDLP allows the CPU and CSD to efficiently parallelize preprocessing from both ends of the datasets, respectively. To this end, we propose two adaptive dynamic selection strategies to make DDLP control the accelerator to automatically read data from different sources. The two strategies trade-off between consistency and efficiency. DDLP achieves sufficient computational overlap between CSD data preprocessing and CPU preprocessing, accelerator computation, and accelerator data reading. In addition, DDLP leverages direct storage technology to enable efficient SSD-to-accelerator data transfer. In addition, DDLP reduces the use of expensive CPU and DRAM resources with more energy-efficient CSDs, alleviating preprocessing bottlenecks while significantly reducing power consumption. Extensive experimental results show that DDLP can improve learning speed by up to 23.5% on ImageNet Dataset while reducing energy consumption by 19.7% and CPU and DRAM usage by 37.6%. DDLP also improves the learning speed by up to 27.6% on the Cifar-10 dataset.


  • Open Access

    ASSG: Enhanced Workload Balancing via Adaptive State Scheduling Granularity Approach for Stateful Distributed Stream Processing

    by Jiawei Tan, Jiapeng Zhang, Zhuo Tang, Xiong Xiao, Bingting Jiang, Jie Zhao, Kenli Li

    The dynamic changes in the distribution of data attributes within the data stream lead to unbalanced workloads in stream processing operators, often necessitating state migration in these operators. However, significant time overhead associated with state migration results in latency peaks and interruptions in stream applications. Ignoring the impact of state scheduling granularity exacerbates the time overhead during migration. Focusing on state migration efficiency, this article presents the Adaptive State Scheduling Granularity (ASSG) approach for stateful distributed stream processing. The ASSG conducts dynamic workload balancing at the granularity of state subgroups, where the number and size of any subgroup can be adaptively adjusted to reduce the time of collecting workloads and generating balancing plans. Moreover, since the idle instances may experience higher workloads than average due to receiving the migrated states, we propose a parallel migration strategy. They are enabled to post low states to other instances while receiving the high ones, alleviating the workload imbalance without extending migration duration. ASSG has been fully implemented on Apache Flink, which demonstrates a 23.84% reduction in average latency compared to state-of-the-art workload balancing schemes and achieves at least a 10.91% reduction in the time required for state migration through adaptive granularity adjustment.


  • Open Access

    Accelerating Parallel Structures in DNNs via Parallel Fusion and Operator Co-Optimization

    by Zhanyuan Di, Leping Wang, Zhaojia Ma, En Shao, Jie Zhao, Ziyi Ren, Siyuan Feng, Dingwen Tao, Guangming Tan, Ninghui Sun

    Parallel structures have become a key pattern in deep neural networks (DNNs), offering improved efficiency and scalability. However, existing machine learning compilers (MLCs) face challenges in optimizing these structures due to limited parallel fusion scope and insufficient analysis of intra-operator characteristics. This paper introduces Magneto, a framework designed to accelerate DNN inference by co-optimizing parallel operators. Magneto broadens the fusion scope and incorporates a specialized co-tuning algorithm to optimize operators jointly. Our approach addresses the unique challenges inherent in optimizing parallel structures, enabling significant performance improvements across various hardware platforms. Experimental results show that Magneto outperforms state-of-the-art NVIDIA TensorRT and AMD MIGraphX, achieving geometric mean speedups of 2.27 × and 2.88 ×, respectively.

  • Open Access

    Optimizing Deep Learning Inference Efficiency through Block Dependency Analysis

    by Zhanyuan Di, Leping Wang, En Shao, Zhaojia Ma, Ziyi Ren, Feng Hua, Lixian Ma, Jie Zhao, Guangming Tan, Ninghui Sun


  • Open Access

    Magneto: Accelerating Parallel Structures in DNNs via Co-Optimization of Operators

    by Zhanyuan Di, Leping Wang, Ziyi Ren, En Shao, Jie Zhao, Siyuan Feng, Dingwen Tao, Guangming Tan, Ninghui Sun

    Deep neural networks (DNNs) increasingly rely on parallel structures to enhance performance and efficiency. However, existing machine learning compilers (MLCs) face challenges in optimizing these structures due to limited parallel fusion scopes and insufficient consideration of intra-operator information. This paper introduces Magneto, a novel framework designed to accelerate parallel structures in DNNs through the co-optimization of parallel operators. By expanding the scope of parallel operator fusion and introducing a dedicated co-tuning algorithm, Magneto unlocks new opportunities for co-optimization. Experimental results demonstrate that Magneto outperforms NVIDIA TensorRT and AMD MIGraphX, achieving speedups of 3.02× and 4.19×, respectively.