KLKai Lu

Kai Lu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Kai Lu has a total of 263 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    RELL-STC: An Efficient SpMM Extension of Sparse Tensor Core with Re-structured ELLPACK and Gustavson Dataflow

    by Jianfeng Cui, Bo Yuan, Zekun Jiang, Kai Lu, Sheng Liu

    Sparsity is crucial for deep neural networks, reducing storage and bandwidth while improving computational efficiency. With the rapid development of deep neural networks, sparse computation accelerators have become a key research focus. NVIDIA introduced 2:4 fine-grained structured sparsity in the Ampere architecture, doubling Tensor Core peak throughput by retaining two out of four elements, but this may compromise model accuracy. In contrast, unstructured sparsity achieves maximum sparsity without accuracy loss. However, the irregular distribution of non-zero elements (NNZs) in unstructured sparse matrices causes complex memory access and load imbalance, preventing direct use of GPU Tensor Cores. Thus, computation relies on CUDA cores via sparse libraries like cuSPARSE, underutilizing Tensor Cores' capabilities. To address this, we extend the Tensor Core architecture based on Gustavson's dataflow and propose a re-structured row offline compression format for inference. This format integrates with the extended architecture to enable efficient unstructured sparse-dense matrix multiplication. Experiments show negligible area and power overheads, with an average speedup of 3.54x over cuSPARSE and 1.2x over state-of-the-art Tensor Core extensions.

  • Open Access

    RELL-STC: An Efficient SpMM Extension of Sparse Tensor Core with Re-structured ELLPACK and Gustavson Dataflow

    by Jianfeng Cui, Bo Yuan, Zekun Jiang, Sheng Liu, Kai Lu

    Sparsity is a crucial optimization technique for deep neural networks, significantly reducing storage and bandwidth requirements while enhancing computational efficiency. In recent years, driven by the rapid advancement of deep neural networks, accelerators targeting sparse computation have emerged as a significant area of research. NVIDIA introduced 2:4 fine-grained structured sparsity starting with the Ampere architecture, which doubles the peak throughput of Tensor Cores by retaining only two out of every four elements. However, the constraints imposed by structured sparsity on parameters may compromise model accuracy. In contrast, unstructured sparsity enables maximum sparsity while maintaining model accuracy. Nevertheless, the irregular distribution of non-zero elements (NNZs) in unstructured sparse matrices introduces complex memory access patterns and load imbalances, preventing the direct utilization of GPU Tensor Cores. Consequently, computation typically relies on CUDA cores via sparse libraries such as cuSPARSE, leaving the powerful computational capabilities of Tensor Cores underutilized. To address this limitation, we extend the Tensor Core architecture inspired by Gustavson's dataflow and propose a re-structured row offline compression format tailored for the inference stage. This format integrates seamlessly with the extended architecture to achieve highly efficient unstructured sparse-dense matrix multiplication. Experimental results demonstrate that with negligible area and power overheads, our method achieves an average speedup of 3.54x compared to cuSPARSE and 1.2x compared to state-of-the-art Tensor Core extensions.

  • Open Access

    Not All Paths Are Equal: Multi-path Optimization for Directed Hybrid Fuzzing

    by Peihong Lin, Pengfei Wang, Xu Zhou, Wei Xie, Gen Zhang, Kai Lu

    Directed Grey-Box Fuzzing (DGF) can improve bug exposure efficiency by stressing bug-prone areas. Recent studies have modeled DGF as the problem of finding and optimizing paths to reach target sites. However, they still face the “ multi-path ” challenge. When a target site is reachable by multiple paths, it is crucial to comprehensively evaluate and effectively select these paths, as this affects the fuzzer’s choice between reaching target sites via optimal paths and enhancing path diversity toward targets to expose hidden bugs in non-optimal paths. In this article, we propose MultiGo, a directed hybrid fuzzer designed for multi-path optimization. First, we propose a new fitness metric called path difficulty to comprehensively evaluate the promising paths. This metric uses the Poisson distribution to estimate the probability of exploring basic blocks along execution paths based on statistical block frequency, distinguishing between optimal and challenging paths. With path difficulty as a key factor, a customized Contextual Multi-Armed Bandit (CMAB) model is employed to efficiently optimize path scheduling by comprehensively considering the impact of testing conditions on path scheduling. We introduce the concept of the fuzzing context to represent and evaluate testing conditions, which encompass factors such as path characteristics (e.g., path difficulty), the testing agent (e.g., fuzzing or symbolic execution), and the testing goal (e.g., path exploitation or exploration). Then, the CMAB model predicts the expected rewards for scheduling paths under different testing agents and goals, thereby optimizing path scheduling. By leveraging the CMAB model, MultiGo enhances DGF’s capability to explore easier paths and symbolic execution’s capacity to handle more complex ones, enabling efficient target reaching through optimal paths while ensuring sufficient coverage of non-optimal paths. MultiGo is evaluated on 136 target sites of 41 real-world programs from 3 benchmarks. The experimental results show that MultiGo outperforms the state-of-the-art directed fuzzers (AFLGo, SelectFuzz, Beacon, WindRanger, and DAFL) and hybrid fuzzers (SymCC and SymGo) in reaching target sites and exposing known vulnerabilities. Moreover, MultiGo also discovered 14 undisclosed vulnerabilities.


  • mtGEMM: An Efficient GEMM Library for Modern Multi-Core DSPs

    by Jianbin Fang, Kainan Yu, Peng Zhang, Dezun Dong, Xinxin Qi, Xingyu Hou, Ruibo Wang, Kai Lu

    The General Matrix Multiplication (GEMM) is a crucial subprogram in high-performance computing (HPC). With the increasing importance of power and energy consumption, modern Digital Signal Processors (DSPs) are being integrated into general-purpose HPC systems. However, due to architecture disparities, traditional optimizations for CPUs and GPUs are not easily applicable to modern DSPs. This paper shares our experience of optimizing the GEMM operation using a CPU-DSP platform as a case study. Our work employs a set of strategies to improve the performance and scalability of GEMM. These strategies focus on developing micro-kernels based on heterogeneous on-chip memory, addressing the memory access bottleneck in multi-core parallelism, and facilitating efficient transpose-GEMM. These approaches, collectively referred to as an efficient and practical library (a.k.a. mtGEMM), maximize computational capabilities and bandwidth utilization of multi-core DSPs, while achieving high performance for variously-shaped GEMMs. Our experimental results demonstrate that mtGEMM can attain between 92% and 96% of the hardware peak, with the multi-core scalability being almost linear.


  • Open Access

    PDGNN: Efficient Micro-batch GNN Training via Degree-Pruned Partitioning and Redundancy Elimination

    by Lizhi Zhang, Menghan Jia, Ping Gong, Zhiquan Lai, Dongsheng Li, Yongquan Fu, Ao Shen, Kai Lu

    Graph Neural Networks (GNNs) have become essential for graph-based applications. Existing GNN training frameworks primarily adopt sampling-based mini-batch approach, which samples vertex neighborhoods to construct subgraphs for training iteratively, but faces a memory-accuracy tradeoff. To address this, naive micro-batch training with gradients accumulation can be applied, yet introduces severe redundancy from repeated neighbor sampling. Recent micro-batch training methods overcome this by adopting batch-level graph partitioning strategies that partition full-batch subgraph sampled from all target vertices to generate micro-batches. Nevertheless, these approaches struggle to balance partitioning overhead with redundancy. This article proposes PDGNN, a system designed for efficient micro-batch GNN training on single machine, reducing both training time and memory consumption. First, we propose a degree-pruning-based graph partitioning method that efficiently processes on simplified and sparsified multi-level bipartite graphs by removing high-degree vertices, significantly reducing partitioning overhead. Second, we propose a dependency-aware redundancy matching and elimination technique that maximizes inter-micro-batch redundancy matching, and eliminates redundant computation by embedding caching and reuse while maintaining model accuracy. Experiments on real-world datasets demonstrate that PDGNN substantially improves the efficiency of large-scale GNN training, achieving up to 3.5× speedup in epoch time, 46% lower peak GPU memory consumption, and 64% reduced peak CPU memory consumption compared to state-of-the-art batch-level training methods.


  • Fully Decentralized Data Distribution for Large-Scale HPC Systems

    by Ruibo Wang, Mingtian Shao, Wenzhe Zhang, Huijun Wu, Jiaxin Li, Lihua Yang, Di Ma, Yiqin Dai, Kai Lu

    For many years, in the HPC data distribution scenario, as the scale of the HPC system continues to increase, manufacturers have to increase the number of data providers to improve the IO parallelism to match the data demanders. In large-scale, especially exascale HPC systems, this mode of decoupling the demander and provider presents significant scalability limitations and incurs substantial costs. In our view, only a distribution model in which the demander also acts as the provider can fundamentally cope with changes in scale and have the best scalability, which is called all-to-all data distribution mode in this paper. We design and implement the BitTorrent protocol on computing networks in HPC systems and propose FD3, a fully decentralized data distribution method. We design the Requested-to-Validated Table (RVT) and the Highest ranking and Longest consecutive piece segment First (HLF) policy based on the features of the HPC networking environment to improve the performance of FD3. In addition, we design a torrent-tree to accelerate the distribution of seed file data and the aggregation of distribution state, and release the tracker load with neighborhood local-generation algorithm. Experimental results show that FD3 can scale smoothly to 11k+ computing nodes, and its performance is much better than that of the parallel file system. Compared with the original BitTorrent, the performance is improved by 8-15 times. FD3 highlights the considerable potential of the all-to-all model in HPC data distribution scenarios. Furthermore, the work of this paper can further stimulate the exploration of future distributed parallel file systems and provide a foundation and inspiration for the design of data access patterns for Exscale HPC systems.