SYSiling Yang

Siling Yang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Siling Yang has a total of 16 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • HyperInfer: Accelerating Large Language Model Inference via Importance-Informed Prefix-KV Caching and Prefetching

    by Ruidong Zhang, Weijian Chen, Ping Chen, Shuibing He, Wenjie Liang, Rongchen Liu, Siling Yang, Xuechen Zhang

  • Open Access

    NVREC: A High-Performance Recoverable Recommendation System for Non-Volatile Memory

    by Ping Chen, Shuibing He, Peiyi Hong, Xuechen Zhang, Siling Yang, Tong Wu, Gang Chen, Xian-He Sun

    Current recommendation systems are designed with the assumption that computer memory is volatile. Thus, they periodically execute checkpointing operations to save model weights and embeddings to slower solid-state drives (SSDs) for failure recovery. Such designs cannot leverage the unique characteristics of non-volatile memory (NVM). This paper proposes a new recommendation model training framework called NVrec , which uses a hybrid memory system consisting of DRAM and NVM to support computing and in-memory checkpointing. NVrec proposes three recommendation-oriented techniques. (1) It partitions embeddings so that only popular vectors are placed in DRAM to hide NVM-induced latency. (2) It maintains a dual-version scheme for in-memory checkpointing: each vector in NVM alternates between a persistent version for crash recovery and a runtime version for training updates. (3) It employs fine-grained parallelism to hide the checkpointing overhead of model parameters in DRAM and GPU. Our experiments demonstrate that NVrec improves recommendation-model training throughput by 1.05 × -6.85 × and the checkpointing process by more than 5 × compared to existing models when evaluated on the synthetic and real-world datasets.

  • ImPACT: Importance-Informed Prefetching and Caching for I/O-Bound DNN Training

    by Weijian Chen, Shuibing He, Ruidong Zhang, Xuechen Zhang, Ping Chen, Siling Yang, Haoyang Qu, Xuan Zhan

    Fetching large amounts of DNN training data from storage systems causes high I/O latency and GPU stalls. Importance sampling can reduce data processing on GPUs while maintaining model accuracy, but current frameworks lack a prefetching and caching layer to optimize data fetches and cache management based on sample importance. This leads to unnecessary fetches, poor cache hit ratios, and random I/Os. We present ImPACT, an importance-informed prefetching and caching system, to accelerate I/O-bound DNN training. First, we propose an importance-informed prefetching technique to reduce the prefetching of unimportant data. Then, we introduce an importance-aware caching layer, partitioned into two regions: H-cache and L-cache, which store samples of high importance and low importance respectively. Rather than using recency or frequency, we manage data items in H-cache according to their corresponding sample importance. When there is a cache miss in L-cache, we use sample substitutability and dynamic packaging to improve the cache hit ratio and reduce the number of random I/Os. Our experimental results show that ImPACT has a negligible impact on training accuracy while speeding up DNN training by up to 3.5$ \times $ compared to state-of-the-art prefetching and caching systems.


  • Open Access

    AUTOHET: An Automated Heterogeneous ReRAM-Based Accelerator for DNN Inference

    by Tong Wu, Shuibing He, Jianxin Zhu, Weijian Chen, Siling Yang, Ping Chen, Yanlong Yin, Xuechen Zhang, Xian-He Sun, Gang Chen

    ReRAM-based accelerators have become prevalent in accelerating deep neural network inference owing to their in-situ computing capability of ReRAM crossbars. However, most existing ReRAM-based accelerators are designed with homogeneous crossbars, leading to either low resource utilization or sub-optimal energy efficiency. In this paper, we propose AutoHet, an automated heterogeneous ReRAM-based accelerator with varied-size crossbars for different DNN layers. To achieve both high crossbar utilization and energy efficiency, AutoHet uses a reinforcement learning algorithm to automatically determine the proper crossbar configuration for each DNN layer. Additionally, AutoHet introduces rectangle crossbars and a tile-shared crossbar allocation scheme to reduce crossbar wastage and energy consumption. Experiment results show that AutoHet effectively improves crossbar utilization by up to 3.1 × and reduces energy consumption by up to 94.6%, compared to approaches with homogeneous ReRAM crossbars.


  • APQ: Automated DNN Pruning and Quantization for ReRAM-Based Accelerators

    by Siling Yang, Shuibing He, Hexiao Duan, Weijian Chen, Xuechen Zhang, Tong Wu, Yanlong Yin

    Emerging ReRAM-based accelerators support in-memory computation to accelerate deep neural network (DNN) inference. Weight matrix pruning is a widely used technique to reduce the size of DNN models, thereby reducing the resource and energy consumption of ReRAM-based accelerators. However, existing pruning works for ReRAM-based accelerators have three major issues. First, they use heuristics or rules from domain experts to prune the weights, leading to sub-optimal pruning policies. Second, they use row or column-level coarse-granularity methods to prune weights, resulting in poor compression rates with model accuracy constraints. Third, they only apply the weight pruning technique individually, losing the compression opportunity of both pruning and quantization. In this article, we propose an Automated DNN Pruning and Quantization framework, named APQ , for ReRAM-based accelerators. First, APQ adopts reinforcement learning (RL) to automatically determine the pruning policy for DNN layers for a global optimum. Second, it prunes and maps weight matrices to a ReRAM-based accelerator in a finer granularity of column-vector, which improves the compression rates with the accuracy constraints. To address the dislocation problem, it uses a new data path in ReRAM-based accelerators to correctly index and feed input to matrix-vector computation. Third, to further reduce resource consumption, APQ also leverages reinforcement learning to automatically determine the quantization bitwidth of each layer of the pruned DNN model. Experimental results show that, APQ achieves up to 4.52X compression rate, 4.11X area efficiency, and 4.51X energy efficiency with similar or even higher model accuracy, compared to the state-of-the-art work.