JWJingyu Wang

Jingyu Wang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Jingyu Wang has a total of 45 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • RE-Specter: Examining the Architectural Features of Configurable CNN With Power Side-Channel

    by Lu Zhang, Dejun Mu, Jingyu Wang, Ruoyang Liu, Yifan He, Yaolei Li, Yu Tai, Shengbing Zhang, Xiaoya Fan, Huazhong Yang, et al.

    As domain-specific training data is recognized as valuable intellectual property, acquiring well-trained weights in convolutional neural networks (CNN) has emerged as a new threat to the neural network design community. To design a CNN accelerator that is resilient to side-channel threats, it is crucial to have an accurate and efficient security-driven framework at the early design stage. However, there is no standard way to perform root-cause analysis on the power side channel that exists in FPGA-based CNN accelerators. Therefore, we build RE-Specter, a framework that facilitates security-driven design space exploration (DSE) across various building components, combination patterns, and parallelism configurations in CNNs. The goal is to fully understand the power side-channel effects resulting from architectural modifications or optimization decisions. We further compare the benchmarks considering precision, resource utilization, and power side-channel leakage. Finally, we experimentally explore the design space of various architectural features. The experimental results show that low-bit precision delivers more secure architectures ( $68.9\times $ among DSPs, $2439\times $ among LUTs) in measurement-to-disclosure (MTD), but mixed-precision strategies are necessary to maintain the model accuracy. For loop optimization, in 16-parallel scenario, accumulator-based architecture outperforms the architecture featuring an adder tree with the improvements of $8.28\times $ in MTD and $1.38\times $ in PST.


  • Open Access

    A Survey of Computing-in-Memory Processor: From Circuit to Application

    by Wenyu Sun, Jinshan Yue, Yifan He, Zongle Huang, Jingyu Wang, Wenbin Jia, Yaolei Li, Luchang Lei, Hongyang Jia, Yongpan Liu

    The computing-in-memory (CIM) technique is emerging with the evolvement of big data and artificial intelligence (AI) application. The manuscript presents a systematic review of existing CIM works in a bottom-up view from circuit to application. Various types of CIM circuits based on different volatile/non-volatile devices are introduced. The micro CIM architectures are illustrated to support multi-bit precision computation. After that, several types of processor-level CIM chips are analyzed to reveal the system architecture design considerations. The corresponding CIM tool chains and applications beyond AI applications are also introduced. From circuit to application levels, this manuscript analyzes the design trade-offs, remained challenges and possible future design trends at different design hierarchies of CIM processors.


  • ULSeq-TA: Ultra-Long Sequence Attention Fusion Transformer Accelerator Supporting Grouped Sparse Softmax and Dual-Path Sparse LayerNorm

    by Jingyu Wang, Lu Zhang, Xueqing Li, Huazhong Yang, Yongpan Liu

    Transformer networks have been increasingly successful in various fields. The input sequence lengths have become much larger as the algorithm and task complexity develops, which is challenging due to high computational and storage cost. Softmax and LayerNorm are bottleneck nonlinear operators in ultra-long sequence Transformer networks. To improve the efficiency of Softmax, assumption-based and quantization-based Softmax approaches are introduced. However, the sparsity potential to accelerate Softmax itself is not fully discovered. To improve the efficiency of LayerNorm, some works reduce the input size, and some works explore the pipeline. However, the sparsity potential is also not yet explored. To address these challenges, this article presents the ULSeq-TA software–hardware co-design framework. The software includes 1) the grouped sparse Softmax method to leverage the data magnifying characteristic to explore the middle and post-Softmax sparse processing and 2) the dual-path sparse LayerNorm method which explores the dimensional significance for sparse calculation. The hardware includes 1) an attention fusion architecture which reduces the on-chip memory with fused operators; 2) the grouped sparse Softmax core; and 3) the dual-path sparse LayerNorm core. Experiments show that the software achieves $4.45\times $ and $7.59\times $ computation reduction with little output difference for Softmax and LayerNorm, respectively. The hardware architecture supports at most 32768 sequence length with only 186-kB on-chip memory and achieves $1.75\times -1.98\times $ and $3.22\times -4.32\times $ speedups for sparse Softmax core and sparse LayerNorm core with little accuracy loss, respectively.


  • An Energy-Efficient Computing-in-Memory NN Processor With Set-Associate Blockwise Sparsity and Ping-Pong Weight Update

    by Jinshan Yue, Yongpan Liu, Xiaoyu Feng, Yifan He, Jingyu Wang, Zhe Yuan, Mingtao Zhan, Jiaxin Liu, Jian-Wei Su, Yen-Lin Chung, et al.

    Computing-in-memory (CIM) chips have demonstrated the potential high energy efficiency for low-power neural network (NN) processors. Even with energy-efficient CIM macros, the existing system-level CIM chips still lack deep exploration on sparsity and large models, which prevents a higher system energy efficiency. This work presents a CIM NN processor with more sufficient support of sparsity and higher utilization rate. Three key innovations are proposed. First, a set-associate blockwise sparsity strategy is designed, which simultaneously saves execution time, power, and storage space. Second, a ping-pong weight update mechanism is proposed for a higher utilization rate, enabling simultaneous execution of CIM and write operations. Third, an efficient CIM macro is implemented with adaptive analog-digital converter (ADC) precision for better sparsity utilization and performance-accuracy trade-off. The 65-nm fabricated chip shows 9.5-TOPS/W system energy efficiency at 4-bit precision, with 6.25 $\times $ actual improvement compared with a state-of-the-art CIM chip. Besides, this work supports high CIM execution accuracy on the ImageNet dataset.


  • A Weight-Reload-Eliminated Compute-in-Memory Accelerator for 60 fps 4K Super-Resolution

    by Yaolei Li, Jinshan Yue, Jingyu Wang, Chen Tang, Yifan He, Wenbin Jia, Kaiwei Zou, Lu Zhang, Huazhong Yang, Yongpan Liu

    Convolutional neural networks (CNNs) have significantly enhanced the quality of image super-resolution (SR). However, deploying CNN with deeper layers for real-time 60fps and higher resolution SR, e.g. 4K, brings higher performance requirement, incurring huge area and power overhead as well. In this paper, a high performance compute-in-memory (CIM) accelerator with high energy and area efficiency is proposed for deep-layer CNN-based SR (CNNSR). It adopts alternate time division serial (ATS) dataflow to guarantee the high performance, which eliminates frequent and time-consuming weight reload operation of CIM macros. Moreover, considering the extremely high ratio of computation quantity to weight amount in the deep-layer CNNSR scenario, an energy and area efficiency enhanced digital CIM macro with high computation-to-memory ratio (CMR) is proposed. The proposed accelerator achieves 33.3TOPS peak performance, supporting 4K SR at no less than 60fps. It is evaluated under TSMC 28nm process, achieving 13.96TOPS/W energy efficiency and 1.85TOPS/mm area efficiency, which are 2.79× and 2.2× better than the state-of-the-art CNNSR accelerator, respectively.


  • Pareto Frequency-Aware Power Side-Channel Countermeasure Exploration on CNN Systolic Array

    by Lu Zhang, Dejun Mu, Yuxuan Huang, Jingyu Wang, Yifan He, Yaolei Li, Lizhou Liu, Kaiwei Zou, Huazhong Yang, Yongpan Liu

    Neural Network (NN) accelerators are widely used in many scenarios, including computer vision, speech recognition, automatic driving, etc. The trained weights of NNs are considered confidential properties due to their commercial value, thus becoming the potential targets for hardware reverse-engineering intruders. In this paper, we propose a low-cost, high-security, easy-to-deploy countermeasure for convolutional neural network (CNN) systolic array against weight reverse-engineering through power side channels. This countermeasure utilizes the clock frequency-switching and phase-shifting techniques to mitigate the power side channels. We further make a design space exploration to trade-off between Side-Channel Analysis (SCA) security and performance by leveraging the non-ideality of CMOS device. The experimental results show that the proposed countermeasure achieves an 81.6% absolute correlation reduction, thus incurring more than 100× power trace consumption (more secure) while maintaining the MAC (multiply-and-accumulate) blocks work at the Pareto-optimal frequencies (Pareto faster).


  • An RRAM-Based Digital Computing-in-Memory Macro With Dynamic Voltage Sense Amplifier and Sparse-Aware Approximate Adder Tree

    by Yifan He, Jinshan Yue, Xiaoyu Feng, Yuxuan Huang, Hongyang Jia, Jingyu Wang, Lu Zhang, Wenyu Sun, Huazhong Yang, Yongpan Liu

    RRAM is a promising candidate to implement large-capacity in-memory computing on edge AI devices due to its high density. However, the efficiency and accuracy of RRAM-based computing-in-memory (CIM) works are limited by large accumulation currents and device variations. The SRAM-based digital CIM achieves superior performance and efficiency while the adder tree dominates the area. In this paper, a digital RRAM CIM macro is proposed to achieve a better trade-off between accuracy, energy, and performance by three techniques. First, a dynamic voltage sense amplifier is designed to reduce >90% read currents of low resistance state (LRS) cell. Second, an OR gate approximate adder tree is proposed to reduce the area of the adder tree by 40%. Third, a sparse-aware finetuning algorithm is proposed to reduce the accuracy loss of approximate arithmetic to 0.4% on Cifar-10 dataset. The proposed design achieves 1966TOPS/W energy efficiency and 51.2GOPS/Kb normalized throughput at 1-bit precision which is 1.2× and 1.8× higher than previous RRAM-based designs. This work demonstrates the advantage of digital CIM using RRAM devices.


  • Bit-Aware Fault-Tolerant Hybrid Retraining and Remapping Schemes for RRAM-Based Computing-in-Memory Systems

    by Yuxuan Huang, Yifan He, Jingyu Wang, Jinshan Yue, Lu Zhang, Kaiwei Zou, Huazhong Yang, Yongpan Liu

    Non-volatile computing-in-memory (nvCIM) has become one of the most efficient methods to deal with increasingly complicated neural networks compared to the traditional von Neumann architecture. However, due to the immature fabrication issues, the yield of resistive random access memory (RRAM) cells is limited, which causes the Stuck-At-Faults (SAFs) and decrease of network classification accuracy. In this brief, the bit-aware fault-tolerant hybrid retraining and remapping schemes are proposed to restore the network classification accuracy caused by SAFs. The low weight bits are retrained with the fault mask and the high weight bits are remapped with column swap and zero correction methods. For those bits whose values are still wrong after remapping, the fault index is used to record and calibrate. Experimental results show that the classification accuracy of MNIST, CIFAR-10, and CIFAR-100 is restored from 12.54% to 98.13%, from 10.29% to 90.60%, and from 0.94% to 74.30% with only 10.3% energy consumption and 5.88% area overhead, respectively.


  • PACA: A Pattern Pruning Algorithm and Channel-Fused High PE Utilization Accelerator for CNNs

    by Jingyu Wang, Songming Yu, Zhuqing Yuan, Jinshan Yue, Zhe Yuan, Ruoyang Liu, Yanzhi Wang, Huazhong Yang, Xueqing Li, Yongpan Liu

    In recent years, convolutional neural networks (CNNs) have achieved significant advancements in various fields. However, the computation and storage overheads of CNNs are overwhelming for Internet-of-Things devices. Both network pruning algorithms and hardware accelerators have been introduced to empower CNN inference at the edge. Network pruning algorithms reduce the size and computational cost of CNNs by regularizing unimportant weights to zeros. However, existing works lack intrakernel structured types to tradeoff between sparsity and hardware efficiency, and the index storage for irregularly pruned networks is significant. Hardware accelerators leverage the sparsity of pruned CNNs to improve energy efficiency. However, their process element (PE) utilization rate is low because of uneven sparsity among input convolutional kernels. To overcome these problems, we propose PACA: a Pattern pruning Algorithm and Channel-fused high PE utilization Accelerator for CNNs. It includes three parts: a pattern pruning algorithm to explore the intrakernel sparsity type and reduce the index storage, a channel-fused hardware architecture to reduce the PEs’ idle rate and improve the performance, and a heuristic and taboo search-based smart fusion scheduler to analyze the idle PE problem and schedule the channel fusion in hardware. To demonstrate the effectiveness of PACA, we have implemented the software parts by Python and the hardware architecture by RTL codes. Experimental results on various datasets show that compared with an existing work, PACA can reduce the index storage overhead by $3.47\times $ – $5.63\times $ with 3.85–9.12 average patterns, and it can improve the hardware performance by $2.01\times $ – $5.53\times $ because of PEs’ idle rate reduction.


  • A 65-nm Energy-Efficient Interframe Data Reuse Neural Network Accelerator for Video Applications

    by Yixiong Yang, Yongpan Liu, Zhe Yuan, Wenyu Sun, Ruoyang Liu, Jingyu Wang, Jinshan Yue, Xiaoyu Feng, Zhuqing Yuan, Xueqing Li, et al.

    An energy-efficient convolutional neural network (CNN) accelerator is proposed for the video application. Previous works exploited the sparsity of differential (Diff) frame activation, but the improvement is limited as many Diff-frame data is small but non-zero. Processing of irregular sparse data also leads to low hardware utilization. To solve these problems, two key innovations are proposed in this article. First, we implement a hybrid-precision inter-frame-reuse architecture which takes advantage of both low bit-width and high sparsity of Diff-frame data. This technology can accelerate 3.2 $\times $ inference speed with no accuracy loss. Second, we design a conv-pattern-aware processing array that achieves the 2.48 $\times $ –14.2 $\times $ PE utilization rate to process sparse data for different convolution kernels. The accelerator chip was implemented in 65-nm CMOS technology. To the best of our knowledge, it is the first silicon-proven CNN accelerator that supports inter-frame data reuse. Attributed to the inter-frame similarity, this video CNN accelerator reaches the minimum energy consumption of 24.7 $\mu \text{J}$ /frame in the MobileNet-slim model, which is 76.3% less than the baseline.