Yun-Chen Lo
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Yun-Chen Lo has a total of 33 co-authors. Here are the top co-authors by number of shared publications:
- Ren-Shuo Liu
Department of Electrical Engineering, National Tsing Hua University (NTHU), Hsinchu, Taiwan
4 shared publications - Hung-Hsi Hsu
National Tsing Hua University (NTHU)
2 shared publications - Meng-Fan Chang
National Tsing Hua University
2 shared publications Tsung-Yung Jonathan Chang
Taiwan Semiconductor Manufacturing Company Limited (TSMC), No. 8, Li-Hsin Rd. 6, Hsinchu Science Park, Hsinchu 300, Taiwan, R.O.C.
2 shared publications- Yu-Chih Tsai
Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan
2 shared publications Yu-Der Chih
Taiwan Semiconductor Manufacturing Company Limited (TSMC), No. 8, Li-Hsin Rd. 6, Hsinchu Science Park, Hsinchu 300, Taiwan, R.O.C.
2 shared publications
Latest Publications
- 4 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 73 (9) , 2136-2149
ISSA: Architecting CNN Accelerators Using Input-Skippable, Set-Associative Computing-in-Memory
Among several emerging architectures, computing in memory (CIM), which features in-situ analog computation , is a potential solution to the data movement bottleneck of the Von Neumann architecture for artificial intelligence (AI). Interestingly, more strengths of CIM significantly different from in-situ analog computation are not widely known yet. In this work, we point out that mutually stationary vectors (MSVs) , which can be maximized by introducing associativity to CIM, are another inherent power unique to CIM. By MSVs, CIM exhibits significant freedom to dynamically vectorize the stored data (e.g., weights) to perform agile computation using the dynamically formed vectors. We have designed and realized an SA-CIM silicon prototype and corresponding architecture and acceleration schemes in the TSMC 28 nm process. More specifically, the contributions of this paper are fivefold: 1) We identify MSVs as new features that can be exploited to improve the current performance and energy challenges of the CIM-based hardware. 2) We propose SA-CIM to enhance MSVs (input-reordering flexibility) for skipping the zeros, small values, and sparse vectors. 3) We propose channel swapping to enhance the zero-skipping technique. 4) We propose a transposed systolic dataflow to efficiently conduct conv3×3 while being capable of exploiting input-skipping schemes. 5) We propose a design flow to search for optimal aggressive skipping scheme setups while satisfying the accuracy loss constraint. The proposed ISSA architecture improves the throughput by 1.91× to 2.97× speedup and the energy efficiency by 2.5× to 4.2×.
- 4 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 73 (9) , 2136-2149
ISSA: Architecting CNN Accelerators Using Input-Skippable, Set-Associative Computing-in-Memory
Among several emerging architectures, computing in memory (CIM), which features in-situ analog computation , is a potential solution to the data movement bottleneck of the Von Neumann architecture for artificial intelligence (AI). Interestingly, more strengths of CIM significantly different from in-situ analog computation are not widely known yet. In this work, we point out that mutually stationary vectors (MSVs) , which can be maximized by introducing associativity to CIM, are another inherent power unique to CIM. By MSVs, CIM exhibits significant freedom to dynamically vectorize the stored data (e.g., weights) to perform agile computation using the dynamically formed vectors. We have designed and realized an SA-CIM silicon prototype and corresponding architecture and acceleration schemes in the TSMC 28 nm process. More specifically, the contributions of this paper are fivefold: 1) We identify MSVs as new features that can be exploited to improve the current performance and energy challenges of the CIM-based hardware. 2) We propose SA-CIM to enhance MSVs (input-reordering flexibility) for skipping the zeros, small values, and sparse vectors. 3) We propose channel swapping to enhance the zero-skipping technique. 4) We propose a transposed systolic dataflow to efficiently conduct conv3×3 while being capable of exploiting input-skipping schemes. 5) We propose a design flow to search for optimal aggressive skipping scheme setups while satisfying the accuracy loss constraint. The proposed ISSA architecture improves the throughput by 1.91× to 2.97× speedup and the energy efficiency by 2.5× to 4.2×.
- Top Cited Papers
- 19 April 2024
- Published by American Association for the Advancement of Science (AAAS) in Science
- Vol. 384 (6693) , 325-332
Fusion of memristor and digital compute-in-memory processing for energy-efficient edge computing
Artificial intelligence (AI) edge devices prefer employing high-capacity nonvolatile compute-in-memory (CIM) to achieve high energy efficiency and rapid wakeup-to-response with sufficient accuracy. Most previous works are based on either memristor-based CIMs, which suffer from accuracy loss and do not support training as a result of limited endurance, or digital static random-access memory (SRAM)–based CIMs, which suffer from large area requirements and volatile storage. We report an AI edge processor that uses a memristor-SRAM CIM-fusion scheme to simultaneously exploit the high accuracy of the digital SRAM CIM and the high energy-efficiency and storage density of the resistive random-access memory memristor CIM. This also enables adaptive local training to accommodate personalized characterization and user environment. The fusion processor achieved high CIM capacity, short wakeup-to-response latency (392 microseconds), high peak energy efficiency (77.64 teraoperations per second per watt), and robust accuracy (<0.5% accuracy loss). This work demonstrates that memristor technology has moved beyond in-lab development stages and now has manufacturability for AI edge processors.
- 19 April 2024
- Published by American Association for the Advancement of Science (AAAS) in Science
- Vol. 384 (6693) , 325-332
Top Cited PapersFusion of memristor and digital compute-in-memory processing for energy-efficient edge computing
Artificial intelligence (AI) edge devices prefer employing high-capacity nonvolatile compute-in-memory (CIM) to achieve high energy efficiency and rapid wakeup-to-response with sufficient accuracy. Most previous works are based on either memristor-based CIMs, which suffer from accuracy loss and do not support training as a result of limited endurance, or digital static random-access memory (SRAM)–based CIMs, which suffer from large area requirements and volatile storage. We report an AI edge processor that uses a memristor-SRAM CIM-fusion scheme to simultaneously exploit the high accuracy of the digital SRAM CIM and the high energy-efficiency and storage density of the resistive random-access memory memristor CIM. This also enables adaptive local training to accommodate personalized characterization and user environment. The fusion processor achieved high CIM capacity, short wakeup-to-response latency (392 microseconds), high peak energy efficiency (77.64 teraoperations per second per watt), and robust accuracy (<0.5% accuracy loss). This work demonstrates that memristor technology has moved beyond in-lab development stages and now has manufacturability for AI edge processors.
- Open Access
- 28 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 1002-1015
Bucket Getter: A Bucket-based Processing Engine for Low-bit Block Floating Point (BFP) DNNs
Block floating point (BFP), an efficient numerical system for deep neural networks (DNNs), achieves a good trade-off between dynamic range and hardware costs. Specifically, prior works have demonstrated that BFP format with 3 ∼ 5-bit mantissa can achieve FP32-comparable accuracy for various DNN workloads. We find that the floating-point adder (FP-Acc), which contains modules for normalization, alignment, addition, and fixed-point-to-floating-point (FXP2FP) conversion, dominates the power and area overheads, hence hindering the hardware efficiency of state-of-the-art low-bit BFP processing engines (BFP-PE). To mitigate the identified issue, we propose Bucket Getter, a novel architecture implemented with the following techniques for improving the energy efficiency and area efficiency: 1) we propose a bucket-based accumulation unit prior to FP-Acc, which uses multiple small accumulators (buckets) that are responsible for a small range of exponent values where intermediate results are distributed accordingly, and b) accumulate in FXP domain. This reduces the activities of power-hungry a) alignment and b) format conversion units. 2) We propose inter-bucket carry propagation, which allows each bucket to transmit overflow to an adjacent bucket and further reduces the activity of FP-Acc. 3) We propose an out-of-bound-aware, adaptive and circular bucket accumulator to significantly reduce the overhead for the bucket-based accumulator. 4) We further propose shared FP-Acc, which exploits the low activity of FP-Acc in the bucket-based architecture and shares an FP-Acc across several MAC engines to reduce the area overhead of FP-Acc. The experimental results based on TSMC 40 nm demonstrate that our proposed Bucket Getter architecture reduces the computational energy by up to 57% and improves the area efficiency by up to 1.4 ×, compared to state-of-the-art BFP engines across seven representative DNN models. Furthermore, our proposed approach helps state-of-the-art floating-point engines reduce up to 32% of the PE area and 81% of the PE power.
- 28 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 1002-1015
Open AccessBucket Getter: A Bucket-based Processing Engine for Low-bit Block Floating Point (BFP) DNNs
Block floating point (BFP), an efficient numerical system for deep neural networks (DNNs), achieves a good trade-off between dynamic range and hardware costs. Specifically, prior works have demonstrated that BFP format with 3 ∼ 5-bit mantissa can achieve FP32-comparable accuracy for various DNN workloads. We find that the floating-point adder (FP-Acc), which contains modules for normalization, alignment, addition, and fixed-point-to-floating-point (FXP2FP) conversion, dominates the power and area overheads, hence hindering the hardware efficiency of state-of-the-art low-bit BFP processing engines (BFP-PE). To mitigate the identified issue, we propose Bucket Getter, a novel architecture implemented with the following techniques for improving the energy efficiency and area efficiency: 1) we propose a bucket-based accumulation unit prior to FP-Acc, which uses multiple small accumulators (buckets) that are responsible for a small range of exponent values where intermediate results are distributed accordingly, and b) accumulate in FXP domain. This reduces the activities of power-hungry a) alignment and b) format conversion units. 2) We propose inter-bucket carry propagation, which allows each bucket to transmit overflow to an adjacent bucket and further reduces the activity of FP-Acc. 3) We propose an out-of-bound-aware, adaptive and circular bucket accumulator to significantly reduce the overhead for the bucket-based accumulator. 4) We further propose shared FP-Acc, which exploits the low activity of FP-Acc in the bucket-based architecture and shares an FP-Acc across several MAC engines to reduce the area overhead of FP-Acc. The experimental results based on TSMC 40 nm demonstrate that our proposed Bucket Getter architecture reduces the computational energy by up to 57% and improves the area efficiency by up to 1.4 ×, compared to state-of-the-art BFP engines across seven representative DNN models. Furthermore, our proposed approach helps state-of-the-art floating-point engines reduce up to 32% of the PE area and 81% of the PE power.
- 2 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. 59 (1) , 116-127
A Nonvolatile AI-Edge Processor With SLC–MLC Hybrid ReRAM Compute-in-Memory Macro Using Current–Voltage-Hybrid Readout Scheme
On-chip non-volatile compute-in-memory (nvCIM) enables artificial intelligence (AI)-edge processors to perform multiply-and-accumulate (MAC) operations while enabling the non-volatile storage of weight data in power-off mode to enhance energy efficiency. However, the design challenges of nvCIM-based AI-edge processors include: 1) lack of a nvCIM-friendly computing flow; 2) a tradeoff between usage of memory devices versus process variations, computing yield and area overhead; 3) long computing latency and low energy efficiency; and 4) small-signal margin and large bitline current. This article presents an nvCIM-friendly AI-edge processor that uses a hybrid-mode resistive random access memory nvCIM (hmRe-nvCIM) macro to overcome the abovementioned challenges by three processor-level schemes: 1) a multimode nvCIM engine controller (mmCIM-EC); 2) a bitwise-input-sparsity and place-value-aware dynamic accumulation (BIS-PVA-DA); and 3) a bitwise weight column inversion (BWCI) and two macro-level schemes: 1) a dynamic-accumulation-aware current quantization (DACQ) and 2) a current–voltage-hybrid analog-to-digital converter (CVH-ADC). The proposed AI-edge processor fabricated using 22-nm technology achieved 51.4 TOPS/W and 472.7- $\mu \text{s}$ wake-up to response time, while the hmRe-nvCIM macro achieved 67.2 TOPS/W under 8-bit input, 8-bit weight, and 22- or 24-bit output precision.
- 2 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. 59 (1) , 116-127
A Nonvolatile AI-Edge Processor With SLC–MLC Hybrid ReRAM Compute-in-Memory Macro Using Current–Voltage-Hybrid Readout Scheme
On-chip non-volatile compute-in-memory (nvCIM) enables artificial intelligence (AI)-edge processors to perform multiply-and-accumulate (MAC) operations while enabling the non-volatile storage of weight data in power-off mode to enhance energy efficiency. However, the design challenges of nvCIM-based AI-edge processors include: 1) lack of a nvCIM-friendly computing flow; 2) a tradeoff between usage of memory devices versus process variations, computing yield and area overhead; 3) long computing latency and low energy efficiency; and 4) small-signal margin and large bitline current. This article presents an nvCIM-friendly AI-edge processor that uses a hybrid-mode resistive random access memory nvCIM (hmRe-nvCIM) macro to overcome the abovementioned challenges by three processor-level schemes: 1) a multimode nvCIM engine controller (mmCIM-EC); 2) a bitwise-input-sparsity and place-value-aware dynamic accumulation (BIS-PVA-DA); and 3) a bitwise weight column inversion (BWCI) and two macro-level schemes: 1) a dynamic-accumulation-aware current quantization (DACQ) and 2) a current–voltage-hybrid analog-to-digital converter (CVH-ADC). The proposed AI-edge processor fabricated using 22-nm technology achieved 51.4 TOPS/W and 472.7- $\mu \text{s}$ wake-up to response time, while the hmRe-nvCIM macro achieved 67.2 TOPS/W under 8-bit input, 8-bit weight, and 22- or 24-bit output precision.
- 25 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 22 (2) , 125-128
LV: Latency-Versatile Floating-Point Engine for High-Performance Deep Neural Networks
Computing latency is an important system metric for Deep Neural Networks (DNNs) accelerators. To reduce latency, this work proposes LV, a latency-versatile floating-point engine (FP-PE), which contains the following key contributions: 1) an approximate bit-versatile multiplier-and-accumulate (BV-MAC) unit with early shifter and 2) an on-demand fixed-point-to-floating-point conversion (FXP2FP) unit. The extensive experimental results show that LV outperforms baseline FP-PE and redundancy-aware FP-PE by up to 2.12× and 1.3× speedup using TSMC 40-nm technology, achieving comparable accuracy on the ImageNet classification tasks.
- 25 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 22 (2) , 125-128
LV: Latency-Versatile Floating-Point Engine for High-Performance Deep Neural Networks
Computing latency is an important system metric for Deep Neural Networks (DNNs) accelerators. To reduce latency, this work proposes LV, a latency-versatile floating-point engine (FP-PE), which contains the following key contributions: 1) an approximate bit-versatile multiplier-and-accumulate (BV-MAC) unit with early shifter and 2) an on-demand fixed-point-to-floating-point conversion (FXP2FP) unit. The extensive experimental results show that LV outperforms baseline FP-PE and redundancy-aware FP-PE by up to 2.12× and 1.3× speedup using TSMC 40-nm technology, achieving comparable accuracy on the ImageNet classification tasks.