Chia-Lin Yang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Chia-Lin Yang has a total of 155 co-authors. Here are the top co-authors by number of shared publications:
- Yao-Wen Chang
National Taiwan University, Taipei, Taiwan, Taiwan
16 shared publications Ping-Hung Yuh
Taiwan Semiconductor Manufacturing Company, Taiwan
13 shared publications- Jian-Jia Chen
TU Dortmund University, Dortmund, Germany
10 shared publications Yi-Jung Chen
National Chi Nan University
9 shared publications- Tei-Wei Kuo
National Taiwan University
8 shared publications Chung-Hsiang Lin
National Taiwan University, Taipei, Taiwan
6 shared publications
Latest Publications
- 31 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 25 (1) , 178-181
Efficient MoE Model Fine-Tuning on Commodity GPU Server With Offloading
Fine-tuning enables the efficient adaptation of pre-trained models on commodity GPUs but is constrained by the memory capacity. To mitigate this, recent systems combine model parallelism with CPU-GPU offloading. We focus on fine-tuning Mixture-of-Experts (MoE) models on commodity GPU server using layer-wise offloading and expert parallelism (EP). However, EP-based MoE training suffers from significant load imbalance because the gating network routes tokens unevenly among experts. To address this load imbalance issue, we propose a Prediction-based Dynamic Expert Placement (PDEP) framework. By employing a layer-wise workload predictor, PDEP utilizes the CPU-to-GPU weight transfer to dynamically adjust expert assignments. Our method reduces GPU idle time, achieves near-balanced workload distribution, and delivers up to 1.49× end-to-end speedup over static expert placement.
- 31 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 25 (1) , 178-181
Efficient MoE Model Fine-Tuning on Commodity GPU Server With Offloading
Fine-tuning enables the efficient adaptation of pre-trained models on commodity GPUs but is constrained by the memory capacity. To mitigate this, recent systems combine model parallelism with CPU-GPU offloading. We focus on fine-tuning Mixture-of-Experts (MoE) models on commodity GPU server using layer-wise offloading and expert parallelism (EP). However, EP-based MoE training suffers from significant load imbalance because the gating network routes tokens unevenly among experts. To address this load imbalance issue, we propose a Prediction-based Dynamic Expert Placement (PDEP) framework. By employing a layer-wise workload predictor, PDEP utilizes the CPU-to-GPU weight transfer to dynamically adjust expert assignments. Our method reduces GPU idle time, achieves near-balanced workload distribution, and delivers up to 1.49× end-to-end speedup over static expert placement.
- Open Access
- 30 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Artificial Intelligence
- Vol. 2 (2) , 114-125
Capacitor Minimization in Analog Synapses for Efficient and Compact Binarized SNNs
Accelerating the immense workloads of Neural Networks (NNs) is a critical challenge, and analog computing-based approaches present a promising solution. Among these, the Integrate-and-Fire (IF) Spiking Neural Network (SNN) stands out for its potential in efficient neural computation. However, achieving high inference accuracy through precise multiply-accumulate (MAC) operations necessitates a large membrane capacitor—224 to 547 times larger than the synapse array at the 14nm technology node—resulting in prohibitive area costs. Additionally, the large capacitor size introduces higher energy consumption and longer access times. Thus, a key research challenge is maintaining accuracy while mitigating the costs associated with the large membrane capacitor. In this work, we propose a HW/SW Codesign method, called CapMin, for capacitor size minimization in analog computing IF-SNNs. CapMin minimizes the capacitor size by reducing the number of spike times needed for accurate operation of the HW, based on the absolute frequency of MAC level occurrences in the SW. To increase the operation of IF-SNNs to current variation, we propose the method CapMin-V, which trades capacitor size for protection based on the reduced capacitor size found in CapMin. In our experiments, CapMin achieves more than a 14× reduction in capacitor size along with a 34% reduction in energy consumption over the state of the art, while CapMin-V achieves increased variation tolerance in the IF-SNN operation, requiring only a small increase in capacitor size.
- 30 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Artificial Intelligence
- Vol. 2 (2) , 114-125
Open AccessCapacitor Minimization in Analog Synapses for Efficient and Compact Binarized SNNs
Accelerating the immense workloads of Neural Networks (NNs) is a critical challenge, and analog computing-based approaches present a promising solution. Among these, the Integrate-and-Fire (IF) Spiking Neural Network (SNN) stands out for its potential in efficient neural computation. However, achieving high inference accuracy through precise multiply-accumulate (MAC) operations necessitates a large membrane capacitor—224 to 547 times larger than the synapse array at the 14nm technology node—resulting in prohibitive area costs. Additionally, the large capacitor size introduces higher energy consumption and longer access times. Thus, a key research challenge is maintaining accuracy while mitigating the costs associated with the large membrane capacitor. In this work, we propose a HW/SW Codesign method, called CapMin, for capacitor size minimization in analog computing IF-SNNs. CapMin minimizes the capacitor size by reducing the number of spike times needed for accurate operation of the HW, based on the absolute frequency of MAC level occurrences in the SW. To increase the operation of IF-SNNs to current variation, we propose the method CapMin-V, which trades capacitor size for protection based on the reduced capacitor size found in CapMin. In our experiments, CapMin achieves more than a 14× reduction in capacitor size along with a 34% reduction in energy consumption over the state of the art, while CapMin-V achieves increased variation tolerance in the IF-SNN operation, requiring only a small increase in capacitor size.
- Open Access
- 20 January 2025
- Published by Association for Computing Machinery (ACM)
- p. 1008-1015
In-Storage Read-Centric Seed Location Filtering Using 3D-NAND Flash for Genome Sequence Analysis
Read mapping is a critical bottleneck in genome sequence analysis, requiring costly approximate string matching to identify potential matches between reads and a reference genome. Pre-alignment filtering methods aim to mitigate this issue by filtering out unnecessary mapping locations, and implementing them with processing-in-memory (PIM) approaches offers potential benefits by offloading filtering from the computing unit. However, the sparse number of potential mapping locations for each read limits the utilization of PIM's parallel computing capabilities, thereby hindering the overlapping of filtering and sequence alignment to hide filtering latency overheads. In this paper, we propose a 3D NAND-based in-storage pre-alignment filtering approach. Leveraging the read depth property, we introduce a read-centric pre-alignment filtering method that enables parallel comparison of multiple reads. We co-design software and hardware for in-situ processing of read-centric pre-alignment filtering within the storage, capitalizing on 3D NAND Flash's approximate parallel search capability. When integrating with a representative read mapping accelerator, our design achieves an average 1.36x performance improvement with comparable energy consumption. Compared to the state-of-the-art (SOTA) PIM solution, our design results in 123.8x and 53.3x performance gain and energy efficiency improvement.
- 20 January 2025
- Published by Association for Computing Machinery (ACM)
- p. 1008-1015
Open AccessIn-Storage Read-Centric Seed Location Filtering Using 3D-NAND Flash for Genome Sequence Analysis
Read mapping is a critical bottleneck in genome sequence analysis, requiring costly approximate string matching to identify potential matches between reads and a reference genome. Pre-alignment filtering methods aim to mitigate this issue by filtering out unnecessary mapping locations, and implementing them with processing-in-memory (PIM) approaches offers potential benefits by offloading filtering from the computing unit. However, the sparse number of potential mapping locations for each read limits the utilization of PIM's parallel computing capabilities, thereby hindering the overlapping of filtering and sequence alignment to hide filtering latency overheads. In this paper, we propose a 3D NAND-based in-storage pre-alignment filtering approach. Leveraging the read depth property, we introduce a read-centric pre-alignment filtering method that enables parallel comparison of multiple reads. We co-design software and hardware for in-situ processing of read-centric pre-alignment filtering within the storage, capitalizing on 3D NAND Flash's approximate parallel search capability. When integrating with a representative read mapping accelerator, our design achieves an average 1.36x performance improvement with comparable energy consumption. Compared to the state-of-the-art (SOTA) PIM solution, our design results in 123.8x and 53.3x performance gain and energy efficiency improvement.
- Open Access
Resilient Endurance-Aware NVM-based PUF against Learning-based Attacks
Physical Unclonable Functions (PUFs) based on Non-Volatile Memory (NVM) technology have emerged as a promising solution for secure authentication and cryptographic applications. By leveraging the multi-level cell (MLC) characteristic of NVMs, these PUFs can generate a wide range of unique responses, enhancing their resilience to machine learning (ML) modeling attacks. However, a significant issue with NVM-based PUFs is their endurance problem; frequent write operations lead to wear and degradation over time, reducing the reliability and lifespan of the PUF. This paper addresses these issues by offering a comprehensive model to predict and analyze the effects of endurance changes on NVM PUFs. This model provides insights into how wear impacts the PUF's quality and helps in designing more robust PUFs. Building on this model, we present a novel design for NVM PUFs that significantly improves endurance. Our design approach incorporates advanced techniques to distribute write operations more evenly and reduce stress on individual cells. The result is an NVM PUF that demonstrates a $62\times$ improvement in endurance compared to current state-of-the-art solutions while maintaining protection against learning-based attacks.Open AccessResilient Endurance-Aware NVM-based PUF against Learning-based Attacks
Physical Unclonable Functions (PUFs) based on Non-Volatile Memory (NVM) technology have emerged as a promising solution for secure authentication and cryptographic applications. By leveraging the multi-level cell (MLC) characteristic of NVMs, these PUFs can generate a wide range of unique responses, enhancing their resilience to machine learning (ML) modeling attacks. However, a significant issue with NVM-based PUFs is their endurance problem; frequent write operations lead to wear and degradation over time, reducing the reliability and lifespan of the PUF. This paper addresses these issues by offering a comprehensive model to predict and analyze the effects of endurance changes on NVM PUFs. This model provides insights into how wear impacts the PUF's quality and helps in designing more robust PUFs. Building on this model, we present a novel design for NVM PUFs that significantly improves endurance. Our design approach incorporates advanced techniques to distribute write operations more evenly and reduce stress on individual cells. The result is an NVM PUF that demonstrates a $62\times$ improvement in endurance compared to current state-of-the-art solutions while maintaining protection against learning-based attacks. - Open Access
- 27 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-8
Co-Designing NVM-based Systems for Machine Learning and In-memory Search Applications
- 27 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-8
Open AccessCo-Designing NVM-based Systems for Machine Learning and In-memory Search Applications
- Open Access
- 16 September 2024
- Published by Association for Computing Machinery (ACM)
- p. 104-117
RecTS: A Temporal-Aware Memory System Optimization for Training Deep Learning Recommendation Models
Deep learning recommendation models (DLRM) are crucial for various services, which makes DLRM training a critical workload for data centers. To meet the memory demands of DLRM workloads, the adoption of SSD has become necessary for large-scale models. However, the irregular access patterns of embedding tables in DLRM training result in poor spatial locality and low cache hit rates in the memory system. Furthermore, the mismatched access granularity between DRAM and SSD leads to write amplification and reduced SSD lifespan. To address these challenges, we propose RecTS: a temporal-aware memory system optimization for training deep learning recommendation models. The system uses an innovative replacement policy and vector grouping to achieve a high cache hit rate and improve spatial locality, which in turn reduces write amplification. These mechanisms are designed based on the access pattern of DLRM. During DLRM training, the access pattern of embedding tables is spatially irregular but predetermined. Leveraging these a priori access patterns, the system evicts the vectors and dynamically gathers the evicted embedding vectors with the closer next access timing into an I/O chunk to enhance spatial locality. This improvement makes the system more effective for DLRM workloads. We conducted experiments on both CPU-only and CPU-GPU architectures, where the difference lies in whether the neural network computation (MLP) is computed on the CPU or GPU. Overall, the replacement policy and vector grouping increase the cache hit rate from 40-85% to over 99% across all experiment settings, speeding up the training time by 1.30-5.55 times. Moreover, the SSD write volume was reduced by 3.13-7.30 times, significantly enhancing SSD lifespan.
- 16 September 2024
- Published by Association for Computing Machinery (ACM)
- p. 104-117
Open AccessRecTS: A Temporal-Aware Memory System Optimization for Training Deep Learning Recommendation Models
Deep learning recommendation models (DLRM) are crucial for various services, which makes DLRM training a critical workload for data centers. To meet the memory demands of DLRM workloads, the adoption of SSD has become necessary for large-scale models. However, the irregular access patterns of embedding tables in DLRM training result in poor spatial locality and low cache hit rates in the memory system. Furthermore, the mismatched access granularity between DRAM and SSD leads to write amplification and reduced SSD lifespan. To address these challenges, we propose RecTS: a temporal-aware memory system optimization for training deep learning recommendation models. The system uses an innovative replacement policy and vector grouping to achieve a high cache hit rate and improve spatial locality, which in turn reduces write amplification. These mechanisms are designed based on the access pattern of DLRM. During DLRM training, the access pattern of embedding tables is spatially irregular but predetermined. Leveraging these a priori access patterns, the system evicts the vectors and dynamically gathers the evicted embedding vectors with the closer next access timing into an I/O chunk to enhance spatial locality. This improvement makes the system more effective for DLRM workloads. We conducted experiments on both CPU-only and CPU-GPU architectures, where the difference lies in whether the neural network computation (MLP) is computed on the CPU or GPU. Overall, the replacement policy and vector grouping increase the cache hit rate from 40-85% to over 99% across all experiment settings, speeding up the training time by 1.30-5.55 times. Moreover, the SSD write volume was reduced by 3.13-7.30 times, significantly enhancing SSD lifespan.
- 6 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Micro
- Vol. 46 (1) , 73-85
CIMNet: Joint Search for Neural Network and Computing-in-Memory Architectures
Computing-in-memory (CIM) architecture has been proven to effectively transcend the memory wall bottleneck, expanding the potential of low-power and high-throughput applications such as machine learning. Neural architecture search (NAS) designs ML models to meet a variety of accuracy, latency, and energy constraints. However, integrating CIM into NAS presents a major challenge due to additional simulation overhead from the non-ideal characteristics of CIM hardware. This work introduces a quantization and device aware accuracy predictor that jointly scores quantization policy, CIM architecture, and neural network architecture, eliminating the need for time-consuming simulations in the search process. We also propose reducing the search space based on architectural observations, resulting in a well-pruned search space customized for CIM. These allow for efficient exploration of superior combinations in mere CPU minutes. Our methodology yields CIMNet, which consistently improves the trade-off between accuracy and hardware efficiency on benchmarks, providing valuable architectural insights.
- 6 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Micro
- Vol. 46 (1) , 73-85
CIMNet: Joint Search for Neural Network and Computing-in-Memory Architectures
Computing-in-memory (CIM) architecture has been proven to effectively transcend the memory wall bottleneck, expanding the potential of low-power and high-throughput applications such as machine learning. Neural architecture search (NAS) designs ML models to meet a variety of accuracy, latency, and energy constraints. However, integrating CIM into NAS presents a major challenge due to additional simulation overhead from the non-ideal characteristics of CIM hardware. This work introduces a quantization and device aware accuracy predictor that jointly scores quantization policy, CIM architecture, and neural network architecture, eliminating the need for time-consuming simulations in the search process. We also propose reducing the search space based on architectural observations, resulting in a well-pruned search space customized for CIM. These allow for efficient exploration of superior combinations in mere CPU minutes. Our methodology yields CIMNet, which consistently improves the trade-off between accuracy and hardware efficiency on benchmarks, providing valuable architectural insights.
- Open Access
- 17 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 11-20
Special Session - Non-Volatile Memories: Challenges and Opportunities for Embedded System Architectures with Focus on Machine Learning Applications
This paper explores the challenges and opportunities of integrating non-volatile memories (NVMs) into embedded systems for machine learning. NVMs offer advantages such as increased memory density, lower power consumption, non-volatility, and compute-in-memory capabilities. The paper focuses on integrating NVMs into embedded systems, particularly in intermittent computing, where systems operate during periods of available energy. NVM technologies bring persistence closer to the CPU core, enabling efficient designs for energy-constrained scenarios. Next, computation in resistive NVMs is explored, highlighting its potential for accelerating machine learning algorithms. However, challenges related to reliability and device non-idealities need to be addressed. The paper also discusses memory-centric machine learning, leveraging NVMs to overcome the memory wall challenge. By optimizing memory layouts and utilizing probabilistic decision tree execution and neural network sparsity, NVM-based systems can improve cache behavior and reduce unnecessary computations. In conclusion, the paper emphasizes the need for further research and optimization for the widespread adoption of NVMs in embedded systems presenting relevant challenges, especially for machine learning applications.
- 17 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 11-20
Open AccessSpecial Session - Non-Volatile Memories: Challenges and Opportunities for Embedded System Architectures with Focus on Machine Learning Applications
This paper explores the challenges and opportunities of integrating non-volatile memories (NVMs) into embedded systems for machine learning. NVMs offer advantages such as increased memory density, lower power consumption, non-volatility, and compute-in-memory capabilities. The paper focuses on integrating NVMs into embedded systems, particularly in intermittent computing, where systems operate during periods of available energy. NVM technologies bring persistence closer to the CPU core, enabling efficient designs for energy-constrained scenarios. Next, computation in resistive NVMs is explored, highlighting its potential for accelerating machine learning algorithms. However, challenges related to reliability and device non-idealities need to be addressed. The paper also discusses memory-centric machine learning, leveraging NVMs to overcome the memory wall challenge. By optimizing memory layouts and utilizing probabilistic decision tree execution and neural network sparsity, NVM-based systems can improve cache behavior and reduce unnecessary computations. In conclusion, the paper emphasizes the need for further research and optimization for the widespread adoption of NVMs in embedded systems presenting relevant challenges, especially for machine learning applications.
- 23 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems I: Regular Papers
- Vol. 70 (11) , 4380-4393
Impact of Non-Volatile Memory Cells on Spiking Neural Network Annealing Machine With In-Situ Synapse Processing
Solving constraint satisfaction problems (CSPs) is in high demand for various applications. SNN serves as a competitive annealing machine that can solve the CSP more efficiently than well-known Metropolis sampling and Hopfield networks. NVM-based crossbars with analog Integrate and Fire (IF) neurons can evolve the state of SNN to solve CSP more efficiently. However, analog computations inherently suffer from imprecisions in NVM cells, e.g., current variation, OFF-state leakage, and temperature-induced drift. We are the first to analyze the impacts of various memory technologies, including 2T-NOR, FeFET, WOx ReRAM, and HfOx ReRAM, on solving the Ising model, Sudoku, and Traveling-salesman-problem (TSP). The results show that both 2T-NOR Flash and FeFET with normalized standard deviation( ${\sigma }/{u}$ ) $< 5\%$ and ON-OFF ratio $>$ 1000 are both ideal candidates as synapse devices at room temperature, while other devices suffer from the effects of current variation and OFF-state leakage, which would require the neuron circuits to have infeasible membrane capacitance size. However, the drift of cell current and the reduction of the ON-OFF ratio drops the success rate as the temperature increases. The success rate of solving TSP drops by 60% and 90% while the temperature increases from 300K to 358K for 2T-NOR and FeFET, respectively. Throughout the simulation, we show that the transistor-based memory is suggested to be a synapse device. Yet, we also find that the tolerance of temperature is inevitable under limited capacitance. Exploration of temperature-tolerated design of circuit and memory design is still in demand for future works.
- 23 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems I: Regular Papers
- Vol. 70 (11) , 4380-4393
Impact of Non-Volatile Memory Cells on Spiking Neural Network Annealing Machine With In-Situ Synapse Processing
Solving constraint satisfaction problems (CSPs) is in high demand for various applications. SNN serves as a competitive annealing machine that can solve the CSP more efficiently than well-known Metropolis sampling and Hopfield networks. NVM-based crossbars with analog Integrate and Fire (IF) neurons can evolve the state of SNN to solve CSP more efficiently. However, analog computations inherently suffer from imprecisions in NVM cells, e.g., current variation, OFF-state leakage, and temperature-induced drift. We are the first to analyze the impacts of various memory technologies, including 2T-NOR, FeFET, WOx ReRAM, and HfOx ReRAM, on solving the Ising model, Sudoku, and Traveling-salesman-problem (TSP). The results show that both 2T-NOR Flash and FeFET with normalized standard deviation( ${\sigma }/{u}$ ) $< 5\%$ and ON-OFF ratio $>$ 1000 are both ideal candidates as synapse devices at room temperature, while other devices suffer from the effects of current variation and OFF-state leakage, which would require the neuron circuits to have infeasible membrane capacitance size. However, the drift of cell current and the reduction of the ON-OFF ratio drops the success rate as the temperature increases. The success rate of solving TSP drops by 60% and 90% while the temperature increases from 300K to 358K for 2T-NOR and FeFET, respectively. Throughout the simulation, we show that the transistor-based memory is suggested to be a synapse device. Yet, we also find that the tolerance of temperature is inevitable under limited capacitance. Exploration of temperature-tolerated design of circuit and memory design is still in demand for future works.
- 18 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Computer
- Vol. 56 (9) , 4-5
Understanding Computer Architecture Sustainability
This installment of Computer’s series highlighting the work published in IEEE Computer Society journals comes from IEEE Computer Architecture Letters.
- 18 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Computer
- Vol. 56 (9) , 4-5
Understanding Computer Architecture Sustainability
This installment of Computer’s series highlighting the work published in IEEE Computer Society journals comes from IEEE Computer Architecture Letters.