Wenlun Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Wenlun Zhang has a total of 20 co-authors. Here are the top co-authors by number of shared publications:
- Kentaro Yoshioka
Keio University, Tokyo, Japan
6 shared publications - Shimpei Ando
Department of Electronics and Electrical Engineering, Keio University, Kanagawa, Japan
4 shared publications - Xinyu Li
School of Electronic Science and Engineering, Nanjing University, Nanjing 210023, China
3 shared publications - Yasuyuki Miyamoto
Department of Electrical and Electronic Engineering, Institute of Science Tokyo, Meguro, Tokyo 152-8550, Japan
2 shared publications - Yung-Chin Chen
Department of Electronics and Electrical Engineering, Keio University, Kanagawa, Japan
2 shared publications - Chuangen Gao
Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
1 shared publication
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 13803-13811
Routing-Stabilized Modality-Aware Quantization for Mixture-of-Experts Vision-Language Models Compression
Mixture-of-Experts (MoE) Vision-Language Models (VLMs) are powerful but extremely large and expensive. Post-Training Quantization (PTQ) offers a training-free compression pathway. However, two critical and interrelated problems undermine its effectiveness on MoE VLMs. The first is quantization-induced routing instability. Small weight perturbations alter router logits near narrow selection boundaries. This redirects tokens to suboptimal experts and cascades errors across layers. The second is modality-mismatched calibration. Uniform quantization strategies fail to account for the distributional discrepancy between vision and language tokens. They also neglect non-uniform expert activation patterns. We propose Routing-Stabilized Modality-Aware Quantization (RSMAQ), a unified PTQ framework that addresses both challenges. It does so through three synergistic components. Routing Margin Preservation (RMP) enforces logit distribution alignment via KL-divergence minimization. It also uses expert-set and boundary-expert consistency metrics to protect discrete routing topology. Modality-Specific Expert Calibration (MSEC) constructs a modality-aware Hessian matrix for second-order quantization. It jointly encodes token-expert affinity and per-modality gradient sensitivity, directing the reconstruction budget toward cross-modal critical weights. Global Expert Consensus Smoothing (GECS) derives a shared activation-smoothing matrix by aggregating quantization error across all experts. It weights this aggregation by routing probability and applies an outlier-robust multi-level quantile search. We conduct extensive experiments on Kimi-VL-Instruct, Qwen3-VL-30B-A3B-Instruct, and a LLaVA-MoE-3.6B variant. We evaluate across nine multimodal benchmarks. RSMAQ consistently outperforms state-of-the-art baselines.
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 13803-13811
Open AccessRouting-Stabilized Modality-Aware Quantization for Mixture-of-Experts Vision-Language Models Compression
Mixture-of-Experts (MoE) Vision-Language Models (VLMs) are powerful but extremely large and expensive. Post-Training Quantization (PTQ) offers a training-free compression pathway. However, two critical and interrelated problems undermine its effectiveness on MoE VLMs. The first is quantization-induced routing instability. Small weight perturbations alter router logits near narrow selection boundaries. This redirects tokens to suboptimal experts and cascades errors across layers. The second is modality-mismatched calibration. Uniform quantization strategies fail to account for the distributional discrepancy between vision and language tokens. They also neglect non-uniform expert activation patterns. We propose Routing-Stabilized Modality-Aware Quantization (RSMAQ), a unified PTQ framework that addresses both challenges. It does so through three synergistic components. Routing Margin Preservation (RMP) enforces logit distribution alignment via KL-divergence minimization. It also uses expert-set and boundary-expert consistency metrics to protect discrete routing topology. Modality-Specific Expert Calibration (MSEC) constructs a modality-aware Hessian matrix for second-order quantization. It jointly encodes token-expert affinity and per-modality gradient sensitivity, directing the reconstruction budget toward cross-modal critical weights. Global Expert Consensus Smoothing (GECS) derives a shared activation-smoothing matrix by aggregating quantization error across all experts. It weights this aggregation by routing probability and applies an outlier-robust multi-level quantile search. We conduct extensive experiments on Kimi-VL-Instruct, Qwen3-VL-30B-A3B-Instruct, and a LLaVA-MoE-3.6B variant. We evaluate across nine multimodal benchmarks. RSMAQ consistently outperforms state-of-the-art baselines.
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 2159-2167
Active Token Pruning for Efficient Multimodal Reasoning
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 2159-2167
Open AccessActive Token Pruning for Efficient Multimodal Reasoning
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems I: Regular Papers
- Vol. 73 (11) , 8057-8070
A 65-nm Compute-in-Read-Only-Memory Macro With Bidirectional Ternary Weight Storage and Sparsity-Aware Local Accumulation for 1.58-bit LLM Inference
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems I: Regular Papers
- Vol. 73 (11) , 8057-8070
A 65-nm Compute-in-Read-Only-Memory Macro With Bidirectional Ternary Weight Storage and Sparsity-Aware Local Accumulation for 1.58-bit LLM Inference
- 4 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
- Vol. 34 (5) , 1556-1566
MCRA: Multicolumn Residue Accumulation Analog Compute-in-Memory Architecture With Time-Domain M-Input ΣΔ ADC
Analog compute-in-memory (ACIM) architectures offer significant throughput and energy benefits by performing multiplication-and-accumulation (MAC) operations directly within memory arrays. However, their overall efficiency is fundamentally constrained by the high power consumption of the per-column high-resolution analog-to-digital converters (ADCs) required to support modern DNNs (e.g., transformers), many of which demand both high computational precision and large-column throughput. In conventional ADC designs, energy in the noise-limited regime scales near-exponentially, typically by $4\times $ per additional bit, making high-resolution ADCs on every column power-prohibitive. This article proposes a high-precision and power-efficient multicolumn residue accumulation (MCRA) ACIM architecture to efficiently support precision-demanding modern DNNs. Each column uses a low-resolution coarse ADC (cADC), while the per-column residuals are accumulated and further quantized by an energy-efficient time-domain multi-input incremental sigma–delta (Mi- $\Sigma \Delta $ ) fine ADC (fADC). This approach amortizes the near-exponential energy growth across columns, while exploiting the more favorable power-resolution scaling of the time-domain Mi- $\Sigma \Delta $ quantization. Postlayout simulations demonstrate a 66.2-dB signal-to-noise-and-distortion ratio (SNDR) per column at only 1/21 the energy of the baseline with per-column high-resolution ADCs, and $0.405\times $ (1/2.47) the energy of an energy-saving ADC per column, which achieves a similar SNDR. Circuit- and system-level simulations demonstrate that our MCRA CIM architecture achieves negligible accuracy degradation in precision-demanding ViT tasks while delivering high energy and area efficiency.
- 4 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
- Vol. 34 (5) , 1556-1566
MCRA: Multicolumn Residue Accumulation Analog Compute-in-Memory Architecture With Time-Domain M-Input ΣΔ ADC
Analog compute-in-memory (ACIM) architectures offer significant throughput and energy benefits by performing multiplication-and-accumulation (MAC) operations directly within memory arrays. However, their overall efficiency is fundamentally constrained by the high power consumption of the per-column high-resolution analog-to-digital converters (ADCs) required to support modern DNNs (e.g., transformers), many of which demand both high computational precision and large-column throughput. In conventional ADC designs, energy in the noise-limited regime scales near-exponentially, typically by $4\times $ per additional bit, making high-resolution ADCs on every column power-prohibitive. This article proposes a high-precision and power-efficient multicolumn residue accumulation (MCRA) ACIM architecture to efficiently support precision-demanding modern DNNs. Each column uses a low-resolution coarse ADC (cADC), while the per-column residuals are accumulated and further quantized by an energy-efficient time-domain multi-input incremental sigma–delta (Mi- $\Sigma \Delta $ ) fine ADC (fADC). This approach amortizes the near-exponential energy growth across columns, while exploiting the more favorable power-resolution scaling of the time-domain Mi- $\Sigma \Delta $ quantization. Postlayout simulations demonstrate a 66.2-dB signal-to-noise-and-distortion ratio (SNDR) per column at only 1/21 the energy of the baseline with per-column high-resolution ADCs, and $0.405\times $ (1/2.47) the energy of an energy-saving ADC per column, which achieves a similar SNDR. Circuit- and system-level simulations demonstrate that our MCRA CIM architecture achieves negligible accuracy degradation in precision-demanding ViT tasks while delivering high energy and area efficiency.
- Open Access
BitROM: Weight Reload-Free CiROM Architecture Towards Billion-Parameter 1.58-bit LLM Inference
Compute-in-Read-Only-Memory (CiROM) accelerators offer outstanding energy efficiency for CNNs by eliminating runtime weight updates. However, their scalability to Large Language Models (LLMs) is fundamentally constrained by their vast parameter sizes. Notably, LLaMA-7B - the smallest model in LLaMA series - demands more than 1,000 cm2 of silicon area even in advanced CMOS nodes. This paper presents BitROM, the first CiROM-based accelerator that overcomes this limitation through co-design with BitNet's 1.58-bit quantization model, enabling practical and efficient LLM inference at the edge. BitROM introduces three key innovations: 1) a novel Bidirectional ROM Array that stores two ternary weights per transistor; 2) a Tri-Mode Local Accumulator optimized for ternary-weight computations; and 3) an integrated Decode-Refresh (DR) eDRAM that supports on-die KV-cache management, significantly reducing external memory access during decoding. In addition, BitROM integrates LoRA-based adapters to enable efficient transfer learning across various downstream tasks. Evaluated in 65nm CMOS, BitROM achieves 20.8 TOPS/W and a bit density of 4,967 kB/mm2 - offering a 10x improvement in area efficiency over prior digital CiROM designs. Moreover, the DR eDRAM contributes to a 43.6% reduction in external DRAM access, further enhancing deployment efficiency for LLMs in edge applications.
Open AccessBitROM: Weight Reload-Free CiROM Architecture Towards Billion-Parameter 1.58-bit LLM Inference
Compute-in-Read-Only-Memory (CiROM) accelerators offer outstanding energy efficiency for CNNs by eliminating runtime weight updates. However, their scalability to Large Language Models (LLMs) is fundamentally constrained by their vast parameter sizes. Notably, LLaMA-7B - the smallest model in LLaMA series - demands more than 1,000 cm2 of silicon area even in advanced CMOS nodes. This paper presents BitROM, the first CiROM-based accelerator that overcomes this limitation through co-design with BitNet's 1.58-bit quantization model, enabling practical and efficient LLM inference at the edge. BitROM introduces three key innovations: 1) a novel Bidirectional ROM Array that stores two ternary weights per transistor; 2) a Tri-Mode Local Accumulator optimized for ternary-weight computations; and 3) an integrated Decode-Refresh (DR) eDRAM that supports on-die KV-cache management, significantly reducing external memory access during decoding. In addition, BitROM integrates LoRA-based adapters to enable efficient transfer learning across various downstream tasks. Evaluated in 65nm CMOS, BitROM achieves 20.8 TOPS/W and a bit density of 4,967 kB/mm2 - offering a 10x improvement in area efficiency over prior digital CiROM designs. Moreover, the DR eDRAM contributes to a 43.6% reduction in external DRAM access, further enhancing deployment efficiency for LLMs in edge applications.
- 5 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
- Vol. 33 (10) , 2838-2851
ASiM: Modeling and Analyzing Inference Accuracy of SRAM-Based Analog CiM Circuits
Static random-access memory (SRAM)-based analog compute-in-memory (ACiM) demonstrates promising energy efficiency for deep neural network (DNN) processing. Nevertheless, efforts to optimize efficiency frequently compromise accuracy, and this trade-off remains insufficiently studied due to the difficulty of performing full-system validation. Specifically, existing simulation tools rarely target SRAM-based ACiM and exhibit inconsistent accuracy predictions, highlighting the need for a standardized, SRAM compute-in-memory (CiM) circuit-aware evaluation methodology. This article presents ASiM, a simulation framework for evaluating inference accuracy in SRAM-based ACiM systems. ASiM captures critical effects in SRAM-based analog compute in memory systems, such as analog-to-digital converter (ADC) quantization, bit-parallel encoding, and analog noise, which must be modeled with high fidelity due to their distinct behavior in charge-domain architectures compared to other memory technologies. ASiM supports a wide range of modern DNN workloads, including CNN and Transformer-based models such as ViT, and scales to large-scale tasks like ImageNet classification. Our results indicate that bit-parallel encoding can improve energy efficiency with only modest accuracy degradation; however, even 1 LSB of analog noise can significantly impair inference performance, particularly in complex tasks such as ImageNet. To address this, we explore hybrid analog-digital execution and majority voting schemes, both of which enhance robustness without negating energy savings. ASiM bridges the gap between hardware design and inference performance, offering actionable insights for energy-efficient, high-accuracy ACiM deployment. The code is available at https://github.com/Keio-CSG/ASiM
- 5 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
- Vol. 33 (10) , 2838-2851
ASiM: Modeling and Analyzing Inference Accuracy of SRAM-Based Analog CiM Circuits
Static random-access memory (SRAM)-based analog compute-in-memory (ACiM) demonstrates promising energy efficiency for deep neural network (DNN) processing. Nevertheless, efforts to optimize efficiency frequently compromise accuracy, and this trade-off remains insufficiently studied due to the difficulty of performing full-system validation. Specifically, existing simulation tools rarely target SRAM-based ACiM and exhibit inconsistent accuracy predictions, highlighting the need for a standardized, SRAM compute-in-memory (CiM) circuit-aware evaluation methodology. This article presents ASiM, a simulation framework for evaluating inference accuracy in SRAM-based ACiM systems. ASiM captures critical effects in SRAM-based analog compute in memory systems, such as analog-to-digital converter (ADC) quantization, bit-parallel encoding, and analog noise, which must be modeled with high fidelity due to their distinct behavior in charge-domain architectures compared to other memory technologies. ASiM supports a wide range of modern DNN workloads, including CNN and Transformer-based models such as ViT, and scales to large-scale tasks like ImageNet classification. Our results indicate that bit-parallel encoding can improve energy efficiency with only modest accuracy degradation; however, even 1 LSB of analog noise can significantly impair inference performance, particularly in complex tasks such as ImageNet. To address this, we explore hybrid analog-digital execution and majority voting schemes, both of which enhance robustness without negating energy savings. ASiM bridges the gap between hardware design and inference performance, offering actionable insights for energy-efficient, high-accuracy ACiM deployment. The code is available at https://github.com/Keio-CSG/ASiM
- 20 June 2025
- Published by Springer Nature
- p. 15-26
LiSA: Leveraging Link Recommender to Attack Graph Neural Networks via Subgraph Injection
- 20 June 2025
- Published by Springer Nature
- p. 15-26
LiSA: Leveraging Link Recommender to Attack Graph Neural Networks via Subgraph Injection
- Open Access
- 27 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-9
PACiM: A Sparsity-Centric Hybrid Compute-in-Memory Architecture via Probabilistic Approximation
- 27 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-9
Open AccessPACiM: A Sparsity-Centric Hybrid Compute-in-Memory Architecture via Probabilistic Approximation
- 31 May 2019
- Published by IOP Publishing in Applied Physics Express
- Vol. 12 (6) , 065005
Performance improvement of a p-MoS2/HfS2 van der Waals heterostructure tunneling FET by UV-O3 treatment
In this study, we fabricated a p-MoS2/HfS2 van der Waals heterostructure tunneling field-effect transistor (TFET), converting the bottom layers of the HfS2 channel into an HfOx insulator for a superior semiconductor/insulator interface using UV-O3 treatment. The TFET exhibited a long steep-slope region with a subthreshold swing (SS) of approximately 100 mV/dec over three decades of channel current, along with a minimum SS of 95 mV/dec. We also implemented a low-temperature measurement and observed a clear negative differential resistance below 150 K, which could be strong evidence of band-to-band tunneling.
- 31 May 2019
- Published by IOP Publishing in Applied Physics Express
- Vol. 12 (6) , 065005
Performance improvement of a p-MoS2/HfS2 van der Waals heterostructure tunneling FET by UV-O3 treatment
In this study, we fabricated a p-MoS2/HfS2 van der Waals heterostructure tunneling field-effect transistor (TFET), converting the bottom layers of the HfS2 channel into an HfOx insulator for a superior semiconductor/insulator interface using UV-O3 treatment. The TFET exhibited a long steep-slope region with a subthreshold swing (SS) of approximately 100 mV/dec over three decades of channel current, along with a minimum SS of 95 mV/dec. We also implemented a low-temperature measurement and observed a clear negative differential resistance below 150 K, which could be strong evidence of band-to-band tunneling.
- 1 May 2019
- Published by IOP Publishing in Chinese Physics Letters
- Vol. 36 (6) , 067301
Improvement of Performance of HfS2 Transistors Using a Self-Assembled Monolayer as Gate Dielectric*
This work details a study based on HfS2 transistors utilizing an n-octadecylphosphonic acid-based self-assembled monolayer (SAM) as the gate dielectric. The fabrication of the SAM-based two-dimensional (2D) material transistor is simple and can be used to improve the quality of the interface of air-sensitive 2D materials. In comparison to HfS2 transistors utilizing a conventional Al2O3 gate insulator by atomic layer deposition, HfS2 transistors utilizing an SAM as the gate dielectric can reduce the operation region from 4V to 2V, enhance the field-effect mobility from 0.03 cm2/Vs to 0.75 cm2/Vs, improve the sub-threshold swing from 404 mV/dec to 156 mV/dec, and optimize the hysteresis to 0.03 V, thus demonstrating improved quality of the semiconductor/insulator interface.
- 1 May 2019
- Published by IOP Publishing in Chinese Physics Letters
- Vol. 36 (6) , 067301
Improvement of Performance of HfS2 Transistors Using a Self-Assembled Monolayer as Gate Dielectric*
This work details a study based on HfS2 transistors utilizing an n-octadecylphosphonic acid-based self-assembled monolayer (SAM) as the gate dielectric. The fabrication of the SAM-based two-dimensional (2D) material transistor is simple and can be used to improve the quality of the interface of air-sensitive 2D materials. In comparison to HfS2 transistors utilizing a conventional Al2O3 gate insulator by atomic layer deposition, HfS2 transistors utilizing an SAM as the gate dielectric can reduce the operation region from 4V to 2V, enhance the field-effect mobility from 0.03 cm2/Vs to 0.75 cm2/Vs, improve the sub-threshold swing from 404 mV/dec to 156 mV/dec, and optimize the hysteresis to 0.03 V, thus demonstrating improved quality of the semiconductor/insulator interface.