WZWenlun Zhang

Wenlun Zhang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Wenlun Zhang has a total of 20 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Routing-Stabilized Modality-Aware Quantization for Mixture-of-Experts Vision-Language Models Compression

    by Tianyi Yu, Wenlun Zhang

    Mixture-of-Experts (MoE) Vision-Language Models (VLMs) are powerful but extremely large and expensive. Post-Training Quantization (PTQ) offers a training-free compression pathway. However, two critical and interrelated problems undermine its effectiveness on MoE VLMs. The first is quantization-induced routing instability. Small weight perturbations alter router logits near narrow selection boundaries. This redirects tokens to suboptimal experts and cascades errors across layers. The second is modality-mismatched calibration. Uniform quantization strategies fail to account for the distributional discrepancy between vision and language tokens. They also neglect non-uniform expert activation patterns. We propose Routing-Stabilized Modality-Aware Quantization (RSMAQ), a unified PTQ framework that addresses both challenges. It does so through three synergistic components. Routing Margin Preservation (RMP) enforces logit distribution alignment via KL-divergence minimization. It also uses expert-set and boundary-expert consistency metrics to protect discrete routing topology. Modality-Specific Expert Calibration (MSEC) constructs a modality-aware Hessian matrix for second-order quantization. It jointly encodes token-expert affinity and per-modality gradient sensitivity, directing the reconstruction budget toward cross-modal critical weights. Global Expert Consensus Smoothing (GECS) derives a shared activation-smoothing matrix by aggregating quantization error across all experts. It weights this aggregation by routing probability and applies an outlier-robust multi-level quantile search. We conduct extensive experiments on Kimi-VL-Instruct, Qwen3-VL-30B-A3B-Instruct, and a LLaVA-MoE-3.6B variant. We evaluate across nine multimodal benchmarks. RSMAQ consistently outperforms state-of-the-art baselines.

  • Open Access

    Active Token Pruning for Efficient Multimodal Reasoning

    by Chuangen Gao, Shixiao Wang, Wenlun Zhang, Shuyang Gu

  • MCRA: Multicolumn Residue Accumulation Analog Compute-in-Memory Architecture With Time-Domain M-Input ΣΔ ADC

    by Xinyu Li, Wenlun Zhang, Shimpei Ando, Zhongfeng Wang, Jun Lin, Kentaro Yoshioka

    Analog compute-in-memory (ACIM) architectures offer significant throughput and energy benefits by performing multiplication-and-accumulation (MAC) operations directly within memory arrays. However, their overall efficiency is fundamentally constrained by the high power consumption of the per-column high-resolution analog-to-digital converters (ADCs) required to support modern DNNs (e.g., transformers), many of which demand both high computational precision and large-column throughput. In conventional ADC designs, energy in the noise-limited regime scales near-exponentially, typically by $4\times $ per additional bit, making high-resolution ADCs on every column power-prohibitive. This article proposes a high-precision and power-efficient multicolumn residue accumulation (MCRA) ACIM architecture to efficiently support precision-demanding modern DNNs. Each column uses a low-resolution coarse ADC (cADC), while the per-column residuals are accumulated and further quantized by an energy-efficient time-domain multi-input incremental sigma–delta (Mi- $\Sigma \Delta $ ) fine ADC (fADC). This approach amortizes the near-exponential energy growth across columns, while exploiting the more favorable power-resolution scaling of the time-domain Mi- $\Sigma \Delta $ quantization. Postlayout simulations demonstrate a 66.2-dB signal-to-noise-and-distortion ratio (SNDR) per column at only 1/21 the energy of the baseline with per-column high-resolution ADCs, and $0.405\times $ (1/2.47) the energy of an energy-saving ADC per column, which achieves a similar SNDR. Circuit- and system-level simulations demonstrate that our MCRA CIM architecture achieves negligible accuracy degradation in precision-demanding ViT tasks while delivering high energy and area efficiency.

  • Open Access

    BitROM: Weight Reload-Free CiROM Architecture Towards Billion-Parameter 1.58-bit LLM Inference

    by Wenlun Zhang, Xinyu Li, Shimpei Ando, Kentaro Yoshioka

    Compute-in-Read-Only-Memory (CiROM) accelerators offer outstanding energy efficiency for CNNs by eliminating runtime weight updates. However, their scalability to Large Language Models (LLMs) is fundamentally constrained by their vast parameter sizes. Notably, LLaMA-7B - the smallest model in LLaMA series - demands more than 1,000 cm2 of silicon area even in advanced CMOS nodes. This paper presents BitROM, the first CiROM-based accelerator that overcomes this limitation through co-design with BitNet's 1.58-bit quantization model, enabling practical and efficient LLM inference at the edge. BitROM introduces three key innovations: 1) a novel Bidirectional ROM Array that stores two ternary weights per transistor; 2) a Tri-Mode Local Accumulator optimized for ternary-weight computations; and 3) an integrated Decode-Refresh (DR) eDRAM that supports on-die KV-cache management, significantly reducing external memory access during decoding. In addition, BitROM integrates LoRA-based adapters to enable efficient transfer learning across various downstream tasks. Evaluated in 65nm CMOS, BitROM achieves 20.8 TOPS/W and a bit density of 4,967 kB/mm2 - offering a 10x improvement in area efficiency over prior digital CiROM designs. Moreover, the DR eDRAM contributes to a 43.6% reduction in external DRAM access, further enhancing deployment efficiency for LLMs in edge applications.

  • ASiM: Modeling and Analyzing Inference Accuracy of SRAM-Based Analog CiM Circuits

    by Wenlun Zhang, Shimpei Ando, Yung-Chin Chen, Kentaro Yoshioka

    Static random-access memory (SRAM)-based analog compute-in-memory (ACiM) demonstrates promising energy efficiency for deep neural network (DNN) processing. Nevertheless, efforts to optimize efficiency frequently compromise accuracy, and this trade-off remains insufficiently studied due to the difficulty of performing full-system validation. Specifically, existing simulation tools rarely target SRAM-based ACiM and exhibit inconsistent accuracy predictions, highlighting the need for a standardized, SRAM compute-in-memory (CiM) circuit-aware evaluation methodology. This article presents ASiM, a simulation framework for evaluating inference accuracy in SRAM-based ACiM systems. ASiM captures critical effects in SRAM-based analog compute in memory systems, such as analog-to-digital converter (ADC) quantization, bit-parallel encoding, and analog noise, which must be modeled with high fidelity due to their distinct behavior in charge-domain architectures compared to other memory technologies. ASiM supports a wide range of modern DNN workloads, including CNN and Transformer-based models such as ViT, and scales to large-scale tasks like ImageNet classification. Our results indicate that bit-parallel encoding can improve energy efficiency with only modest accuracy degradation; however, even 1 LSB of analog noise can significantly impair inference performance, particularly in complex tasks such as ImageNet. To address this, we explore hybrid analog-digital execution and majority voting schemes, both of which enhance robustness without negating energy savings. ASiM bridges the gap between hardware design and inference performance, offering actionable insights for energy-efficient, high-accuracy ACiM deployment. The code is available at https://github.com/Keio-CSG/ASiM


  • Open Access

    PACiM: A Sparsity-Centric Hybrid Compute-in-Memory Architecture via Probabilistic Approximation

    by Wenlun Zhang, Shimpei Ando, Yung-Chin Chen, Satomi Miyagi, Shinya Takamaeda-Yamazaki, Kentaro Yoshioka


  • Performance improvement of a p-MoS2/HfS2 van der Waals heterostructure tunneling FET by UV-O3 treatment

    by Wenlun Zhang, Toru Kanazawa, Yasuyuki Miyamoto

    In this study, we fabricated a p-MoS2/HfS2 van der Waals heterostructure tunneling field-effect transistor (TFET), converting the bottom layers of the HfS2 channel into an HfOx insulator for a superior semiconductor/insulator interface using UV-O3 treatment. The TFET exhibited a long steep-slope region with a subthreshold swing (SS) of approximately 100 mV/dec over three decades of channel current, along with a minimum SS of 95 mV/dec. We also implemented a low-temperature measurement and observed a clear negative differential resistance below 150 K, which could be strong evidence of band-to-band tunneling.


  • Improvement of Performance of HfS2 Transistors Using a Self-Assembled Monolayer as Gate Dielectric*

    by Wen-Lun Zhang

    This work details a study based on HfS2 transistors utilizing an n-octadecylphosphonic acid-based self-assembled monolayer (SAM) as the gate dielectric. The fabrication of the SAM-based two-dimensional (2D) material transistor is simple and can be used to improve the quality of the interface of air-sensitive 2D materials. In comparison to HfS2 transistors utilizing a conventional Al2O3 gate insulator by atomic layer deposition, HfS2 transistors utilizing an SAM as the gate dielectric can reduce the operation region from 4V to 2V, enhance the field-effect mobility from 0.03 cm2/Vs to 0.75 cm2/Vs, improve the sub-threshold swing from 404 mV/dec to 156 mV/dec, and optimize the hysteresis to 0.03 V, thus demonstrating improved quality of the semiconductor/insulator interface.