Ning Yang

Vision Transformers (ViTs) achieve state-of-the-art performance across various vision tasks but pose significant deployment challenges due to their high computational complexity and memory footprint. Post-training quantization (PTQ) offers an efficient, retraining-free compression strategy; however, existing solutions typically enforce a uniform quantization scheme (fixed bit-width and data format) across all layers, constraining the potential accuracy-efficiency trade-off. We propose LOAD-ViT , a novel PTQ framework that leverages both mixed-precision and mixed-data-format quantization to allocate resources at a fine granularity based on layer sensitivity. LOAD-ViT optimizes per-layer configurations via a calibration-based sensitivity metric and a constrained search algorithm, minimizing accuracy loss under strict hardware budgets. To support this scheme, we design a dedicated hardware accelerator featuring heterogeneous compute engines and lightweight format-conversion units, facilitating the efficient execution of mixed formats. Extensive evaluations on various ViT models and benchmarks demonstrate that LOAD-ViT achieves up to 5.3 × speedup and 4.2 × energy savings compared to state-of-the-art quantization accelerators, with negligible accuracy degradation.

Mixture-of-Experts (MoE) large language models (LLMs) leverage dynamic routing and sparse activation to improve efficiency and scalability, achieving high performance with reduced computational cost. However, their complex architecture and large memory footprint pose significant challenges for deployment, particularly on resource-constrained hardware. Post-training quantization (PTQ) is a widely used technique to reduce model size and memory usage. Existing PTQ approaches for MoE models are predominantly layer-wise and task-agnostic, optimizing reconstruction error independently within each layer. As a result, they ignore cross-layer differences in expert importance and fail to leverage task-specific signals, causing pivotal experts to be over-compressed, rarely activated experts to be over-provisioned, and overall accuracy to degrade. To overcome these limitations, we propose ExQuant, a PTQ framework for MoE LLMs that enables global, expert-level mixed-precision quantization. ExQuant first constructs a Globally Comparable Expert Importance Metric by integrating expert routing frequency and post-ablation performance. Based on this metric, it assigns tiered bit-widths to experts and employs a precision-aware load balancing strategy to dynamically schedule computation across processing elements, fully exploiting slack between low- and high-precision workloads. Experiments demonstrate that ExQuant significantly reduces memory footprint, improves inference efficiency, and achieves 2.87 – 5.93% accuracy improvement over existing MoE quantization methods.

Mixed-precision quantization has shown great potential in reducing memory requirements and improving inference efficiency and has received extensive attention. However, narrowing bit lengths does not always translate into significant efficiency gains for various reasons. Firstly, computation kernels often lack support for mixed-precision operations, limiting their utility in systems that rely on such kernels. Secondly, as DNN computation involves multiple stages, the precision pattern can vary significantly across these stages. This complicates the development of precision-aware optimizations for achieving end-to-end efficiency gains. Ultimately, effective precision-aware optimization necessitates additional support throughout the entire vertical stack. This article proposes Mix-PC, a framework that enables efficient mixed-precision compilation for high-performance DNN inference. On one hand, we delve into the cause of inefficiencies in the current compilation for supporting mixed-precision operators. On the other hand, we propose Mix-PC, a general-purpose compilation framework that leverages adaptive patterns for different numeric precision. It leverages adaptive patterns tailored for different numeric precision and progressive fusion strategies, enabling enhanced compatibility with diverse models. The results show that compared to the SOTA quantization efforts, Mix-PC achieves 1.70×, 1.34×, and 2.13× speedup, respectively, in BERT, ViT, and HuBERT.

The exponential scaling of Large Language Models (LLMs) has exposed a growing mismatch between computational demands and hardware efficiency. Although model compression is essential for mitigating this gap, two bottlenecks fundamentally limit its effectiveness in practice: (1) high-magnitude activation outliers that degrade the accuracy of conventional uniform quantization, and (2) dynamically varying activation sparsity, that is, difficult to exploit with rigid hardware datapaths. Existing index-based schemes, such as mixed-precision or non-uniform quantization, often incur prohibitive energy and area overheads due to the complex decoding logic required at runtime. In this article, we propose NICE, an index-assisted algorithm-hardware co-design framework that achieves high-efficiency DNN acceleration by synergistically exploiting unstructured weight sparsity and centroid-based activation indexing. The core philosophy of NICE is to reformulate computation-intensive Multiply-Accumulate (MAC) operations into a unified table-lookup (LUT) mechanism. Specifically, we design a new hardware primitive that treats static weight sparsity and dynamic activation indices as lookup coordinates, enabling the direct retrieval of pre-computed or quantized results. Building on this, we develop the NICE Architecture, a systolic array-based accelerator that introduces novel dimensions of data-and-index reuse, effectively alleviating the data-movement bottlenecks of traditional architectures. Experiments show that NICE can achieve 51.3% energy reduction and 3.83x speedup on average.

Transformer-based large language models (LLMs) have achieved remarkable success. However, their growing size presents challenges due to the increasing mismatch between model scale and hardware capacity. Model compression techniques have been proposed to address this issue, but existing methods often struggle to cope with the large dynamic range of model parameters, including both activations and weights. Additionally, current outlier-aware encoding schemes introduce complex logic, which limits their effectiveness in both compression ratio and hardware efficiency. To overcome these challenges, we propose SPADE, a co-designed algorithm–architecture solution for post-training quantized INT8 large language models, introducing a hybrid-precision encoding scheme. SPADE employs variable-length data representation guided by metadata, enabling local, value-dependent encoding with only 3.66% hardware area overhead while delivering up to 4.88× speedup and 70.5% energy reduction compared with state-of-the-art encoding-based accelerators. Our key insight is that only a small portion of activations are outliers that require high-precision such as INT8, while the vast majority can be accurately represented with INT4. This allows us to exploit the sparsity of high-significance bits across the data. By assigning an identifier bit to each data element, SPADE dynamically selects between compact INT4 encoding for normal values and higher INT8 encoding for rare outliers, eliminating unnecessary bit-width and reducing data redundancy. Moreover, we analyze the limitations of existing variable-length encoding methods, which typically require serialization and deserialization due to uncertainty in the precision format of loaded data blocks. SPADE integrates precision information directly into its encoding rules, enabling parallel decoding and computation on systolic arrays. This design significantly boosts computational throughput and helps realize the theoretical performance advantages of structured encoding. We evaluate SPADE-based accelerators against state-of-the-art encoding-based accelerators, and our results demonstrate that the SPADE-based accelerator achieves up to 4.88× speedup and 70.5% energy reduction, while maintaining superior model accuracy. The codes are available at https://github.com/jlsbz/SPADE.

Hyper-Dimensional Computing (HDC) has emerged as a lightweight computational model, renowned for its robust and efficient learning capabilities, particularly suitable for resource-constrained hardware. As HDC often finds its application in edge devices, the associated security challenges pose a critical concern that cannot be ignored. In this work, we aim to quantitatively delve into the robustness of binary hyper-dimensional computing, which is widely recognized for its robustness. Employing the bit-flip attack as our initial focal point, we meticulously devise both an attack mechanism and a corresponding defense mechanism. Our objective is to comprehensively explore the robustness of the binary hyper-dimensional computation model, aiming to gain a deeper understanding of its security vulnerabilities and potential defenses. Specifically, we introduce a novel attack framework for HDC, named HyperAttack, which is capable of compromising a robust binary HDC model by maliciously flipping a minimal number of bits within its memory system (specifically, the DRAM) that houses the associative memory. The bit-flip operation is executed through the well-known Row Hammer attack, and HyperAttack optimizes the accuracy degradation by pinpointing the most vulnerable bits in the hyper-dimensional vectors (represented as binary vectors within the associative memory) of the HDC model. The proposed HyperAttack framework is grounded in the principles of fuzziness, seamlessly integrating dimensional ranking and feature similarity analysis within hypervectors to precisely identify the bits to be flipped. Furthermore, we have developed a defense mechanism named HyperDefense, designed to bolster the robustness of binary hyper-dimensional computational models against bit-flip attacks. This defense scheme is tailored specifically for HDC models, providing a robust safeguard against potential threats. HyperDefense operates directly on the associative memory of HDC models, strengthening their defenses. By meticulously modifying selected bits, HyperDefense maintains a high level of accuracy close to the original model, even in the face of increased bit flip rates. This defense mechanism leverages redundant dimensions as backups for critical information. Through a thorough analysis of dimension importance, HyperDefense achieves superior robustness by gracefully sacrificing non-critical dimensions, thus ensuring the model’s robustness against potential attacks.