Haomin Li

Vision Transformers (ViTs) achieve state-of-the-art performance across various vision tasks but pose significant deployment challenges due to their high computational complexity and memory footprint. Post-training quantization (PTQ) offers an efficient, retraining-free compression strategy; however, existing solutions typically enforce a uniform quantization scheme (fixed bit-width and data format) across all layers, constraining the potential accuracy-efficiency trade-off. We propose LOAD-ViT , a novel PTQ framework that leverages both mixed-precision and mixed-data-format quantization to allocate resources at a fine granularity based on layer sensitivity. LOAD-ViT optimizes per-layer configurations via a calibration-based sensitivity metric and a constrained search algorithm, minimizing accuracy loss under strict hardware budgets. To support this scheme, we design a dedicated hardware accelerator featuring heterogeneous compute engines and lightweight format-conversion units, facilitating the efficient execution of mixed formats. Extensive evaluations on various ViT models and benchmarks demonstrate that LOAD-ViT achieves up to 5.3 × speedup and 4.2 × energy savings compared to state-of-the-art quantization accelerators, with negligible accuracy degradation.

Mixed-precision quantization has shown great potential in reducing memory requirements and improving inference efficiency and has received extensive attention. However, narrowing bit lengths does not always translate into significant efficiency gains for various reasons. Firstly, computation kernels often lack support for mixed-precision operations, limiting their utility in systems that rely on such kernels. Secondly, as DNN computation involves multiple stages, the precision pattern can vary significantly across these stages. This complicates the development of precision-aware optimizations for achieving end-to-end efficiency gains. Ultimately, effective precision-aware optimization necessitates additional support throughout the entire vertical stack. This article proposes Mix-PC, a framework that enables efficient mixed-precision compilation for high-performance DNN inference. On one hand, we delve into the cause of inefficiencies in the current compilation for supporting mixed-precision operators. On the other hand, we propose Mix-PC, a general-purpose compilation framework that leverages adaptive patterns for different numeric precision. It leverages adaptive patterns tailored for different numeric precision and progressive fusion strategies, enabling enhanced compatibility with diverse models. The results show that compared to the SOTA quantization efforts, Mix-PC achieves 1.70×, 1.34×, and 2.13× speedup, respectively, in BERT, ViT, and HuBERT.

The exponential scaling of Large Language Models (LLMs) has exposed a growing mismatch between computational demands and hardware efficiency. Although model compression is essential for mitigating this gap, two bottlenecks fundamentally limit its effectiveness in practice: (1) high-magnitude activation outliers that degrade the accuracy of conventional uniform quantization, and (2) dynamically varying activation sparsity, that is, difficult to exploit with rigid hardware datapaths. Existing index-based schemes, such as mixed-precision or non-uniform quantization, often incur prohibitive energy and area overheads due to the complex decoding logic required at runtime. In this article, we propose NICE, an index-assisted algorithm-hardware co-design framework that achieves high-efficiency DNN acceleration by synergistically exploiting unstructured weight sparsity and centroid-based activation indexing. The core philosophy of NICE is to reformulate computation-intensive Multiply-Accumulate (MAC) operations into a unified table-lookup (LUT) mechanism. Specifically, we design a new hardware primitive that treats static weight sparsity and dynamic activation indices as lookup coordinates, enabling the direct retrieval of pre-computed or quantized results. Building on this, we develop the NICE Architecture, a systolic array-based accelerator that introduces novel dimensions of data-and-index reuse, effectively alleviating the data-movement bottlenecks of traditional architectures. Experiments show that NICE can achieve 51.3% energy reduction and 3.83x speedup on average.

Hyper-Dimensional Computing (HDC) has emerged as a lightweight computational model, renowned for its robust and efficient learning capabilities, particularly suitable for resource-constrained hardware. As HDC often finds its application in edge devices, the associated security challenges pose a critical concern that cannot be ignored. In this work, we aim to quantitatively delve into the robustness of binary hyper-dimensional computing, which is widely recognized for its robustness. Employing the bit-flip attack as our initial focal point, we meticulously devise both an attack mechanism and a corresponding defense mechanism. Our objective is to comprehensively explore the robustness of the binary hyper-dimensional computation model, aiming to gain a deeper understanding of its security vulnerabilities and potential defenses. Specifically, we introduce a novel attack framework for HDC, named HyperAttack, which is capable of compromising a robust binary HDC model by maliciously flipping a minimal number of bits within its memory system (specifically, the DRAM) that houses the associative memory. The bit-flip operation is executed through the well-known Row Hammer attack, and HyperAttack optimizes the accuracy degradation by pinpointing the most vulnerable bits in the hyper-dimensional vectors (represented as binary vectors within the associative memory) of the HDC model. The proposed HyperAttack framework is grounded in the principles of fuzziness, seamlessly integrating dimensional ranking and feature similarity analysis within hypervectors to precisely identify the bits to be flipped. Furthermore, we have developed a defense mechanism named HyperDefense, designed to bolster the robustness of binary hyper-dimensional computational models against bit-flip attacks. This defense scheme is tailored specifically for HDC models, providing a robust safeguard against potential threats. HyperDefense operates directly on the associative memory of HDC models, strengthening their defenses. By meticulously modifying selected bits, HyperDefense maintains a high level of accuracy close to the original model, even in the face of increased bit flip rates. This defense mechanism leverages redundant dimensions as backups for critical information. Through a thorough analysis of dimension importance, HyperDefense achieves superior robustness by gracefully sacrificing non-critical dimensions, thus ensuring the model’s robustness against potential attacks.

Spiking neural networks (SNNs) are an alternative computational paradigm to artificial neural networks (ANNs) that have attracted attention due to their event-driven execution mechanisms, enabling extremely low energy consumption. However, a significant challenge and opportunity in SNNs is to optimize memory and compute costs while maintaining accuracy, thereby further reducing energy consumption. Model quantization has been proposed as a promising technique to improve the running efficiency via the number of data bits reduction. Whereas, this technique has yet to be well studied in the neuromorphic computing domain. The underlying reason is that the behaviors of SNNs are quite different from those of ANNs, making 1) the accuracy of SNNs usually sensitive to data precision, and 2) the introduction of a temporal dimension to characterize neuronal dynamics. In this paper, we present NeuronQuant, an accurate and energy-efficient quantization framework to reduce the precision of neurons while maintaining accuracy. The key insight is to design a post-training quantization method guided by the activity of neurons, efficiently reducing the bit-width of parameters based on local relationships within neurons. Additionally, a budget-aware mixed bit-width allocation strategy for the total model size enables the adaptive growth and narrowing of precision in each layer, leading to a mixed-precision quantization scheme of the desired size. Extensive evaluations demonstrate that NeuronQuant can achieve a compressed SNN with 5.2 bits on average and 1.51× power consumption reduction with a superior model accuracy which is quite impressive for SNN. Code is released at https://github.com/shieldforever/NeuronQuant.

Network sparsification serves as an effective technique to accelerate Deep Neural Network (DNN) inference. However, existing sparsification techniques often rely on structured sparsity, which yields limited benefits. This is primarily due to the significant memory and computational overhead introduced by numerous sparse storage formats during address generation and gradient updates. Additionally, many of these solutions are tailored solely for the inference phase, neglecting the crucial training phase. In this article, we introduce STCO, a novel Sparse Tensor Compilation Optimization technique that significantly enhances training efficiency through structured sparse tensor compilation. Central to STCO is the Tensorization-aware Index Entity (TIE) format, which effectively represents structured sparse tensors by eliminating redundant indices and minimizing storage overhead. The TIE format plays a pivotal role in the Address-carry flow (AC flow) pass, which optimizes the data layout at the computational graph level. This pass leverages the TIE format to enhance the efficiency of tensor representations, enabling more compact and efficient sparse tensor storage. Meanwhile, a shape inference pass utilizes the AC flow to derive optimized tensor shapes, further refining the performance of sparse tensor operations. Moreover, the Address-Carry TIE Flow dynamically tracks nonzero addresses, extending the benefits of sparse optimization to both forward and backward propagation. This seamless integration into the training pipeline enables a smooth transition to sparse tensor compilation without significant modifications to existing codebases. To further boost training performance, we implement an operator-level AC flow optimization pass tailored for structured sparse tensors. This pass generates efficient addresses, ensuring minimal computational overhead during sparse tensor operations. The flexibility of STCO allows it to be efficiently integrated into various frameworks or compilers, providing a robust solution for enhancing training efficiency with structured sparse tensors. Experiments demonstrated that STCO achieved impressive speedups of 3.64×, 5.43×, 4.89×, and 3.91× when compared to state-of-the-art sparse formats on VGG16, ResNet-18, MobileNetV1, and MobileNetV2, respectively. These findings underscore the efficiency and superiority of our proposed approach in leveraging unstructured sparsity for DNN inference acceleration.