Kejie Huang

Deep spiking neural networks (SNNs) tend to suffer from long inference latency. While existing encoding schemes are insufficient, this work applies a bit-level loosely coupled (BLC) SNN in which output bits are generated sequentially from the current input bit. Consequently, the time steps can be further reduced with minor accuracy loss. The BLC SNN is optimized for hardware implementation and configured based on quantized artificial neural networks (ANNs). A hybrid early termination (ET) scheme is employed to skip redundant computation cycles without accuracy degradation. A pipelined digital accelerator architecture is designed to implement the BLC SNN, in which line buffers are utilized to maximize data reuse. Simulation results show that the proposed hybrid ET scheme reduces computation cycles by 28.31% on LeNet-5. Simulated in SMIC 28 nm technology, the accelerator consumes $0.10~\mu $ J/img at 500 MHz with 30.49 TSOPS/W and is adaptable to 8/4 time steps of inference.

Shifted-and-Duplicated-Kernel (SDK) mapping has emerged as an effective strategy to accelerate convolutional layers on compute-in-memory (CIM) hardware. However, existing SDK variants (e.g., VWC-SDK) merely optimize mapping for a single CIM macro, leaving inter-macro parallelism unexplored. Moreover, their mapping methodologies are still suboptimal. To address these limitations, we present TetrisG-SDK, a novel framework that employs adaptive windows to boost mapping performance. The proposed windows accommodate more input channels, increase array utilization at marginal space, and adapt to different channel depths. More importantly, TetrisG-SDK reduces compute latency by searching for optimal window configurations across multiple CIM macros with a fixed hardware budget. Besides, it incorporates grouped convolution to further decrease computing cycles while maintaining near-lossless model accuracy. In addition, TetrisG-SDK integrates a validated CIM hardware simulator to provide accurate system-/application-level estimations of latency, area and energy. Compared to the single-macro VWC-SDK, the proposed framework achieves a speed-up by 1.2×, 1.3×, and 1.3× for CNN8, GoogLeNet Inception, and DenseNet40 models, respectively. When deployed on the simulator, it reduces system-level latency and energy by 2.4× and 1.7× for CNN8, 1.3× and 1.2× for Inception, and 1.3× and 1.6× for DenseNet40, respectively. When leveraging macro-level parallelism, TetrisG-SDK reduces the Energy-Delay-Area-Product (EDAP) by 70% for CNN8, 68% for Inception, and 36% for DenseNet40 compared to its non-grouped counterpart. These results manifest that TetrisG-SDK is a promising solution to efficiently mapping convolutional layers on CIM hardware.

Non-volatile memories (NVM) based computing-in-memory (CIM) architectures suffer from device non-uniformity that causes output deviations. Charge-domain computation mitigates this issue by using capacitor-based charge accumulation but increases area overhead. Earlier approaches reuse the computation array capacitors for Analog-to-Digital Conversion (ADC), yet the conventional structure allows less than half of the capacitors to be reused, limiting both utilization efficiency and conversion precision. The reduced ADC resolution further leads to accuracy degradation in practical neural network applications. To address these challenges, we propose a robust and adaptive 1FeFET-1T-1C CIM macro that reuses nearly all array capacitors within a pipeline SAR ADC architecture, achieving high resolution while maintaining compact area efficiency and lowering latency. Moreover, a charge redistribution mechanism enables analogdomain multi-bit accumulation, eliminating the need for external adder trees and shift circuits. In addition, an adaptive row activation scheme dynamically optimizes the number of active rows based on real-time workload demands. Implemented in a SMIC 55nm PDK, the proposed macro achieves a peak energy efficiency of 97.8 TOPS/W and an area efficiency of 1.153 TOPS/mm². The efficiency is further enhanced under lightweight models, with adaptive row activation achieving up to 51.17% power reduction.

Privacy-Preserving Machine Learning (PPML) has demonstrated great potential in data-sensitive industries, driving the development of low-latency CNN architectures using Fully Homomorphic Encryption (FHE). However, existing methods encounter two primary challenges when processing large image datasets: 1) Multithreading approaches, which classify one image per thread, necessitate a large number of threads and demand significant memory and CPU resources. 2) Batch packing methods are hampered by inflated ciphertext counts and inefficient handling of manage image padding and consecutive convolutions, limiting their use in deep networks. These issues create a clear need for more resource-efficient and architecturally flexible FHE-based CNN inference. To address this, we propose PipFHE, an FHE-friendly privacy-preserving CNN inference approach based on RNS-CKKS. We leverage the batch packing method and introduce two effective padding strategies for efficient encrypted image convolution. Furthermore, we propose a Channel Merging method, which notably reduces the ciphertext numbers, enabling deep network architecture. PipFHE is also compatible with pre-trained standard model parameters, ensuring high flexibility. Evaluations on the CIFAR-10 and CIFAR-100 show that PipFHE achieves an amortized inference speedup and throughput increase of 1.35x to 1.83x compared to state-of-the-art designs on the same test platform. Moreover, PipFHE performs inference on 227 encrypted images using only 36 threads and 144 GB of memory, which is 2.8x lower than other research. While PipFHE incurs an accuracy drop of <0.9% compared to plaintext inference, this strategic trade-off delivers substantial reductions in hardware requirements and enable deep network architectures. Its significant resource efficiency and support for deep networks make PipFHE a practical solution for processing large image batches in resource-constrained, privacy-sensitive cloud environments.

Homomorphic Encryption (HE) technology allows for processing encrypted data, breaking through data isolation barriers and providing a promising solution for privacy-preserving computation. The integration of HE technology into Convolutional Neural Network (CNN) inference shows potential in addressing privacy issues in identity verification, medical imaging diagnosis, and various other applications. The CKKS HE algorithm stands out as a popular option for homomorphic CNN inference due to its capability to handle real number computations. However, challenges such as computational delays and resource overhead present significant obstacles to the practical implementation of homomorphic CNN inference, largely due to the complex nature of HE operations. In addition, current methods for speeding up homomorphic CNN inference primarily address individual images or large batches of input images, lacking a solution for efficiently processing a moderate number of input images with fast homomorphic inference capabilities. In response to these challenges, we introduce a novel leveled homomorphic CNN inference scheme aimed at reducing latency and improving throughput using the CKKS scheme. Our proposed inference strategy involves mapping multiple inputs to a set of ciphertext by exploiting the sliding window properties of convolutions to utilize CKKS's inherent Single-Instruction-Multiple-Data (SIMD) capability. To mitigate the delay associated with homomorphic CNN inference, we introduce optimization techniques, including mask-weight merging, rotation multiplexing, stride convolution segmentation, and folding rotations. The efficacy of our homomorphic inference scheme is demonstrated through evaluations carried out on the MNIST and CIFAR-10 datasets. Specifically, results from the MNIST dataset on a single CPU thread show that inference for 163 images can be completed in 10.4 seconds with an accuracy of 98.95%, which is a 6.9× throughput improvement over state-of-the-art works. Comparative analysis with existing methodologies highlights the superior performance of our proposed inference scheme in terms of latency, throughput, communication overhead, and memory utilization.

Computing-in-Memory (CIM) has emerged as a promising paradigm to address the Von-Neumann bottleneck, demonstrating significant advantages in enhancing the energy efficiency of Deep Neural Networks (DNNs). However, its reliability is challenged by device variation and Stuck-at Faults (SAFs) in Non-Volatile Memories (NVMs). Conventional quantization methods do not adequately address the device variation issue in CIM, leading to substantial degradation in the inference accuracy using CIM architectures. To address these challenges, this work proposes a hybrid optimization framework tailored for CIM architectures, combining a Variation-and-Fault-Aware Canonical Signed Digit (VFA-CSD) quantization scheme and a Progressive Mapping Approach (PMA). Device non-idealities are modeled through normalized conductance and fault matrices. VFA-CSD effectively reduces quantization error caused by variation and faults, keeping error below 0.5 Least Significant Bit (LSB) under variation and reducing error by nearly 50% compared to binary weights under combined faults. PMA further enhances inference accuracy with minimal retraining. Experimental results show that at a 10% fault rate, 4-bit models suffer less than 2.5% accuracy loss across various datasets, while 8-bit models achieve accuracy nearly identical to pretrained baselines. These results demonstrate the robustness and efficiency of the proposed scheme for reliable CIM deployment.

Epilepsy is a chronic, noncommunicable brain disorder, and sudden seizure onsets can significantly impact patients' quality of life and health. However, wearable seizure-predicting devices are still limited, partly due to the bulky size of EEG-collecting devices. To relieve the problem, we proposed a novel two-stage channel-aware Set Transformer Network that could perform seizure prediction with fewer EEG channel sensors. We also tested a seizure-independent division method which could prevent the adjacency of training and test data. Experiments were performed on the CHB-MIT dataset which includes 22 patients with 88 merged seizures. The mean sensitivity before channel selection was 76.4% with a false predicting rate (FPR) of 0.09/hour. After channel selection, dominant channels emerged in 20 out of 22 patients; the average number of channels was reduced to 2.8 from 18; and the mean sensitivity rose to 80.1% with an FPR of 0.11/hour. Furthermore, experimental results on the seizure-independent division supported our assertion that a more rigorous seizure-independent division should be used for patients with abundant EEG recordings.