Seongjong Kim

A binary neural network (BNN) chip explores the limits of energy efficiency and computational density for an all-digital deep neural network (DNN) inference accelerator. The chip intersperses data storage and computation using computation near memory (CNM) to reduce interconnect and data movement costs. It performs wide inner product operations to leverage parallelism inherent in DNN computations. The BNN chip leverages lightweight pipelining at a near-threshold voltage (NTV) to reduce the overhead of sequential elements. It employs optimized data access patterns to reduce memory accesses for convolutional operation with pooling layers. The combination of these techniques enables the BNN chip to achieve a peak energy efficiency of 617 TOPS/W. The digital BNN chip approaches the energy efficiency of analog in-memory techniques while also ensuring deterministic, scalable, and bit-accuracy operation. Moreover, the all-digital design leverages process scaling and does not require additional memory transistors or passive devices to attain a peak compute density of 418 TOPS/mm² and a memory density of 414 KB/mm². The binary design is extended to enable bit-serial integer precision operation with a reconfigurable 1-b multiplication circuit and element-wise partial sum shift and accumulate. This technique allows for fine-grain mixed precision and retains energy efficiency by exploiting parallelism inherent in DNNs. The bit-serial binary operation allows for bit-accurate operation and high DNN accuracy that multibit analog compute-in-memory designs struggle to attain. It provides favorable energy tradeoffs compared with small-integer digital DNN accelerators.

A 10nm DNN inference accelerator compresses model size with tabulation hash-based fine-grained weight sharing and increases 8b-compute density by 3.4× to 1.6 TOPS/mm2. The compressed model DNN implements lightweight hashing circuits to compress fully connected and recurrent neural networks. Optimized shared weight address generation reduces MUX tree area overhead by 40%. Runtime hash table generation and weight mapping circuits enable a peak energy efficiency of 9.0TOPS/W at 450mV, 25 ∘C. A 128×-compressed 3-layer LSTM classifies TIMIT phonemes with 85.6% accuracy for a total energy of 14μJ/classification, with <0.5% degradation in accuracy over an uncompressed network.

A 10-nm compute-near-memory (CNM) accelerator augments SRAM with multiply accumulate (MAC) units to reduce interconnect energy and achieve 2.9 8b-TOPS/W for matrix–vector computation. The CNM provides high memory bandwidth by accessing SRAM subarrays to enable low-latency, real-time inference in fully connected and recurrent neural networks with small mini-batch sizes. For workloads with greater arithmetic intensity, such as large-batch convolutional neural networks, the CNM reconfigures into a 2-D systolic array to amortize memory access energy over a greater number of computations. Variable-precision 8b/4b/2b/1b MACs increase throughput by up to $8\times $ for binary operations at 33.0 1b-TOPS/W.

Specialized architecture combined with near- and subthreshold voltage circuits emerges as a promising candidate to improve the energy efficiency in performing complex computing kernels in a resource-constrained device. One of the critical challenges in such design is the large delay variability across process, voltage, and temperature (PVT) variations. The in situ error detection and correction (EDAC) technique can potentially handle such variations; however, since existing techniques have targeted von-Neumann architecture and nominal voltage circuits, it becomes nontrivial to apply them on near and subthreshold voltage accelerators, many of which do not base on the von-Neumann architecture. In particular, those accelerators often have no instruction, making it difficult to use the popular instruction-replay-based error correction. To tackle this challenge, in this paper, we propose a novel in situ EDAC technique that utilizes dynamic, temporarily, and spatially fine-grained body swapping for error correction without instruction replay. Using the proposed technique, we prototyped a spiking neural network (SNN) sorter in non-von-Neumann architecture. The prototyped chip can successfully remove the worst-case margin and, thus, achieve 49.3% higher energy efficiency and 35.6% higher throughput compared to the baseline that operates with the worst-case margin. The proposed technique incurs only 4.1% silicon area overhead and requires no additional supply voltage.

This paper presents an on-chip temperature sensor circuit for dynamic thermal management in VLSI systems. The sensor directly senses the threshold voltage that contains temperature information using a single PMOS device. This simple structure enables the sensor to achieve an ultra-compact footprint. The sensor also exhibits high accuracy and voltage-scalability down to 0.4 V, allowing the sensor to be used in dynamic voltage frequency scaling systems without requiring extra power distribution or regulation. The compact footprint and voltage scalability enables our proposed sensor to be implemented in a digital standard-cell format, allowing aggressive sensor placement very close to target hotspots in digital blocks. The proposed sensor frontend prototyped in a 65 nm CMOS technology has a footprint of 30.1 µm², 3σ-error of ±1.1 °C across 0 to 100 °C after one temperature point calibration, marking a significant improvement over existing sensors designed for dynamic thermal management in VLSI systems.