AdaptBit-HD: Adaptive Model Bitwidth for Hyperdimensional Computing
Authors:
- 1 October 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 93-100
Abstract
Brain-inspired Hyperdimensional (HD) computing is a novel computing paradigm emulating the neuron’s activity in high-dimensional space. The first step in HD computing is to map each data point into high-dimensional space (e.g., 10,000). This poses several problems. For instance, the size of the data can explode and all subsequent operations need to be performed in parallel in D = 10,000 dimensions. Prior work alleviated this issue with model quantization. The HVs could then be stored in less space than the original data and lower bitwidth operations can be used to save energy. However, prior work quantized all samples to the same bitwidth. We propose, AdaptBit-HD, an Adaptive Model Bitwidth Architecture for accelerating HD Computing. AdaptBit-HD operates on the bits of the quantized model one bit at a time to save energy when fewer bits can be used to find the correct class. With AdaptBit-HD, we can achieve both high accuracy by utilizing all the bits when necessary and high energy efficiency by terminating execution at lower bits when our design is confident in the output. We additionally design an endto-end FPGA accelerator for AdaptBit-HD. Compared to 16-bit models, AdaptBit-HD is 14× more energy efficient and compared to binary models, AdaptBit-HD is 1.1% more accurate, which is comparable in accuracy to 16-bit models. This demonstrates that AdaptBit-HD is able to achieve the accuracy of full precision models, with the energy efficiency of binary models.Keywords
This publication has 13 references indexed in Scilit:
- BRIC2019
- A Binary Learning Framework for Hyperdimensional Computing2019
- Weighted Quantization-Regularization in DNNs for Weight Memory Minimization Toward HW ImplementationIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2018
- Compensated-DNN: Energy Efficient Low-Precision Deep Neural Networks by Compensating Quantization Errors2018
- VoiceHD: Hyperdimensional Computing for Efficient Speech Recognition2017
- In-Datacenter Performance Analysis of a Tensor Processing Unit2017
- Exploring Hyperdimensional Associative Memory2017
- Edge-centric ComputingACM SIGCOMM Computer Communication Review, 2015
- Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random VectorsCognitive Computation, 2009
- Encoding Structure in Boolean Space1998