Hardware Optimizations of Dense Binary Hyperdimensional Computing: Rematerialization of Hypervectors, Binarized Bundling, and Combinational Associative Memory
Authors:
Open Access
- 10 October 2019
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Journal on Emerging Technologies in Computing Systems
- Vol. 15 (4) , 1-25
Abstract
Brain-inspired hyperdimensional (HD) computing models neural activity patterns of the very size of the brain’s circuits with points of a hyperdimensional space, that is, with hypervectors . Hypervectors are D -dimensional (pseudo)random vectors with independent and identically distributed (i.i.d.) components constituting ultra-wide holographic words: D =10,000 bits, for instance. At its very core, HD computing manipulates a set of seed hypervectors to build composite hypervectors representing objects of interest. It demands memory optimizations with simple operations for an efficient hardware realization. In this article, we propose hardware techniques for optimizations of HD computing, in a synthesizable open-source VHDL library, to enable co-located implementation of both learning and classification tasks on only a small portion of Xilinx UltraScale FPGAs: (1) We propose simple logical operations to rematerialize the hypervectors on the fly rather than loading them from memory. These operations massively reduce the memory footprint by directly computing the composite hypervectors whose individual seed hypervectors do not need to be stored in memory. (2) Bundling a series of hypervectors over time requires a multibit counter per every hypervector component. We instead propose a binarized back-to-back bundling without requiring any counters. This truly enables on-chip learning with minimal resources as every hypervector component remains binary over the course of training to avoid otherwise multibit components. (3) For every classification event, an associative memory is in charge of finding the closest match between a set of learned hypervectors and a query hypervector by using a distance metric. This operator is proportional to hypervector dimension ( D ), and hence may take O( D ) cycles per classification event. Accordingly, we significantly improve the throughput of classification by proposing associative memories that steadily reduce the latency of classification to the extreme of a single cycle. (4) We perform a design space exploration incorporating the proposed techniques on FPGAs for a wearable biosignal processing application as a case study. Our techniques achieve up to 2.39× area saving, or 2,337× throughput improvement. The Pareto optimal HD architecture is mapped on only 18,340 configurable logic blocks (CLBs) to learn and classify five hand gestures using four electromyography sensors.Keywords
All Related Versions
Funding Information
- H2020 (780215)
- ETH Zurich Postdoctoral Fellowship Program
This publication has 12 references indexed in Scilit:
- PULP-HD2018
- Classification and Recall With Binary Hyperdimensional Computing: Tradeoffs in Choice of Density and Mapping CharacteristicsIEEE Transactions on Neural Networks and Learning Systems, 2018
- Language Geometry Using Random Indexing2017
- Hyperdimensional computing with 3D VRRAM in-memory kernels: Device-architecture co-design for energy-efficient, error-resilient language recognition2016
- Computing with 10,000-bit words2014
- Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random VectorsCognitive Computation, 2009
- Real-time intelligent pattern recognition algorithm for surface EMG signalsBioMedical Engineering OnLine, 2007
- Binary spatter-coding of ordered K-tuples1996
- Holographic reduced representationsIEEE Transactions on Neural Networks, 1995
- Random sequence generation by cellular automataAdvances in Applied Mathematics, 1986