Hyper-Dimensional Computing Challenges and Opportunities for AI Applications
Open Access
- 16 February 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 97651-97664
Abstract
Brain-inspired architectures are gaining increased attention, especially for edge devices to perform cognitive tasks utilizing its limited energy budget and computing resources. Hyperdimensional computing (HDC) paradigm is an emerging framework inspired by an abstract representation of neuronal circuits’ attributes in the human brain. That includes a fully holographic random representation, high-dimension vectors representing data, and robustness to uncertainty. The basic HDC pipeline consists of an encoding, training and comparison stages. The encoding algorithm maps different representations of inputs into a single class and stores them in the associative memory (AM) throughout the training stage. Later, during the inference stage, the similarity is computed between the query vector, which is encoded using the same encoding model, and the stored classes in the AM. HDC has shown promising results for 1D applications using less power, and lower latency than state-of-the-art digital neural networks (DNN). While in 2D applications, convolutional neural network (CNN) still achieves higher classification accuracy at the expense of more computations. In this paper, a comprehensive study on the HDC paradigm, main algorithms, and its implementation is presented. Moreover, the main state-of-the-art HDC architectures for 1D and 2D applications are highlighted. The article also analyzes two competing paradigms, namely, HDC and CNN, in terms of accuracy, complexity, and the number of operations. The paper concluded by highlighting challenges and recommendations for future directions on the HDC framework.Keywords
Funding Information
- Khalifa University Competitive Internal Research Award (CIRA-2019-026, RC2-2018-020)
This publication has 63 references indexed in Scilit:
- Imitation of honey bees’ concept learning processes using Vector Symbolic ArchitecturesBiologically Inspired Cognitive Architectures, 2015
- Sequence Prediction With Sparse Distributed Hyperdimensional Coding Applied to the Analysis of Mobile Phone Use PatternsIEEE Transactions on Neural Networks and Learning Systems, 2015
- Encoding Sequential Information in Semantic Space Models: Comparing Holographic Reduced Representation and Random PermutationComputational Intelligence and Neuroscience, 2015
- Reasoning with vectors: A continuous model for fast robust inferenceLogic Journal of the IGPL, 2014
- Modeling Dependencies in Multiple Parallel Data Streams with Hyperdimensional ComputingIEEE Signal Processing Letters, 2014
- On Bidirectional Transitions between Localist and Distributed Representations: The Case of Common Substrings Search Using Vector Symbolic ArchitectureProcedia Computer Science, 2014
- Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random VectorsCognitive Computation, 2009
- A Hadamard matrix of order 428Journal of Combinatorial Designs, 2004
- Binding and Normalization of Binary Sparse Distributed Representations by Context-Dependent ThinningNeural Computation, 2001
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998