Classification using Hyperdimensional Computing: A~Review~with~Comparative Analysis
Preprint
- 13 October 2023
- preprint
- Published by Springer Nature in Research Square
Abstract
Hyperdimensional computing (HD), also known as vector symbolic architectures (VSA), is an emerging and promising paradigm for cognitive computing. At its core, HD/VSA is characterized by its distinctive approach to representing information using high-dimensional random vectors. The recent surge in research within this field gains momentum from its remarkable computational efficiency and ability to excel in few-shot learning scenarios. Nonetheless, the current literature is deficient in providing a comprehensive comparative analysis of various methodologies since each method uses a different benchmark to evaluate its performance. This gap obstructs the monitoring of the field's state-of-the-art advancements and acts as a significant barrier to its overall progress. To address this gap, this review not only offers a conceptual overview of the latest literature but also introduces a comprehensive comparative study of HD/VSA classification techniques. Our exploration starts with an overview of the strategies proposed to encode information as high-dimensional vectors. These vectors serve as integral components in the construction of classification models. Furthermore, we evaluate diverse classification methods as proposed in the existing literature. This evaluation encompasses techniques such as retraining and regenerative training to augment the model's performance. To conclude our study, we present a comprehensive empirical study. This study serves as an in-depth analysis, systematically comparing various HD/VSA classification approaches using two benchmarks, the first being a set of seven popular datasets used in HD/VSA and the second consisting of 121 datasets extracted from the UCI Machine Learning repository.To facilitate future research on classification with HD/VSA, we open-sourced the benchmarking and the implementations of the methods we review. Our findings yield significant insights. Firstly, encodings based on key-value pairs emerge as optimal choices, boasting superior accuracy while maintaining high efficiency. Secondly, iterative adaptive methods demonstrate remarkable efficacy, potentially complemented by a regenerative strategy depending on the specific problem. Moreover, we show how HD/VSA is able to generalize while training with a limited number of training instances. Lastly, we demonstrate the robustness of HD/VSA methodologies by subjecting the model memory to a large number of bit-flips. Our results illustrate that the model's performance remains reasonably stable until the occurrence of 40\% of bit flips, where the model's performance is drastically degraded. Overall, we have performed a thorough performance evaluation on different methods and have observed a positive trend in both evaluation and execution time.Keywords
All Related Versions
This publication has 50 references indexed in Scilit:
- Discovering faster matrix multiplication algorithms with reinforcement learningNature, 2022
- A comparison of vector symbolic architecturesArtificial Intelligence Review, 2021
- Towards a generalized theory comprising digital, neuromorphic and unconventional computingNeuromorphic Computing and Engineering, 2021
- Task-Projected Hyperdimensional Computing for Multi-task Learning2020
- Applied Linear Algebra2018
- Advances in Neuromorphic Hardware Exploiting Emerging Nanoscale Devices2017
- Reasoning with vectors: A continuous model for fast robust inferenceLogic Journal of the IGPL, 2014
- Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random VectorsCognitive Computation, 2009
- Representation and processing of structures with binary sparse distributed codesIEEE Transactions on Knowledge and Data Engineering, 2001
- The dynamical hypothesis in cognitive scienceBehavioral and Brain Sciences, 1998