π§ππ£π±π€π : Utilizing Hyperdimensional Computing for a More Robust and Efficient Machine Learning System
Open Access
- 18 October 2022
- journal article
- research article
- Published byΒ Association for Computing Machinery (ACM)Β inΒ ACM Transactions on Embedded Computing Systems
- Vol.Β 21 Β (6) , 1-25
Abstract
Todayβs systems rely on sending all the data to the cloud and then using complex algorithms, such as Deep Neural Networks, which require billions of parameters and many hours to train a model. In contrast, the human brain can do much of this learning effortlessly. Hyperdimensional (HD) Computing aims to mimic the behavior of the human brain by utilizing high-dimensional representations. This leads to various desirable properties that other Machine Learning (ML) algorithms lack, such as robustness to noise in the system and simple, highly parallel operations. In this article, we propose π§ππ£π±π€π , a HyperDimensional Computing system that is Robust, Efficient, and Accurate. We propose a Processing-in-Memory (PIM) architecture that works in a federated learning environment with challenging communication scenarios that cause errors in the transmitted data. π§ππ£π±π€π adaptively changes the bitwidth of the model based on the signal-to-noise ratio (SNR) of the incoming sample to maintain the accuracy of the HD model while achieving significant speedup and energy efficiency. Our PIM architecture is able to achieve a speedup of 28Γ and 255Γ better energy efficiency compared to the baseline PIM architecture for Classification and achieves 32 Γ speed up and 289 Γ higher energy efficiency than the baseline architecture for Clustering. π§ππ£π±π€π is able to achieve this by relaxing hardware parameters to gain energy efficiency and speedup while introducing computational errors. We show experimentally, HD Computing is able to handle the errors without a significant drop in accuracy due to its unique robustness property. For wireless noise, we found that π§ππ£π±π€π is 48 Γ more robust to noise than other comparable ML algorithms. Our results indicate that our proposed system loses less than 1% Classification accuracy, even in scenarios with an SNR of 6.64. We additionally test the robustness of using HD Computing for Clustering applications and found that our proposed system also looses less than 1% in the mutual information score, even in scenarios with an SNR under 7 dB, which is 57 Γ more robust to noise than K-means.Keywords
Funding Information
- CRISP, one of six centers in JUMP, an SRC program sponsored by DARPA, in part by SRC-Global Research Collaboration Grant (2988.001)
- NSF (1527034, 1730158, 1826967, 1911095, and 2003277)
This publication has 29 references indexed in Scilit:
- VoiceHD: Hyperdimensional Computing for Efficient Speech Recognition2017
- BER comparison between Convolutional, Turbo, LDPC, and Polar codes2017
- Customizing Neural Networks for Efficient FPGA Implementation2017
- Exploring Hyperdimensional Associative Memory2017
- Low Power Wide Area Networks: An OverviewIEEE Communications Surveys & Tutorials, 2017
- Hyperdimensional computing with 3D VRRAM in-memory kernels: Device-architecture co-design for energy-efficient, error-resilient language recognition2016
- Hyperdimensional biosignal processing: A case study for EMG-based hand gesture recognition2016
- ISAACACM SIGARCH Computer Architecture News, 2016
- Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random VectorsCognitive Computation, 2009
- On the use of the universal Okumura-Hata propagation prediction model in rural areas2002