Jürgen Teich

Binary neural networks (BNNs) are known for their minimal memory requirements, making them an attractive choice for resource-constrained environments. SBNN-nps are a more recent advancement that extend the benefits of BNNs by compressing them even further, achieving sub-bit level representations to maximize efficiency. However, effectively compressing and accelerating BNNs presents challenges. In this letter, we propose a novel approach to compress BNNs using a fixed-length compression scheme that can be efficiently decoded at runtime. We then propose RISC-V extensions, implemented as a custom function unit (CFU), to decode compressed weights via a codebook stored on an FPGA on-board memory, followed by XOR and population count operations. This approach achieves a speedup of up to 2× compared to conventional BNNs deployed on the RISC-V softcore, with Significantly less accuracy degradation, and provides a foundation for exploring even higher compression configurations to improve performance further.

Deploying deep neural networks (DNNs) on microcontrollers (TinyML) is a common trend to process the increasing amount of sensor data generated at the edge, but in practice, resource and latency constraints make it difficult to find optimal DNN candidates. Neural architecture search (NAS) is an excellent approach to automate this search and can easily be combined with DNN compression techniques commonly used in TinyML. However, many NAS techniques are not only computationally expensive, especially hyperparameter optimization (HPO), but also often focus on optimizing only a single objective, e.g., maximizing accuracy, without considering additional objectives such as memory requirements or computational complexity of a DNN, which are key to making deployment at the edge feasible. In this article, we propose a novel NAS strategy for TinyML based on multi-objective Bayesian optimization (MOBOpt) and an ensemble of competing parametric policies trained using augmented random search (ARS) reinforcement learning (RL) agents. Our methodology aims at efficiently finding tradeoffs between a DNN’s predictive accuracy, memory requirements on a given target system, and computational complexity. Our experiments show that we consistently outperform existing MOBOpt approaches on different datasets and architectures such as ResNet-18 and MobileNetv3.