FPGA Implementation of Keyword Spotting System Using Depthwise Separable Binarized and Ternarized Neural Networks
Open Access
Abstract
Keyword spotting (KWS) systems are used for human–machine communications in various applications. In many cases, KWS involves a combination of wake-up-word (WUW) recognition for device activation and voice command classification tasks. These tasks present a challenge for embedded systems due to the complexity of deep learning algorithms and the need for optimized networks for each application. In this paper, we propose a depthwise separable binarized/ternarized neural network (DS-BTNN) hardware accelerator capable of performing both WUW recognition and command classification on a single device. The design achieves significant area efficiency by redundantly utilizing bitwise operators in the computation of the binarized neural network (BNN) and ternary neural network (TNN). In a complementary metal-oxide semiconductor (CMOS) 40 nm process environment, the DS-BTNN accelerator demonstrated significant efficiency. Compared with a design approach where BNN and TNN were independently developed and subsequently integrated as two separate modules into the system, our method achieved a 49.3% area reduction while yielding an area of 0.558 mm. The designed KWS system, which was implemented on a Xilinx UltraScale+ ZCU104 field-programmable gate array (FPGA) board, receives real-time data from the microphone, preprocesses them into a mel spectrogram, and uses this as input to the classifier. Depending on the order, the network operates as a BNN or a TNN for WUW recognition and command classification, respectively. Operating at 170 MHz, our system achieved 97.1% accuracy in BNN-based WUW recognition and 90.5% in TNN-based command classification.
Keywords
Funding Information
- Ministry of Trade, Industry, and Energy (MOTIE, Korea) (00144288, 00144290)
- IDEC (00144288, 00144290)
This publication has 21 references indexed in Scilit:
- An Always-On 3.8 $\mu$ J/86% CIFAR-10 Mixed-Signal Binary CNN Processor With All Memory on Chip in 28-nm CMOSIEEE Journal of Solid-State Circuits, 2018
- Low Bits: Binary Neural Network for Vad and Wakeup2018
- A 9.02mW CNN-stereo-based real-time 3D hand-gesture recognition processor for smart mobile devices2018
- FP-BNN: Binarized neural network on FPGANeurocomputing, 2018
- BRein Memory: A Single-Chip Binary/Ternary Reconfigurable in-Memory Deep Neural Network Accelerator Achieving 1.4 TOPS at 0.6 WIEEE Journal of Solid-State Circuits, 2017
- A Neuromorphic Chip Optimized for Deep Learning and CMOS Technology With Time-Domain Analog and Digital Mixed-Signal ProcessingIEEE Journal of Solid-State Circuits, 2017
- LPC and LPCC method of feature extraction in Speech Recognition System2016
- Prototype experience for MIMO BLAST over third-generation wireless systemIEEE Journal on Selected Areas in Communications, 2003
- The challenge of inverse-E: the RASTA-PLP method2002
- Perceptual linear predictive (PLP) analysis of speechThe Journal of the Acoustical Society of America, 1990