Streaming keyword spotting on mobile devices
Preprint
Abstract
In this work we explore the latency and accuracy of keyword spotting (KWS) models in streaming and non-streaming modes on mobile phones. NN model conversion from non-streaming mode (model receives the whole input sequence and then returns the classification result) to streaming mode (model receives portion of the input sequence and classifies it incrementally) may require manual model rewriting. We address this by designing a Tensorflow/Keras based library which allows automatic conversion of non-streaming models to streaming ones with minimum effort. With this library we benchmark multiple KWS models in both streaming and non-streaming modes on mobile phones and demonstrate different tradeoffs between latency and accuracy. We also explore novel KWS models with multi-head attention which reduce the classification error over the state-of-art by 10% on Google speech commands data sets V2. The streaming library with all experiments is open-sourced.Keywords
All Related Versions
- Version 1, 2020-05-14, ArXiv
- Version 2, 2020-07-29, ArXiv
- Published version: .
This publication has 14 references indexed in Scilit:
- Training Keyword Spotters with Limited and Synthesized Speech Data2020
- End-to-end Streaming Keyword Spotting2019
- Streaming End-to-end Speech Recognition for Mobile Devices2019
- Efficient Keyword Spotting Using Dilated Convolutions and Gating2019
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference2018
- Max-pooling loss training of long short-term memory networks for small-footprint keyword spotting2017
- Small-footprint keyword spotting using deep neural networks2014
- LSTM recurrent networks learn simple context-free and context-sensitive languagesIEEE Transactions on Neural Networks, 2001
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentencesIEEE Transactions on Acoustics, Speech, and Signal Processing, 1980