Adapting Whisper for Lightweight and Efficient Automatic Speech Recognition of Children for On-device Edge Applications
Preprint
Abstract
Reliability on cloud providers for ASR inference to support child-centered voice-based applications is becoming challenging due to regulatory and privacy challenges. Motivated by a privacy-preserving design, this study aims to develop a lightweight & efficient Whisper ASR system capable of running on a Raspberry Pi. Upon evaluation of the MyST corpus and by examining various filtering strategies to fine-tune the `tiny.en' model, a Word Error Rate (WER) of 15.9% was achieved (11.8% filtered). A low-rank compression reduces the encoder size by 0.51M with 1.26x faster inference in GPU, with 11% relative WER increase. During inference on Pi, the compressed version required ~2 GFLOPS fewer computations. The RTF for both the models ranged between [0.23-0.41] for various input audio durations. Analyzing the RAM usage and CPU temperature showed that the PI was capable of handling both the tiny models, however it was noticed that small models initiated additional overhead/thermal throttling.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- Exploring discrete speech units for privacy-preserving and efficient speech recognition for school-aged and preschool childrenInternational Journal of Human-Computer Studies, 2025
- LiteASR: Efficient Automatic Speech Recognition with Low-Rank Approximation2025
- Benchmarking Children's ASR with Supervised and Self-supervised Speech Foundation Models2024
- Adaptation of Whisper models to child speech recognition2023
- Challenges remain in Building ASR for Spontaneous Preschool Children Speech in Naturalistic Educational Environments2022
- WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech ProcessingIEEE Journal of Selected Topics in Signal Processing, 2022
- End-to-end neural systems for automatic children speech recognition: An empirical studyComputer Speech & Language, 2022