Eve Said Yes: AirBone Authentication for Head-Wearable Smart Voice Assistant
- 1 April 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 24 (9) , 7836-7850
Abstract
Recent advances in speech and language processing have led to the rise of smart voice services like Alexa, Google Home, and Siri. However, these advancements also increase security risks due to sophisticated voice domain attacks. Instead of relying on acoustic clues to detect replayed or synthesized speech, we utilize microphones and motion sensors in head-wearable devices to authorize legitimate users through bone-conducted vibrations, enabling multi-factor authentication (MFA) for spoken voice. Our proposed two-stage authentication system, AirBone, captures air and bone conduction (AirBone) signals and exploits two authentication factors sequentially. The first stage, called temporal consistency scoring (TCS), employs signal processing to verify the recorded AC and BC signals are concurrent and originate from the same vocalization process. Statistical tools are employed to distinguish legitimate attempts against false-triggering or acoustic attacks. The second stage leverages deep learning to verify the user’s unique bone conduction patterns in the vibration domain. Specifically, we enhance the robustness through data augmentation with constant-Q transform and adversarial training, improving the model’s ability to detect impersonation and machine-induced vibrations. Thanks to these designs, AirBone authentication offers enhanced security via MFA with no extra cost of user effort. In addition, our experimental results demonstrate a $96.3\%$ overall accuracy, robustness against AirBone noise and room impulse responses, and $0.3\%$ Equal Error Rate (EER) against acoustic and cross-domain attacks.
Keywords
All Related Versions
Funding Information
- National Science Foundation (CNS-2107057, CNS-2318664, CSR-2403249, CNS-2431596)
- National Science Foundation (2425812, 2438898, 2315613)
This publication has 32 references indexed in Scilit:
- ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speechComputer Speech & Language, 2020
- Advances in anti-spoofing: from the perspective of ASVspoof challengesAPSIPA Transactions on Signal and Information Processing, 2020
- Multi‐tier authentication schemes for fog computing: Architecture, security perspective, and challengesInternational Journal of Communication Systems, 2019
- Adversarial Discriminative Domain Adaptation2017
- Deep Residual Learning for Image Recognition2016
- On the vulnerability of speaker verification to realistic voice spoofing2015
- Multisensory speech enhancement in noisy environments using bone-conducted and air-conducted microphones2014
- Vulnerability of speaker verification systems against voice conversion spoofing attacks: The case of telephone speech2012
- Multi-sensory microphones for robust speech detection, enhancement and recognition2004
- Kerberos: an authentication service for computer networksIEEE Communications Magazine, 1994