Towards Neural Audio Codec Source Parsing
Authors:
- Orchid Chetia PhukanOPOrchid Chetia Phukan
- GirishGIGirish
Preprint
Abstract
A new class of audio deepfakes-codecfakes (CFs)-has recently caught attention, synthesized by Audio Language Models that leverage neural audio codecs (NACs) in the backend. In response, the community has introduced dedicated benchmarks and tailored detection strategies. As the field advances, efforts have moved beyond binary detection toward source attribution, including open-set attribution, which aims to identify the NAC responsible for generation and flag novel, unseen ones during inference. This shift toward source attribution improves forensic interpretability and accountability. However, open-set attribution remains fundamentally limited: while it can detect that a NAC is unfamiliar, it cannot characterize or identify individual unseen codecs. It treats such inputs as generic ``unknowns'', lacking insight into their internal configuration. This leads to major shortcomings: limited generalization to new NACs and inability to resolve fine-grained variations within NAC families. To address these gaps, we propose Neural Audio Codec Source Parsing (NACSP) - a paradigm shift that reframes source attribution for CFs as structured regression over generative NAC parameters such as quantizers, bandwidth, and sampling rate. We formulate NACSP as a multi-task regression task for predicting these NAC parameters and establish the first comprehensive benchmark using various state-of-the-art speech pre-trained models (PTMs). To this end, we propose HYDRA, a novel framework that leverages hyperbolic geometry to disentangle complex latent properties from PTM representations. By employing task-specific attention over multiple curvature-aware hyperbolic subspaces, HYDRA enables superior multi-task generalization. Our extensive experiments show HYDRA achieves top results on benchmark CFs datasets compared to baselines operating in Euclidean space.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Towards explainable spoofed speech attribution and detection: A probabilistic approach for characterizing speech synthesizer componentsComputer Speech & Language, 2025
- The Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake AudioIEEE Transactions on Audio, Speech and Language Processing, 2025
- ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the WildIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023
- WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech ProcessingIEEE Journal of Selected Topics in Signal Processing, 2022
- SoundStream: An End-to-End Neural Audio CodecIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021
- ASVspoof 2019: Spoofing Countermeasures for the Detection of Synthesized, Converted and Replayed SpeechIEEE Transactions on Biometrics, Behavior, and Identity Science, 2021
- ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification2020
- X-Vectors: Robust DNN Embeddings for Speaker Recognition2018
- Constant Q cepstral coefficients: A spoofing countermeasure for automatic speaker verificationComputer Speech & Language, 2017
- Combining evidences from mel cepstral, cochlear filter cepstral and instantaneous frequency features for detection of natural vs. spoofed speech2015