SpectroStream: A Versatile Neural Codec for General Audio
Authors:
- Yunpeng LiYLYunpeng Li
- Kehang HanKHKehang Han
- Brian McWilliamsBMBrian McWilliams
- Zalan BorsosZBZalan Borsos
- Marco TagliasacchiMTMarco Tagliasacchi
Preprint
Abstract
We propose SpectroStream, a full-band multi-channel neural audio codec. Successor to the well-established SoundStream, SpectroStream extends its capability beyond 24 kHz monophonic audio and enables high-quality reconstruction of 48 kHz stereo music at bit rates of 4--16 kbps. This is accomplished with a new neural architecture that leverages audio representation in the time-frequency domain, which leads to better audio quality especially at higher sample rate. The model also uses a delayed-fusion strategy to handle multi-channel audio, which is crucial in balancing per-channel acoustic quality and cross-channel phase consistency.Keywords
All Related Versions
This publication has 5 references indexed in Scilit:
- AudioLM: A Language Modeling Approach to Audio GenerationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023
- SoundStream: An End-to-End Neural Audio CodecIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021
- Wavesplit: End-to-End Speech Separation by Speaker ClusteringIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021
- Streaming Keyword Spotting on Mobile Devices2020
- ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metric2020