An Exploration of ECAPA-TDNN and x-vector Speaker Representations in Zero-shot Multi-speaker TTS
Preprint
Abstract
Zero-shot multi-speaker text-to-speech (TTS) systems rely on speaker embeddings to synthesize speech in the voice of an unseen speaker, using only a short reference utterance. While many speaker embeddings have been developed for speaker recognition, their relative effectiveness in zero-shot TTS remains underexplored. In this work, we employ a YourTTS-based TTS system to compare three different speaker encoders - YourTTS's original H/ASP encoder, x-vector embeddings, and ECAPA-TDNN embeddings - within an otherwise fixed zero-shot TTS framework. All models were trained on the same dataset of Czech read speech and evaluated on 24 out-of-domain target speakers using both subjective and objective methods. The subjective evaluation was conducted via a listening test focused on speaker similarity, while the objective evaluation measured cosine distances between speaker embeddings extracted from synthesized and real utterances. Across both evaluations, the original H/ASP encoder consistently outperformed the alternatives, with ECAPA-TDNN showing better results than x-vectors. These findings suggest that, despite the popularity of ECAPA-TDNN in speaker recognition, it does not necessarily offer improvements for speaker similarity in zero-shot TTS in this configuration. Our study highlights the importance of empirical evaluation when reusing speaker recognition embeddings in TTS and provides a framework for additional future comparisons.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Evidential-TTS: High Fidelity Zero-Shot Text-to-Speech Using Evidential Deep Learning2025
- Spontts: Modeling and Transferring Spontaneous Style for TTS2024
- ECAPA-TDNN for Multi-speaker Text-to-speech Synthesis2022
- WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech ProcessingIEEE Journal of Selected Topics in Signal Processing, 2022
- TitaNet: Neural Model for Speaker Representation with 1D Depth-Wise Separable Convolutions and Global Context2022
- Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings2020
- Generalized End-to-End Loss for Speaker Verification2018
- X-Vectors: Robust DNN Embeddings for Speaker Recognition2018
- Deep neural networks for small footprint text-dependent speaker verification2014
- Front-End Factor Analysis for Speaker VerificationIEEE Transactions on Audio, Speech, and Language Processing, 2010